Higher Speciality Training Boot Camp in Otolaryngology: A Quantitative and Qualitative Analysis of the Northern National Formative Specialty Training 3 Induction Course

Background Opportunities for new otolaryngology trainees to develop their skills as they embark on specialty training can be limited. Our facility hosted a national simulation-based boot camp for new otolaryngology trainees in the UK. This study aimed to assess the effectiveness of the boot camp in improving trainee confidence as they transitioned from core surgical training (CST) to higher specialty training (HST) in otolaryngology. Methodology We conducted a prospective study on the effectiveness of the boot camp on trainee induction. The boot camp included hands-on simulation, small group teaching and didactic lectures addressing technical skills in the fields of otology, laryngology, rhinology, facial plastics, and paediatrics, as well as non-technical skills involving human factors, simulated ward round, and cognitive simulation. The boot camp curriculum reflected the competencies expected by the Joint Committee of Surgical Training (JCST) at this level of training. Participants completed a pre- and post-course questionnaire addressing their self-confidence for the technical and non-technical skills they developed during the boot camp. All participants were invited to participate in an interview 12 months after the boot camp. Results A total of 27 new otolaryngology trainees (approximately half of all new otolaryngology trainees in the UK) participated in the boot camp. A significant increase in median confidence was observed for all technical and non-technical stations (p < 0.0001). The increase in confidence observed was similar for participants regardless of prior experience in otolaryngology. Five candidates were interviewed a year after the boot camp. Analysis of the transcripts generated distinct comments that were grouped into five key themes. Conclusions A simulation-based boot camp mapped to the JCST curriculum can increase the confidence of new otolaryngology Specialty Training 3 trainees during their transition from CST to HST. It can provide valuable and durable technical and non-technical skills to aid trainees in the clinic, theatre environment, and when on-call during their inaugural year of training.


Introduction Otolaryngology training pathway in the UK
The otolaryngology training pathway in the UK is a six-year programme. Following the completion of a twoyear Core Surgical Training (CST) programme, trainees are recruited through a highly competitive national selection process to embark on the otolaryngology Specialty Training (ST) programme. Trainees are allocated the title ST3 on entry to higher surgical training (HST) and complete training at the completion of ST8 [1].
At each stage of surgical training, trainees are expected to attain specific surgical and clinical competencies defined by the Joint Committee of Surgical Training (JCST) [2]. JCST develops and maintains standards across all surgical training programmes in the UK, within the General Medical Council's (GMC) regulatory 1 2 3 4 1 1 2 5 6 Surgical boot camps to deliver simulation-based education are increasing in popularity in the UK [9][10][11].
They are designed to be intensive, highly focused courses that integrate core theoretical and practical methods allowing participants to gain new skills and knowledge effectively and efficiently both in a short time period and in a safe learning environment. Simulation-based boot camps can help trainees develop their management skills for uncommon but critical events that trainees may otherwise not get exposed to until they are faced with a real-life emergency scenario for the first time [5,6,12,13]. Although the effectiveness of surgical boot camps has previously been assessed by pre-and post-course evaluation [10,11,14], less is known about the durability and relevance of such learning once the new surgical trainees have gained exposure during their first year of speciality training.

Formation of the ST3 otolaryngology boot camp
Following the success of the junior doctors' ENT emergency training course by Swords et al., which aimed at improving confidence and skill competency for new on-call otolaryngology doctors [13], we hypothesised that an ST3 induction boot camp would enhance the experience of CSTs and increase their confidence, professional practice, and performance as they commence otolaryngology ST. We sought to evaluate the impact on their first year of HST.

Participants
New otolaryngology ST3 trainees were recruited by national advertisement on the ENT UK website. ENT UK is the national representative body of otolaryngology surgeons.

Course design
A course curriculum was designed to reflect the ST3 competencies expected as per the JSCT guidelines. Over the two-day intensive course, regional consultants and senior specialty trainees led a total of eight technical and non-technical/human factors stations. Each station was 30 minutes long and was mapped to the course curriculum ( Table 1). 2021   Technical and non-technical stations employed actors, high-fidelity mannequins, and cadaveric models to enhance the immersive simulation experience of participants ( Figure 1). The skills gained in each station were reinforced through didactic lectures throughout the course. Real-time feedback was provided by the consultants during each station as well as at the end of each scenario with informal assessments mapped to the JCST curriculum to guide candidates.

Study design
A prospective, single-blinded study was performed to assess the impact of the boot camp using quantitative pre-and post-course evaluation. Qualitative interviews were conducted to assess the effectiveness and clinical impact of the course 12 months after completion.
Independent academic approval to carry out this study was granted by the local Director of Medical Education and the Research and Development team at Wrightington, Wigan and Leigh Foundation Trust.

Data collection
Feedback questionnaires were provided to participants both at the start and end of the course. Participants were asked to rate their confidence level for each station using a seven-point Likert scale (one -strongly disagree, seven -strongly agree). Confidence levels were assessed using two questions of each station. These confidence levels were summed to give an overall confidence rating out of 14 for each technical and nontechnical station.
All participants were invited to participate in an interview 12 months after the boot camp. These voluntary interviews were conducted in a semi-structured format with open-ended questions, followed by generic probes by a single investigator (RA). Verbal informed consent was obtained during the interview. Audio recordings of the participants were transcribed by a medical secretary. All transcriptions were rendered anonymous during this transcription process.

Course Quantitative Outcomes
Participant confidence ratings pre-and post-course for each station were compared using the Wilcoxon signed-rank test. In addition, the pre-course confidence ratings of participants with ≤18 months of prior experience in ENT compared to those with >18 months were compared using the Mann-Whitney test. Comparisons between participants with greater than and less than 18 months of prior experience in ENT were also made with this cut-off used to penalise otolaryngology hopefuls at ST3 applications by Health Education England [15].

Course Qualitative Outcomes
Grounded theory, a fundamental qualitative descriptive approach, was used to analyse the interview transcripts. Independent analysis of participant statements was undertaken to generate initial codes through careful interpretation of the transcripts. Several iterative cycles were used to develop preliminary codes using a constant comparison technique, and emerging categories from the codes were refined to identify core themes. All qualitative analysis was undertaken using Dedoose Version 8.3.17 web software.

Ethical approval
Ethical approval was not required for this study as advised by The National Research Ethics Service.

Results
The study recruited 27 doctors to the boot camp. This cohort represented half the number of all new ST3 doctors in the UK.

Quantitative analysis of pre-and post-course confidence levels
All 27 participants completed the pre-and post-course questionnaire. Candidate confidence level pre-course was highest for laryngology (median confidence rating of 11), followed by human factors and simulated ward round station (median confidence ratings of 10). Candidates rated themselves the least confident pre-course for otology, paediatrics, and cognitive simulation (all had a median confidence rating of 7).

Comparison of Pre-and Post-Course Confidence Levels
There was a significant increase in confidence ratings of all candidates' post-course for each station with an overall increase of median confidence level from 9 to 12 out of 14 (p < 0.0001) ( Figure 2). points, (p < 0.0001). This was followed closely by otology and paediatrics, both of which had a median confidence increase of 4 points (p < 0.0001). Laryngology and human factors stations showed the smallest increase in confidence ratings [mean confidence increase of 1 point; laryngology (p < 0.0001), human factors (p < 0.005)].

Relationship of Previous Experience in ENT With Pre-and Post-Course Confidence
Participants with greater previous experience in ENT (>18 months) had higher pre-course confidence for otology compared to those with ≤18 months experience (p < 0.05). However, no such significant difference was observed for all other technical and non-technical stations. No significant difference was observed in the percentage increase from pre-to post-course confidence for all technical and non-technical stations when comparing participants with prior experience of ≤18 and >18 months in ENT.

Thematic analysis of post-course interviews
In total, five of the 27 attendees of the Northern ST3 Induction Boot Camp participated in the post-course interview 12 months after the initial boot camp.
Interview transcripts were anonymised by ascribing each interviewee a unique identification label from P1-5. Analysis of these transcripts generated distinct comments that were mapped to 22 distinct codes and subsequently grouped into five major themes: (1) content and design of the stations; (2) impact on the transition from CST to ST3 in ENT; (3) development of technical and non-technical skills; (4) impact on clinical performance in ST3; and (5) recommendations for future boot camps.

Theme 1: Content and Design of the Stations
Participants felt that the curriculum for the boot camp was comprehensive and was delivered in a realistic simulated manner when compared to similar scenarios the participants had subsequently encountered during their first year of surgical training ( Table 2).

Codes n Representative participant statements
Comprehensiveness 2 "I cannot think of anything that I would take away or add to it" P4 Realism 7 "I think that the station on grommets was very realistic. There was a lot of distraction with the anaesthetist playing music and you were thrown into it, as you are at work. It is very easy to go in and do a grommet on the wrong side" P4

Theme 2: Impact on the Transition From CST to ST3 in ENT
The majority of trainees stated that the boot camp increased their confidence at work as a new ST3 trainee and addressed their anxieties and fears related to their transition from CST to ST3 (Table 3). They felt that the skills gained from the boot camp helped them transition to specialty training and become a safer and more responsible ST3 trainee. Ten individual statements were coded directly as impacting trainee transition from CST to ST training.

Codes n Representative participant statements
Confidence 10 "It really helped my confidence, especially being a new ST3 going into ENT" (P1) Competence 3 "I think that the boot camp was somewhat useful in making me more competent at an ST3 level" (P5) Responsibility 2 "The boot camp made me feel more responsible" (P5) Safety 10 "I felt that after the course I was more confident. I also felt safer which helped with my clinics and theatre performance" (P1) Anxiety 3 "The boot camp was useful in addressing my anxieties more than anything, mostly about decision-making but also regarding my clinical skill and generally stepping up" (P5)

Theme 3: Development of Technical and Non-Technical Skills
Participants reported the boot camp helped enhance their technical and non-technical skills. Specific new technical skills that they were exposed to for the first time by the boot camp were highlighted. These included facial plastics, functional endoscopic sinus surgery (FESS), paediatric bronchoscopy, and assessment of temporal anatomy. One participant felt that these increased their "learning of different subspecialties such as rhinology, head & neck and paediatrics" (P1). They also felt that the non-technical stations had been incredibly useful as they had noted the impact of non-technical factors on clinical outcomes during ST3 ( Table 4).

Codes n Representative participant statements
Exposure to new skills 10 "I think that paediatric bronchoscopy is something that not a lot of people are exposed to and it was useful to get a

Theme 4: Impact on Clinical Performance in ST3
Candidates felt that the technical and non-technical skills had a direct impact on their clinic and theatre performance during their subsequent ST3 year ( Table 5). They also felt that they were better equipped to deal with on-call emergencies after the simulation sessions in the boot camp.

Theme 5: Recommendations for Future Boot Camps
All candidates felt the boot camp would be of benefit to new otolaryngology trainees ( Table 6). Suggestions for improvement included ideas to include lectures addressing intra-and post-operative complications as well as simulations set in a clinic setting to practise common clinic scenarios for ST3.

Codes n Representative participant statements
Recommendations to colleagues 3 "I would recommend the boot camp to upcoming CST2s" (P3) Making the course mandatory 6 "I think that the boot camp should be mandatory for all trainees becoming ST3s" (P5) Recommendations of new topics for subsequent boot camps 3 "I know that clinics are relatively new to a person starting as an ST3 and I think that one or two lectures on how consultants will expect us to deal with common scenarios in clinics that we are faced with would be very helpful" (P1) Recommendations for other specialties 5 "I think it would be a very good idea for every specialty to have these types of courses" (P1)

Discussion
To our knowledge, this study is the first to assess the benefits and relevance of an ENT induction boot camp for new higher surgical trainees in the UK immediately after and one year following a boot camp using quantitative and qualitative methods, respectively.
We found that an ST3 induction boot camp is effective in achieving a statistically significant improvement in the confidence levels of new otolaryngology trainees for technical and non-technical skills expected by the JCST at the ST3 level. We observed the greatest increase in confidence for cognitive simulation, otology and paediatrics stations. The significantly lower pre-course confidence for these stations and the subsequent greatest increase from pre-to post-confidence is likely secondary to the lack of exposure to these skills during CST. For example, paediatric airway training only occurs at specialised tertiary centres within the UK and is not included in the CST curriculum, as highlighted in the participant comments that the boot camp taught them skills such as paediatric bronchoscopy. The observed increase in confidence levels suggests the inductive boot camp offered an effective introduction to these specialist areas and addressed the deficiencies in trainees' knowledge.
In contrast, we observed the smallest increase in confidence for laryngology and human factors stations. Trainees have greater exposure to laryngology during CST, and therefore, it is unsurprising that trainees had a higher pre-course confidence level in this area. Despite this, we observed a small but statistically significant increase in confidence levels. This suggests that such stations provide a useful opportunity to revisit and consolidate pre-existing knowledge and skills.
The boot camp aimed to accelerate learning by creating high-fidelity stations using computer-aided simulations and mannequins in the technical stations, as well as the use of actors to role play theatre personnel in the non-technical stations, to achieve high equipment, environment, and psychological fidelity. High-fidelity simulations have been shown to be useful for practising technical skills and management of specific intra-operative scenarios [16]. The value of high-fidelity simulations is reinforced by effectively recreating a surgical operating environment while being financially and logistically feasible.
By conducting interviews one year following the boot camp, we were able to assess the effect the boot camp had on trainees' clinical practice. Our qualitative analysis found that the boot camp had a positive impact on trainees' transition from CST and ENT, as trainees reported a direct impact on clinical, theatre, and on-call performance.
Despite a small increase in confidence levels immediately following the boot camp, participants reported a year on that they found the non-technical or human factors stations useful and had experienced first-hand how important such skills were in theatre during ST3 training, suggesting it still has a positive impact on clinical practice. This is encouraging as poor communication and deficiencies in non-technical skills have been linked to a large proportion of fatal medical accidents, near misses in practice, and malpractice claims [17,18]. The importance of non-technical skills and surgical safety is now recognised and is reflected by the dissemination of non-technical skills for surgeons courses, first developed by the Royal College of Surgeons of Edinburgh in 2006 [19].
However, there is much debate surrounding the relationship between confidence and skill, with an increase in confidence not necessarily translating into an increase in competence [20]. Some studies in the literature have proposed that confidence levels in practising doctors may not be proportional to their skill set, whereas others have identified self-confidence as being important in acquiring skills and improving performance [21,22]. Other boot camps that have analysed the change in confidence and objective improvement in knowledge or skills have tended to show a similar increase in both parameters [23]. We did not investigate whether the increased level of participant-reported confidence impacted patient outcomes. However, McGaghie et al. reported that simulation-based education can improve patient safety and contribute to patient care [7].
The effectiveness of a surgical boot camp compared to no intervention has been demonstrated in randomised control trials [24,25]. However, there are no studies comparing the effectiveness of boot camps versus other teaching courses. Few studies reviewed the benefit of simulation-based education compared to other traditional medical education in medical students. In this cohort, the benefit of simulation on knowledge and skill acquisition compared to other teaching methods, including case-based discussion and apprenticeship, is inconclusive but has been shown to improve participant confidence [26][27][28]. However, there is a lack of data on surgical trainees and outcomes from undergraduate education cannot be assumed to reflect postgraduate training.

Limitations
Our main limitation is the lack of knowledge and skill-based assessment before and after the boot camp. This was intentional to encourage participant engagement without fear of judgement by consultants or peers they will encounter in HST. However, it limited our ability to gauge improvement in competence and skill objectively. The study did not have a comparative control group and the sample size for interviews was small. With interviews conducted one year after the boot camp, it is possible that recall bias would have affected the accuracy of the results. Further analysis with a larger sample size at a chosen set point following the course is required to better understand the effects of boot camps on otolaryngology training.

Conclusions
Our study suggests that an induction boot camp course, mapped to the JCST curriculum and delivered in intensive small group simulation sessions, can increase the confidence of new otolaryngology ST3 trainees in their transition from CST to HST. It can provide valuable and durable technical and non-technical skills to aid trainees in the clinic, theatre environment, and when on-call during their inaugural year of training. The integration of simulation-based induction boot camps into postgraduate training may further support and enhance trainees transitioning from CST to HST in ENT and other surgical specialties within the UK.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. The National Research Ethics Service issued approval NA. Ethical approval was not required for this study as advised by The National Research Ethics Service. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.