Impact of Simulation Training on Undergraduate Clinical Decision-making in Emergencies: A Non-blinded, Single-centre, Randomised Pilot Study

Introduction There is an increasing evidence base for the use of simulation-based medical education. Simulation is superior to more didactic methods for teaching a range of technical and non-technical skills, and students report they often derive more educational value from it compared with other teaching methods. There is currently limited evidence that simulation training improves clinical decision-making and, therefore, this pilot study sought to explore this further. Methods Students were randomised to take part in either five classroom tutorials and simulation training sessions or five classroom tutorials and an online e-learning module. On completion of the teaching, students all undertook an unseen assessment scenario (managing a simulated patient with anaphylaxis), where they were scored using a weighted marking scheme. The time taken to make decisions and student-reported confidence in decisions were also measured. Results 14/14 simulation-group participants and 12/14 e-learning-group participants completed the post-learning assessment. The simulation group identified anaphylaxis and gave adrenaline more quickly (p 0.008 and 0.005, respectively), and this cohort was more confident in making the diagnosis (p 0.044). There was no statistically significant difference between weighted global assessment scores for each group (p 0.705). The e-learning group called for help more quickly (p.0.049), although fewer students in this group called for help (five vs. nine). There was no statistical difference in confidence in decisions to administer adrenaline or call for help (p 0.539 and 0.364 respectively). Conclusions Participants who undertook simulation training were able to more confidently and quickly identify the diagnosis and initiate emergency treatment. However, there was not a statistically significant difference between groups using an overall weighted score. Using simulation to train students to perform better in emergencies and improve their decision-making shows promise but a further quantitative study is required.


Introduction
Medical students in the United Kingdom (UK) often feel under-prepared for entering clinical practice as a foundation doctor, often citing a lack of practical clinical experience and training in managing uncertainty as reasons for this. Particular domains in which they lack confidence include managing emergency situations and clinical decision-making [1][2][3]. The use of simulation training shows promise for helping to improve these skills in medical undergraduates, as it provides learners with the opportunity to experience realistic clinical scenarios, to make decisions, and face uncertainty [4]. Simulation training is increasingly being incorporated into undergraduate curricula, and there is a growing evidence base that it is beneficial for learners [5]. The United Kingdom General Medical Council, in its standards for medical education and training, states that "Learners must have access to technology enhanced and simulation-based learning opportunities within their training programme…" [6]. Some of the key mechanisms by which simulation is thought to support effective learning and, in particular, the development of prioritisation and decision-making skills, include repetitive practice, provision of personalised feedback, and the option of varying difficulty [7]. Improved clinical decision-making is arguably a more patient-centred outcome than considering improved student confidence, which has been more extensively studied, although it is more complex to objectively measure the efficacy of decision-making or prioritisation.
There is a well-developed evidence base, particularly from post-graduate studies, that simulation training is superior to traditional educational methods (such as lectures) for training specific procedural skills such as laparoscopic surgical procedures [5,[8][9]. Additionally, medical students consistently report that they find simulation training to be educationally valuable, and it improves their confidence in challenging and uncertain situations [10][11]. This specifically includes improving confidence in making decisions in emergency situations [12][13].
At present, however, there is limited evidence that simulation training actually leads to better clinical decision-making in emergency situations, and there is not necessarily a consensus on what comprises 'better' decision-making. Studies have demonstrated improved performance by undergraduate medical students in Objective Structured Clinical Examination (OSCE) assessments and quicker and more accurate decisions by resuscitation team leaders in a simulated setting following simulation training [14][15]. The present study sought to build on these findings, by using a global weighted marking scheme to attempt to quantify students' overall decision-making and by comparing simulation training to an online learning (elearning) package. The use of e-learning as a comparison was selected, as this modality is now frequently being used in undergraduate medical education, with some of its advantages reported to be that it can deliver a standardised experience to every student while being flexible in terms of how and when it is engaged with [16]. Moreover, the use of e-learning instead of classroom-based teaching potentially represents a cost-saving of up to 50%. As such a finding that e-learning is non-inferior to simulation training could be particularly significant for those involved in curriculum design and budget management [17].

Objective
This study aimed to determine whether undergraduate medical students participating in simulation and classroom tutorial teaching make clinical decisions that are quicker, more accurate, and in which they are more confident than students who receive classroom teaching and an e-learning module.

Materials And Methods
Third-year undergraduate medical students from a UK medical school were invited to participate in the study at one hospital site. All students within this group were eligible to participate. Recruitment ran from September 2018 until March 2019. The sample was convenience-based, as the researchers were already working at the study hospital site and used an established teaching programme as part of the research. All students reported they had limited simulation exposure (less than five sessions) and training on the 'Airway, Breathing, Circulation, Disability, Exposure' (ABCDE) approach to assessing and treating an unwell patient at the time of enrolment [18]. Twenty-eight (28) students volunteered and were randomised to take part in either five weekly classroom tutorials and simulation training sessions or the same classroom tutorials with five online e-learning sessions. In two cohorts of 14 students, each student was assigned a number between one and 14 in order of enrolment. These numbers were then drawn from a hat and alternately allocated to each study group to ensure equal group sizes for the teaching sessions.
For the simulation group students, sessions were delivered as a one-hour classroom tutorial followed by a one-hour simulation session, during which each student led on a simulated scenario assessing and managing an unwell patient. In the e-learning group, students completed the same one-hour classroom tutorials (separately from the other group) and then completed the associated e-learning (roughly one hour of material provided for each session) in their own time each week, with all students self-reporting completion of the e-learning. Over the five weeks, students covered a different letter of the ABCDE approach each week ('Airway' on week one, 'Breathing' on week two, and so on), 'Disability' and 'Exposure' were covered together on week four, and week five was a summary session, incorporating all aspects of the ABCDE approach. Each week involved the consolidation of learning and concepts from the prior week(s).
Two separate cohorts of 14 students, with seven students in each group, were recruited. Ethical approval for this study was received through both the Anglia Ruskin University research ethics committee and local hospital research and development department.
The teaching on the ABCDE approach to assessing and treating an acutely unwell patient was based upon guidance from the UK Resuscitation Council (RCUK). Students were taught by the same tutors (who had formal training in both simulation and education) and using identical lesson plans, although they were taught separately. The e-learning materials were designed by the same tutors and were a collection of single best-answer questions with detailed feedback.
Once students had completed all the teaching, they individually participated in a sevenminute-long, unseen simulated assessment scenario, with two assessors present. All students completed the same scenario, which was based upon the assessment and treatment of a patient with anaphylaxis. Anaphylaxis was chosen as the assessment topic, as there are both specific signs and symptoms of this condition and specific treatments that must be given that represent an effective response. The essential treatments include the administration of intramuscular adrenaline, steroids, chlorphenamine, intravenous fluids, and supplemental oxygen. Assessors followed a structured script and the scenario followed a fixed progression; see Table 1 below.

Assessment scenario script
Candidate brief: You are called to assess 26-year-old George, who is complaining of feeling generally unwell, with an itchy chest. He has just had his lunchtime dose of co-amoxiclav for community-acquired pneumonia.   The weighting for each domain was agreed by expert consensus. Bold points reflect 'key' decisions and non-bold points can only be scored if the preceding, overarching bold decision has been made. Points were only given for hydrocortisone or chlorphenamine if students gave these at the correct dose and/or route, hence the dash next to these bold points.
The marking scheme was based upon the Resuscitation Council (UK) (RCUK) anaphylaxis algorithm (Figure 1). Source: Reference [20] During the scenario, the timing of key student decisions was recorded, and immediately on completion of the scenario, students were asked to rate their confidence on a 10-point scale in various decisions they made, with their confidence rating reflecting an overall impression of how timely decisions were made and how confident they were with the doses and routes of medications, for example.

Results
Twenty-six (26) out of 28 students completed the assessment scenario: 14/14 in the simulation group and 12/14 in the e-learning group (two missed the assessment due to sickness). The primary outcome measure was the difference in the total weighted score for the assessment. These data were not normally distributed so an independent samples Mann-Whitney U test was used for analysis. The simulation group (n = 14) had a mean score of 20.86/32 and the elearning group (n=12) had a mean score of 19.58/32; there was no significant difference between the two groups, p 0.705.
Secondary outcome measures included time to administer adrenaline, time to identify anaphylaxis, time to call for help, and participant subjective confidence in these decisions. Table 3 shows the primary and secondary outcome measure results. An independent samples Mann-Whitney U test was used to analyse non-parametric data, while an independent samples t-test was used for parametric data. Figure 2 also graphically represents the key outcome measures of time to identify anaphylaxis, administer adrenaline, and call for help for both groups.

Graph (B) represents the average time for the three key outcome decisions and is separated by intervention (simulation) and control (e-learning) group
The error bars on both graphs represent the 95% confidence interval.

Discussion
The findings that students who completed simulation training made more rapid decisions, and were more confident in these decisions, are consistent with evidence from previous studies. This supports the idea that simulation training is beneficial for developing a wide range of clinical and non-clinical skills [4][5]8]. There are a number of potential reasons why simulation training leads to better performance, and there is not necessarily a consensus in the current literature although there are recurrent themes. Compared with other educational methods, students often receive more personalised feedback and they frequently benefit from a formal scenario 'debrief,' which is immediate and structured feedback directly relevant to the learning event, which further supports learning and development [21]. Furthermore, students participating in the simulation have the opportunity to make mistakes in a safe and supported environment, and through this experiential learning process, may subsequently perform better and make fewer mistakes in similar scenarios in future [21]. There is also likely to be a benefit from the experience of repetitively practising the ABCDE approach to an acutely unwell patient in simulation training, combined with improved familiarity with key resuscitation equipment as a result of this and familiarity with the structure and progression of a simulated (or real) patient assessment. This may mean students are able to focus less on the 'process' of assessing a simulated patient and are able to devote more attention to making decisions pertaining to the prompt diagnosis and initiation of key investigations and treatments. While students completing classroom tutorials and e-learning are still going through a repetition of relevant theory, they lack exposure to the authentic practice of working in a team, making clinical decisions, and prioritising clinical tasks. This may, in part, account for the improved performance of students who have completed simulation training since supervised practice is likely an important mechanism by which students learn and improve their future practice [7].
The pragmatic use of e-learning for our control group of learners is reflective of the increased use of this teaching modality in medical schools, in place of lectures or other didactic methods that are becoming less popular in modern education [22]. Simulation is costly and requires expensive simulation equipment and facilities, as well as a high instructor to students ratio. As such, if an e-learning package is almost as effective as delivering simulation training, there is a question as to whether, overall, it is the most efficient for educational institutions to use elearning preferentially.
While the overall weighted score was used as the primary outcome measure, the time to administer adrenaline is arguably an equally important outcome measure, as evidence suggests that in iatrogenic cases of anaphylaxis, the median time to cardiac or respiratory arrest is five minutes, so administering adrenaline more quickly can improve patient outcomes [23]. The time to administer adrenaline may, therefore, better represent a student's ability to prioritise and make an effective clinical decision in this instance, recognising it is the critical intervention in treating anaphylaxis. The use of a weighted binary mark scheme in this study did not account for the order in which students made decisions nor how quickly they were made. For this reason, the finding that students who had received simulation training made this critical decision significantly more quickly is potentially more important than the finding that there was no difference in overall weighted total scores.
The fact that no significant difference was identified between the groups in the context of the overall weighted score may be due to a number of factors, the most obvious of which is the small sample size. The marking scheme itself was based on expert consensus of what decisions and interventions are most important when managing anaphylaxis, but the relative weighting of each decision remains subjective and open to debate. The advantage of using a marking scheme where points are awarded for completing a given task is that it quantifies performance.
Higher scores indicate that students have completed more key interventions (all scoring domains represented 'key' decisions only), but it does not reflect the timeliness, nor the order, of those interventions. It could be argued that the timeliness or order of decisions is less important in the context of a brief clinical scenario, and the number of accurate decisions in this time better represents 'good' decision-making, but, nonetheless, in a true emergency, such as anaphylaxis, the time to giving adrenaline remains critical.

Strengths and limitations
There are several limitations to this study. There was no baseline testing of students to identify significant differences between candidates pre-intervention and to compare the relative improvement in individual students' scores following the teaching intervention. To mitigate for the potential differences in prior knowledge, care was taken to ensure all students had been taught about anaphylaxis during the educational component of the study through the tutorials, simulation sessions, and e-learning package. We were unable to control for any separate learning that took place outside of the study. Regarding the marking scheme, students were only scored for making decisions that were verbalised and because students were not prompted or asked clarifying questions, it is possible that students may have missed out on marks due to not verbalising their thoughts.
Undoubtedly, the simulation-based assessment method that was used may have benefitted the simulation group of learners due to familiarity. The method was used primarily for its utility in testing authentic decision-making in the context of an emergency scenario, where noncognitive skills are important [24]. To mitigate this effect, the simulation-based teaching was distinctly different from the assessment scenario, where students participated in teams rather than as individuals, as was required by the assessment.
Researcher bias was a risk, as the two researchers acted as both tutors during the educational component and assessors in the test scenario. This was mitigated to some degree by the use of a structured marking scheme and a formatted script for the scenario progression. Given the obvious difference in teaching methods used for each study group, it was not possible to blind participants or researchers.
The sample size and use of a single-site with medical students from a single medical school limit the generalisability of the findings, but data from this pilot can inform a power calculation for a larger-scale study in the future.

Conclusions
There are already established benefits for using simulation training in a variety of contexts, but its specific role in improving decision-making and emergency care skills is not proven. It is likely that simulation training can be as good as, if not superior to, classroom-based teaching or e-learning for training these skills, although further quantitative study on a larger scale is required to provide more conclusive data. It remains to be seen whether the improved confidence students report as a result of simulation training persists after a significant period of time or if they do not participate in regular simulation, as repeated practice is important for the retention of skills and knowledge. The use of weighted decision-making marking schemes needs to be studied further, to identify if they can reliably score the effectiveness of decisionmaking and differentiate between different groups who have been exposed to different educational methods. In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.