Effectiveness of High-Fidelity Simulation in Training Emergency Medicine Physicians in Point of Care Ultrasonography in Pakistan: A Quasi-Experimental Study

Background Point-of-care ultrasound (PoCUS) is frequently utilized in emergency medicine (EM), with an extended-focused assessment with sonography in trauma (e-FAST) being the most widely used PoCUS modality. This modality is not only time- and cost-efficient, but it is highly accurate in the diagnosis and management of surgical patients in the emergency department, as well as being highly predictive of patient outcomes. Targeted training is essential to ensure a learner's confidence in image acquisition, interpretation, and translation of knowledge to making clinical decisions. High-fidelity simulation offers a uniquely safe and "mistake-forgiving" environment to teach and train medical professionals. The present study evaluated the effectiveness of a high-fidelity simulator to train EM physicians in e-FAST at a tertiary care teaching hospital in a lower-middle-income country. Methods This quasi-experimental study was performed at a state-of-the-art simulation center of a multidisciplinary university hospital in Karachi, Pakistan. Subjects were included if they were EM physicians who volunteered to participate and were available for the entire training and testing period. The educational intervention included lectures and hands-on practice on a high-fidelity simulator (SonoSim, Santa Monica, CA). Knowledge and image interpretation on e-FAST were evaluated using a questionnaire, administered before and after the training course. Each participant's ability to acquire and interpret satisfactory images was assessed by experienced EM physicians and recorded. Participants were also administered a needs assessment survey and a course evaluation. Data were analyzed using IBM SPSS Statistics for Windows, Version 20.0 (Armonk, NY: IBM Corp.). All the tests were two-sided, and p-values ≤0.05 were considered significant. Baseline characteristics and outcome variables were recorded and compared by Wilcoxon signed-rank tests. Results A total of 31 EM physicians, 12 (38.7%) men and 19 (61.3%) women, were enrolled in the study, with 24 (77.3%) having one to three years of EM experience. Mean and percentage group performance improved from 6 and 40% before the intervention to 14.5 and 96.6% after the intervention (Z=4.867, p≤0.05). Most improvement in image acquisition on high-fidelity simulation was observed in the upper right quadrant of the suprapubic window (29/31; 93.5%), followed by the upper left quadrant (27/31; 87%) and the subxiphoid window (21/31; 67%). All 31 participants reported improvements in comfort and confidence level with PoCUS after attending the workshop. Conclusions EM physicians who attended a brief workshop incorporating simulation demonstrated improvements in knowledge and image acquisition skills in all domains tested. High-fidelity simulation training is an effective modality for training EM physicians in e-FAST.


Introduction
Point-of-care ultrasound (PoCUS) is defined as ultrasonography at the patient's bedside performed in real time by a care provider [1]. Focused assessment with sonography for trauma (FAST) is an integrated, goal-directed, bedside examination performed to detect fluid, which is likely to be hemorrhage in cases of trauma [2]. In addition to pericardial and peritoneal windows, extended FAST (e-FAST) also includes an examination of the chest. Studies from the developed world have shown that e-FAST is an effective and sensitive technique for the detection of blunt abdominal trauma and that it is equally accurate when administered by radiologists and non-radiologists [3].
Similar to other aspects of physical examinations, PoCUS is dependent on the operator [4]. Confidence and competence in both image acquisition and interpretation are essential, as is knowledge of how to incorporate findings into clinical decisions [5]. Because optimal training of physicians is necessary, many institutions worldwide include PoCUS in their undergraduate and postgraduate training and curricula [6]. Little is known, however, about the ability of physicians to acquire e-FAST skills using high-fidelity simulators to diagnose life-threatening conditions in trauma patients [7,8]. Moreover, the feasibility and effectiveness of this program in a lower-middle-income country (LMIC) like Pakistan have not been determined [2,3]. The present study assessed the knowledge and skills of emergency medicine (EM) physicians after a brief training workshop on e-FAST, including high-fidelity simulators, in a tertiary care teaching hospital in Karachi, Pakistan.

Materials And Methods
This quasi-experimental study enrolled EM physicians, including postgraduate medical trainees (residents) and non-trainees (medical officers and senior medical officers), working in an urban tertiary care university hospital. The study was approved by the ethical review committee of Aga Khan University and was conducted in a simulation center located in a multidisciplinary university hospital in Karachi, Pakistan. All EM physicians at the hospital were invited to participate, with all those agreeing and enrolled providing written informed consent.
Participants were eligible if they were EM physicians who volunteered to participate, and were available for the entire training and testing period. Each physician participated in a half-day (3.5-hour) workshop on PoCUS-eFAST, which included didactic lectures on knobology, knowledge of image acquisition, and knowledge of image interpretation, followed by a handson practice session and then assessment on a high-fidelity simulator (SonoSim, Santa Monica, CA) and by a human volunteer ( Figure 1). Knowledge regarding image acquisition and interpretation skills was evaluated by a questionnaire and by experienced EM physicians. The preintervention questionnaire included questions about demographic characteristics, need assessments, cognitive skills and interpretation of e-FAST images, and barriers to PoCUS practice. After the training session, the participants were administered the same questionnaire, which included questions about subjects' knowledge and feedback on the workshop. To assess changes in knowledge, each trainee's results before and after the training session were compared. Data were compiled anonymously by a blinded reviewer and analyzed statistically using IBM SPSS Statistics for Windows, Version 20.0 (Armonk, NY: IBM Corp.).
All statistical analyses were two-sided, with p-values ≤0.05 considered statistically significant. Normally distributed continuous variables were reported as mean ± standard deviation, and skewed variables were reported as median and range. Results obtained before and after the training session were compared by the Wilcoxon signed-rank test. Generalized estimation equation analysis was used to determine factors influencing the variability in scores before and after the intervention.

Results
The study enrolled 31 qualified medical practitioners, including 12 (38.7%) men and 19 (61.3%) women, with 24 (77.4%) having one to three years of work experience. Eight (27.6%) were residents, 11 (38%) were medical officers, and 10 (34.5%) were senior medical officers. Only eight (26%) participants had previously received formal ultrasound training. Although 29 (93.6%) reported that an ultrasound machine was easily available in the emergency department, only three (9.86%) were using it routinely for FAST examinations. A total of 24 participants (77.4%) identified a lack of training, and 18 (58.1%) identified a lack of knowledge as the most important barriers to PoCUS utilization. Following training, all 31 (100%) selfreported improvement in levels of comfort and confidence with PoCUS, with 20 (64%) stating that ultrasound teaching sessions should take place on monthly ( Table 1).

TABLE 1: Baseline characteristics of the study participants (N=31)
A comparison of scores before and after training showed significant improvements ( Table 2), with mean group performance improving from 6 to 14.5, and percentage improving from 40% to 96.6% (Z=4.867, p<0.05 by Wilcoxon signed-rank tests; Table 3).   Generalized estimation equation analysis of factors influencing the variability in scores before and after intervention showed that high scores were more likely to be achieved by participants in their mid-level career stage, with work experience of four to five years (Table 4). Also, those identifying lack of training as the most significant barrier to PoCUS utilizations scored higher on postintervention tests (  Physicians were inquired about the needs assessment of PoCUS in emergency department ( Table 5).  Participants were tested on four trauma scenarios using a high-fidelity simulator (SonoSim). Images of the right upper quadrant, left upper quadrant, and subxiphoid and suprapubic windows were accurately acquired by 29 (93.5%), 27 (87%), 27 (87%), and 29 (93.5%) participants, respectively, whereas images of these windows were correctly interpreted by 24 (77.4%), 27 (87%), 19 (61.3%), and 20 (64.5%) subjects, respectively ( Figure 2).

Discussion
This study showed that a high-fidelity simulation-based workshop improved the knowledge and skills of emergency physicians performing e-FAST. Learners displayed a significant improvement in written post-training scores that assessed knowledge and image interpretation abilities. Although subjects participated in a half-day workshop focused on e-FAST, their post-training test scores were comparable to those of a three-day PoCUS workshop [9]. This is important for educators when planning skill-based simulation courses, as even a short course improved knowledge and interpretation of images by EM physicians. Secondary generalized estimation equation analysis showed that physicians with four to five years of experience did better on test scores than those with more than five years of experience, confirming earlier findings [9].
All 31 participants reported subjective improvements in comfort and confidence level regarding the use of eFAST at the bedside, comparable to previous results [10]. A positive correlation has been observed between confidence and knowledge [11]. Moreover, both can decay after initial training and can be regained after refresher courses [12].
Pre-training needs assessment found that, despite the availability of an ultrasound machine, it was mostly used for central venous line access and fluid assessment only, whereas other modalities, such as bedside echo and FAST, were mostly underutilized. This observation is not only consistent with other studies from LMICs but also highlighted the urgent need of integrating PoCUS into the EM residency curriculum in developing countries [8].
Similar to previous studies, the major barriers to PoCUS utilization identified by the participants in our were lack of training and limited knowledge [13,14]. The introduction of longitudinal teaching programs for both trainees and non-trainee physicians in EM may not only enhance their knowledge but result in better patient-centered outcomes [11,15].
Additional studies are needed to ascertain the optimal duration of high-fidelity-based simulation courses on PoCUS. In most developed countries, physicians participate in two-to three-day, multiple level sign-out courses in ultrasound [11]. However, two-hour sessions may be beneficial, with flattening of operators' learning curves after 10 to 30 examinations [16][17][18]. Moreover, written tests of knowledge regarding the use of PoCUS do not correlate with procedural skills, suggesting that evaluation of both skillsets may be necessary to tailor training [19].

Limitations
This study had several limitations. First, it was performed at a single academic center with a small group of participants, with all participants being invited volunteers. Secondly, we did not evaluate hands-on skills before the training session; therefore, improvements in image acquisition could not be evaluated. Also, retention and application of knowledge in clinical settings were not evaluated (Kirkpatrick's level 3 evaluation), and the confidence level was assessed only after the training session [20]. Although we observed a positive outcome, we did not compare outcomes after our 3.5-hour session with those after other courses of different lengths. Thus, an optimal course length requires further evaluation. Additional studies are needed to determine long-term retention of knowledge and skills after training and to determine whether knowledge gained in a simulation setting is used in real-life clinical practice.

Conclusions
PoCUS is not being used to its full potential in emergency departments of LMICs. The major barriers are the lack of training and knowledge. High-fidelity simulation training is effective in training EM physicians in e-FAST, with a half-day workshop that included simulations resulting in significant improvements in written test scores that assessed cognitive and image interpretation abilities. Further studies are needed to determine whether knowledge gained in simulation settings can be translated into clinical practice.

Additional Information Disclosures
Human subjects: Consent was obtained by all participants in this study. Ethical Review Committee Aga Khan University Hospital issued approval 2019-0882-2832. After ethical review committee approval data collection was done. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.