Evaluation of a Hands-On Wrist Fracture Simulator for Fracture Management Training in Emergency Medicine Residents

Background Fractures are common in the emergency department, and fracture management training poses certain challenges. Recent emergency medicine (EM) residency graduates feel only somewhat prepared to manage fractures. In this study, our objectives were to determine the effect of introducing a wrist fracture simulator (Sawbones®) to traditional EM fracture management education and to assess resident attitudes, comfort with fracture management, and perceptions of the simulator. Methodology This six-month prospective study involved postgraduate year one residents at two academic EM programs. For convenience, each residency was considered as one test group. One residency group was deemed the traditional group (n = 10), while the other was the intervention simulator group (n = 16). Identical traditional lectures and buddy splinting workshops were provided. The simulator group received supplemental training with the Sawbones® simulator. Groups were filmed using this simulator for fracture management before the teaching sessions and at six months. Grading utilized a 27-point scale, with a subscale covering reduction. Data were collected regarding attitudes, comfort with fracture management, and perceptions of the simulator. Results In total, 26 residents participated in the study. There was no significant difference between groups at six months in overall fracture management scores (traditional group: 15.8 ± 3.1; simulator group: 15.4 ± 3.9; p = 0.92). On the subscale of fracture reduction skills, the simulator group showed significant improvement (p = 0.0078), while the traditional training group did not (p = 0.065). Both groups reported satisfaction with the simulator, improved comfort, and knowledge of fracture management. Conclusions Fracture management is an essential competency, and prior research has shown that most graduating EM residents do not feel comfortable with these skills. All participating residents in this study struggled with adequate fracture management, even after the teaching session. Our study suggests that there is a benefit to supplementing traditional training with a fracture simulator.


Introduction
Fractures are frequently managed in the emergency department (ED), and it is estimated that fracture rates will increase with a more active and aging population [1]. Specifically, distal radius fractures are commonly managed by emergency physicians [2,3]. Orthopedic education and training for emergency medicine (EM) residents are lacking [4]. Recent EM residency graduates reported feeling only somewhat prepared to care for fractures in the ED [5]. The Accreditation Council for Graduate Medical Education (ACGME) has no specific milestones addressing competency in fracture management skills [6]. Additionally, in a cognitive assessment of common upper extremity disorders, EM residents struggled to reach competencies [7]. The challenges of attaining competency in orthopedic management extend beyond EM residency programs and have similarly been seen in internal medicine, radiology, and pediatric programs [7][8][9]. Furthermore, the pervasive issue of inadequate musculoskeletal education has been documented in medical school, compounding these training challenges [10,11]. Further stressing the need for adequate orthopedic training in EM residents is the shortage of orthopedic surgery on-call coverage. Only one-third of emergency physicians reported they worked at hospitals with full-time orthopedic surgery coverage, and 8% stated they never had orthopedic coverage when working in the ED [12]. Orthopedic training interventions have shown promising results among both EM and orthopedic physicians [13,14]. Adequate fracture management training is essential for improving patient outcomes and has been shown to decrease the need for surgical intervention [14,15]. An area requiring further study in EM is orthopedic simulator training to augment traditional education. Fracture simulators have been successfully implemented in orthopedic residency training programs [16]. Simulation education, when compared to traditional medical training, has been shown to enhance the quality and outcomes of advanced cardiac life support, improved surgical skills, medical team communication, and central venous access [17][18][19][20]. In addition, the vast majority (94%) of resident physicians report wanting simulation used for procedure teaching [21]. However, there is a lack of research evaluating the use of fracture simulators for EM resident education. In this study, we sought to assess the effects of integrating a novel wrist fracture simulator into traditional fracture management training. Our primary objective was to determine the effect of introducing a novel wrist fracture simulator (Sawbones®) to traditional EM fracture management education for postgraduate year (PGY)-1 EM residents. The secondary objectives were to assess PGY-1 EM resident attitudes and comfort levels with fracture management and perceptions of the simulator.

Materials And Methods
This study was conducted among PGY-1 EM residents at two ACGME-accredited three-year programs at tertiary care centers. Both sites obtained institutional review board approval. The study subjects were PGY-1 EM residents at both sites. There were no further exclusion criteria for participation in the study. Site 1 had 16 EM PGY1 residents (simulator group) and site 2 had 10 EM PGY1 residents (traditional training group). Informed consent was obtained from all study participants. EM PGY1s at both sites completed a demographic form and pre-assessment survey asking about prior orthopedic experience and comfort and attitudes toward fracture management. All participants at both institutions were filmed performing fracture management on a dorsally displaced distal radius fracture using a Sawbones® fracture simulator arm [22]. After initial filming, both groups received identical traditional training on fracture management that included a lecture and a hands-on buddy splinting workshop. Participants in the simulator group received an additional 30-minute hands-on workshop specifically focused on fracture reduction techniques using the Sawbones® wrist fracture simulator. All participants completed a post-teaching survey regarding their attitudes and perceptions toward the teaching session and their comfort levels with fracture management immediately following their educational sessions. Six months post-teaching session, all residents were again filmed performing fracture management on a distal radius fracture using the Sawbones® simulator. After filming, all participants completed a six-month post-training survey regarding their attitudes and perceptions toward the simulator and their comfort levels with fracture management. A standardized fracture management assessment checklist for grading the video recordings was created in a multi-step approach (Supplemental Appendix) The checklist focused on the core elements of emergency fracture management, including assessment, reduction, and stabilization. First, a literature review was conducted using EM and orthopedic primary source material and was used to create a list of required fracture management actions [23][24][25]. A panel of five attending emergency physicians at both sites then voted on which items should be included, modified, or removed from the initial checklist. There had to be a group consensus for each item on the checklist for it to remain. Four rounds of voting and modification took place to reach a final checklist consensus. Items 7-11 on the checklist specifically focused on fracture reduction actions rather than fracture assessment or splint application. The two filmed sessions of each EM resident's fracture management skills were viewed and scored independently by two attending emergency physicians (one from each site) who used the grading assessment checklist to score each video session. Two additional attending emergency physicians watched the filmed assessments and adjudicated any grading discrepancies. Reliability assessments were completed on the fracture management checklist using the multi-grader assessments. The internal reliability of the checklist was assessed by computing Cronbach's alpha. Interrater reliability for the total score and items 7-11 (fracture reduction subscore) were assessed using the Shrout-Fleiss intraclass correlations for continuous variables calculation. The p-values for the demographics, pre-teaching, and six-month post-teaching surveys were calculated using Fisher's exact test or Wilcoxon test. The p-values for pre and six-month post-training video scores were calculated using the Student's t-test or Wilcoxon sign-rank test. All p-values were set at <0.05 for statistical significance.

Results
A total of 26 EM PGY1s participated in this study. There were 16 residents in the simulator group and 10 residents in the traditional training group. There were no statistically significant differences in preassessment demographic information between the groups ( Table 1)

IQR: interquartile range
Post-training surveys immediately following the teaching session revealed no significant differences in satisfaction with teaching sessions or improved knowledge and comfort with reduction and splinting between the two groups ( Table 2). However, both groups were satisfied with the Sawbones® simulator and rated it similar to a human fracture and felt that the teaching sessions improved their knowledge and comfort levels with fracture management skills ( Table 2). Six-month post-training survey results are shown in Table 2. In the six-month post-training survey, there was a statistically significant difference in the number of orthopedic rotations completed by the interns in the traditional training group (40%) versus the simulator group (0%) (p = 0.014) ( Table 2). Residents in both groups had decreased comfort levels with orthopedic reductions and splinting at six months compared to their comfort levels immediately after training.

Simulator (n = 16)
Traditional training (n = 10) P-value  There was no statistically significant difference in overall fracture management scores or in the subscale of fracture reduction skills between groups at six months post-training (p = 0.807 and p = 0.461, respectively) ( Table 3). There was, however, a statistically significant increase from pre-training to six-month posttraining total scores for both the simulator and traditional training groups (p < 0.0001 and p = 0.0004, respectively) ( Table 4). There was a statistically significant increase from the pre-training scores to sixmonth post-training scores for fracture reduction-specific items 7-11 for the simulator group (p = 0.0078), but not for the traditional training group (p = 0.063) ( Table 4). The checklists demonstrated internal reliability using Cronbach's alpha for the post-training videos total score of 0.63 and for items 7-11 of 0.67. Inter-rater reliability for the video checklist was excellent, with a Shrout-Fleiss intraclass correlation of 0.78 and 0.75 for the total score and items 7-11 score.
As expected in our study, there was a statistically significant increase in procedural competency from the initial pre-training to post-training video assessments for all participants. While there was no significant difference in the overall scores between the simulator and traditional training group residents, the subgroup analysis for items 7-11 (fracture reduction skills) showed a statistically significant increase in assessment scores for the simulator group compared to no significant increase in assessment scores for the traditional training group. These items (7-11) correlate specifically with the technical aspects of fracture reduction that were the focus of the additional 30 minutes of hands-on practice with the Sawbones® simulator that the simulator group received. Gaining adequate levels of hands-on practice with real or realistic fractures has traditionally been an area where EM orthopedic education has struggled. It is crucial as adequate fracture reduction has been shown to decrease surgical requirements for distal wrist fractures [13][14][15]. This was also seen in this study with the low number of reported reduction/splinting/casting experiences both study groups were involved with ( Table 2).
While there was no difference in the overall scores between the groups, the finding that the simulator group showed significant improvement in reduction-specific maneuvers while the traditional training group did not is important. Fracture injury assessment and splinting skills were expected to be similar between the groups as they shared common traditional training for these skills. This finding is especially notable considering that 40% of the traditional training group completed an orthopedic rotation during the course of our study compared to no residents in the simulator group. The traditional training group, likely because of their early exposure to an orthopedic rotation, reported significantly more reduction/splinting opportunities during the six-month period between study training and assessment. Our study demonstrates a potential benefit with respect to fracture reduction skills of the addition of a fracture reduction simulator to traditional EM orthopedic training.
One interesting result from the study was lower than expected overall scores on the post-training assessments for both groups. The traditional and simulator groups scored 15.8 ± 3.1 and 15.4 ± 3.9, respectively, on the 27-point assessment. While both groups did show a significant improvement from their pre-training scores, their post-training scores likely do not reflect reaching a level of adequate competency for independent fracture management practice. This is likely multifactorial including knowledge and skill degradation during the six-month gap between training and post-training assessment. In addition, this was compounded by the limited number of fracture management opportunities the residents had during the sixmonth study period. This also likely corresponds to the decrease in six-month comfort levels, with reductions seen in both groups and lack of comfort also found in recent EM graduates [5]. As such, these findings highlight the challenges in EM orthopedic training that procedure simulators may help address.
Finally, the EM residents involved in the study were satisfied with the simulator and deemed that the Sawbones® simulator was similar to a human fracture. This is similar to prior research from orthopedic surgery literature suggesting that hands-on fracture reduction simulators are deemed realistic and can be reliably used in fracture reduction and casting assessments [16,[26][27][28].
Our study had several limitations. First, some degree of incorporation bias is present. During their training, the simulator group received an additional 30 minutes of time with the Sawbones® model. This likely allowed increased familiarity with the model which could have inflated the fracture reduction benefits noted in the simulator group. However, there were six months between the initial training didactics and the postassessment which likely helped mitigate this bias. Another limitation was the lack of a validated fracture management checklist for EM residents. Although the checklist created for this study demonstrated strong internal reliability and inter-rater reliability, it would benefit from further validation. Additionally, there was a discrepancy between the completion of an orthopedic rotation between the two groups; 40% of the traditional training group residents completed an orthopedics rotation compared to none in the simulator group during the study period and therefore had more fracture management exposure. This may have caused a negative skew making it harder for the simulator group to show improved reduction skills. There could have possibly been a statistically significant difference between the two groups in overall competency that was masked by this bias.

Conclusions
Fracture management is an essential skill that most graduating EM residents do not feel comfortable with.
Although there was no overall benefit to the introduction of a Sawbones® wrist fracture simulator to traditional EM orthopedic training in procedural competence with fracture management, assessment, reduction, and splinting, there was a benefit in introducing the Sawbones® wrist fracture simulator with respect to fracture reduction skills. Our study, although small, suggests that there is a possible benefit to adding a fracture simulator to supplement traditional EM orthopedic training.

Additional Information
Disclosures