A Blindfolded Pediatric Trauma Simulation and Its Effect on Communication and Crisis Resource Management Skills

Background Miscommunication is a common cause of medical errors and patient harm. Simulation is a good tool to improve communication skills, but there is little literature on advanced techniques to improve closed loop communication (CLC) in an effort to minimize medical errors. This study looks to evaluate whether blindfolding simulation participants is an effective tool in improving communication, and whether this advanced teaching technique is useful for critical pediatric scenarios. Methods Participants included Emergency Medicine (EM) residents and Pediatric EM fellows with Advanced Trauma Life Support (ATLS) certification. Participants were randomized into groups and completed a pediatric trauma scenario. Recorded simulations were reviewed by three independent faculty for primary objective measures of total instances of communication and CLC utilization during critical actions in the simulation. The secondary objective was the perceived stress load by participants when utilizing this teaching methodology. Wilcoxon rank sum test (WRS), Fisher’s exact test (FET), and Cochran-Armitage test (CAT) were utilized for statistical analysis. Results Statistically significant differences were noted in total communication between groups. Median and interquartile ranges (IQR) of total instances of communication were 17.0 (14.7-17.1) in non-blindfolded groups versus 21.0 (19.0-22.0) in blindfolded groups (p-value=0.002). Statistically significant increase in CLC was noted during the critical action of monitor placement in the blindfolded group (OR=13.7, 95% CI=1.4-133.8). No differences were noted in crisis resource management (CRM) scores. NASA Task Load Index (NASA-TLX) scores of both groups revealed similar stress levels. Statistical testing based upon the year of training was limited by small sample size and large number of categories. Conclusions Blindfolded simulations increased total instances of communication overall and improved CLC in one critical action without increasing stress levels. The blindfolded trauma simulation exercise is an effective advanced technique to reinforce CLC utilization and communication skills.


Materials And Methods
This was an exploratory, randomized, pilot cohort study of pediatric trauma simulations utilizing a novel training methodology. Akron Children's Hospital IRB issued approval for this study (1166938-1). A single pediatric trauma scenario of moderate to high complexity was constructed prior to subject recruitment and was used uniformly for all participants. The constructed scenario was not based on any actual patient information, and any resemblance to persons living or deceased is a coincidence. Participants included pediatric emergency medicine fellows (n=8) and emergency medicine residents (n=20) that had completed their Advanced Trauma Life Support (ATLS) certification and were recruited on a volunteer basis (n=28) with informed consent. Participants were randomized into groups as blindfolded or non-blindfolded simulation leaders via a random number draw, with odd numbers representing blindfolded scenarios and even numbers representing non-blindfolded scenarios. The study participants were assessed on utilization of CLC orders and total instances of communication. Prior to starting the simulation, each participant was shown a standardized PowerPoint presentation reviewing CLC strategies and then a video demonstrating a sample blindfolded simulation scenario. Upon entering the simulated resuscitation bay, all participants were seated in a chair five feet from the patient's bed to minimize variance, with those wearing blindfolds facing backwards. Comprising the rest of the trauma team were three embedded participant nurses assigned the roles of airway support, procedure completion, or medication preparation/administration. Nurses were provided the scenario in advance to review and were instructed on appropriate CLC and how to communicate with the leaders during the scenario. A rehearsal simulation was conducted with the scheduled nursing staff at the beginning of each simulation day to acclimate them both to the mannequins, the scenario, and how they were expected to communicate with the participants.
The trauma scenario consisted of pediatric patient presenting in uncompensated hemorrhagic shock secondary to blunt abdominal trauma and an open pelvic fracture. The simulation scenario and critical actions can be found in Appendix 1. Each simulation was 10 minutes in duration followed by a 10-minute debriefing session. After the debriefing, participants were provided a post-participation survey and NASA Task Load Index (NASA-TLX) forms for completion. NASA-TLX is a multi-dimensional tool that assesses perceived workload stress in order to assess team effectiveness. NASA-TLX is scored on a scale of 0-450 for each subcategory, and weighted ratings are on a scale from 0-100, where an optimal level of perceived stress for a training tool is from 40-70. Emergency medicine, pediatric emergency medicine, and simulation faculty members completed CRM forms after each scenario was completed. All simulations were recorded for review and transferred to secured flash drives. Recorded scenarios were reviewed by three independent pediatric emergency medicine faculty for total instances of communication, CLC orders, and completion of critical actions. An appropriately completed CLC order included verbal communication of 1). an initial order; 2). confirmation of the order; 3). completion of the order; and 4). acknowledgement of completion of the order. Each independent faculty reviewer was provided the CLC presentation prior to completing reviews.
Data analysis utilized summary statistics and distributional assessments for continuous data, and frequencies and percentages for categorical data. Wilcoxon rank sum test (WRS), Fisher's exact test (FET), and Cochran-Armitage test (CAT) were used for statistical analysis. Unless otherwise noted all testing was two-tailed and evaluated at the Type 1 error rate of alpha=0.05 level of statistical significance. An inter-rater reliability assessment was conducted by determining the intraclass correlation coefficient (ICC) for each step (1)(2)(3)(4) of CLC individually as well as steps 1-4 combined (a completely closed communication loop) for each critical action as well as overall instances of communication throughout the entire simulation scenario. ICC was 0.92 for step 1, 0.58 for step 2, 0.69 for steps 3 and 4, 0.76 for completion of loop, and 0.71 for overall instances of communication, demonstrating significant variability in scores between raters. For purposes of statistical analyses, when counting the frequency of each step of CLC and number of loop closures, the number value used was the one agreed upon by at least two of the three reviewers, and the average of the numbers provided by the three raters was used for overall instances of communication.

Results
There were 13 non-blindfolded participants (46%) and 15 blindfolded participants (54%) enrolled in the study. Both blindfolded and non-blindfolded groups were similar in terms of age (CAT: p-value=0.964), years of training (CAT: p-value=0.953), and gender (FET: p-value=0.276). Years of medical training between both groups were similar, with both of them having an even distribution of higher and lower level trainees.
As seen in Table 1, the number of months that participants worked in an adult emergency department was similar between groups (p-value=0.771). The number of months that participants had worked in pediatric rotations was similar between groups as well (p-value=0.557), with over 50% having less than six months of total experience in both groups. This indicates a trend towards more emergency medicine than pediatric experience amongst participants, consistent with a larger number of participants being emergency medicine residents.  Regarding prior experience with real-life trauma scenarios, the groups were evenly distributed when it came to participation in (p-value=0.300), and leading (p-value=0.797), adult traumas. As observed in Table 2, over 50% of the participants from both groups had involvement in >20 adult trauma scenarios. There was a difference between groups in history of experience leading adult traumas that was not significant, with under 50% of the blindfolded group having led >16 traumas when compared to under 50% in the nonblindfolded group. When it came to real-life pediatric traumas, the groups were evenly distributed in both history of participation in (p-value=0.688), and leading (p-value=0.849), pediatric traumas. The majority of learners in both groups were inexperienced in leading pediatric traumas, with most (66.7% and 69.2% in the blindfolded and non-blindfolded groups respectively) having been leaders in two or less pediatric traumas throughout their medical training.

TABLE 2: Trauma experience of study cohorts (*Cochran-Armitage Trend Test)
Comparison of communication utilization revealed evidence of a statistically significant difference in total instances of communication between the two groups. Median and interquartile ranges (IQR) of total number of communications were 17.0 (14.7-17.1) in non-blindfolded groups versus 21.0 (19.0-22.0) in blindfolded groups (WRS: p-value=0.002). Although CLC completion was poor in both groups, there were statistically significant differences in CLC during the critical action of placing monitors, with blindfolded groups more likely to close the loop (OR=13.7, 95% CI = 1.4-133.8). During monitor placement, the most commonly missed steps of CLC were steps 3 and 4, with non-blindfolded participants 6.2 (95% CI of OR = 1.2 -32.0) and 14.6 (95% CI of OR = 2.2 -97.6) times more likely to fail completing those steps respectively compared to blindfolded groups. No significant differences were noted in CRM scores or NASA TLX scores. NASA TLX in both groups were above the validated cut off of 70 for "too stressful" of an experience (71.7 in the nonblindfolded group, 73.7 in the blindfolded group).
CLC utilization was most consistent in the critical action of assessing and securing the patient's airway, but it was still only utilized by approximately 60% of the participants. The critical actions with the least completion of CLC were activation of the massive transfusion protocol and disposition to the operating room.
As a secondary, exploratory analysis, when reviewing CRM and NASA TLX scores based on levels of training, the small sample sizes precluded conduction of an inferential statistical analysis. However, evaluation of the summary statistics could yield insight into trends of both CRM and NASA TLX scores that could benefit from further examination in future research.
As observed in Table 3, overall CRM scores trended toward improvement with level of training. There is a drop in overall scores around those with four years of experience before the scores begin to improve with year of training. Leadership, resource utilization, and communication skills also shared this same trend. Problem solving skills improved consistently with more years of training. There appears to be improvement in the majority of the CRM skills categories with increased years of medical training, but we were unable to interpret for statistical significance due to the small sample size.  As observed in Table 4, the scores of the overall and weighted stress ratings did not show any consistent trend with years of medical training. Notably those with one, four, and five years of training felt like the simulation was not excessively stressful of an experience, whereas those with two, three, and six years of training did give the simulation experience a score that would make it too stressful to be useful. The most consistently demanding TLX subcategory during the simulations was Mental Demand for most groups of years of training. Amongst those participants with three years of training, the most demand and stress came from Frustration, although it is not clear whether this is from frustration with the scenario, with their lack of pediatric experience, or with the blindfolding. The TLX score category that was universally the lowest in this project was that of Physical Demand across all participants regardless of years of training.

Weighted
As observed in Table 5, post-simulation surveys indicate this was a very well received training exercise, with most participants enjoying the opportunity to improve clinical skills in pediatric resuscitations in a controlled environment and getting experience in a leadership role. Participants appreciated the time spent in the debriefing to help cement the learning points regarding appropriate management of pediatric traumas and how to better communicate with staff in a position of leadership. All the participants that were blindfolded agreed that being blindfolded made the simulation more challenging by removing their regularly relied on visual cues, but still considered it a great learning experience and would volunteer again if they were readily available.

Survey questions Common responses and selected quotes
What was the best part of this training session?
"Practicing a high stress clinical scenario without the stress of a real child decompensating." "Forcing me to verbalize each step in the process, which will help me to remember each step when I am under more pressure in a real world scenario." "Blindfold forces the team leader to step back. EM is so procedure driven we get used to hands on at bedsides and sometimes miss the 'big overall picture.'" "Practice running peds trauma" What was the Try to encourage junior residents to volunteer more The beginning instructions could have been clarified more, especially for the blindfolded group Other comments "Honestly, great exercise and well put together." "Helpful to do a simulation focused on communication, allowed me to recognize importance of it."

Discussion
Blindfolding of the team leader increased total instances of communication overall and also improved CLC in the critical action of monitor placement. The most commonly missed steps of CLC in this critical action were verbalizing completion of the order and verbalizing acknowledgment of order completion, with the blindfolded group performing better than the non-blindfolded group. Overall, the blindfold technique did not demonstrate improved CRM scores. Although summary statistics showed a trend toward improved CRM scores based on years of training, the small number of participants per group did not provide enough power to establish statistical significance in this trend.
The results obtained from the study appear to show a trend consistent with our assumption that removing visual cues would increase verbal communication and overall improve communication skills. The measured increase in total instances of communication could theoretically allow more opportunities to improve the frequency of good CLC behaviors if practiced in a controlled training environment. More regular blindfolded simulations could be useful as a teaching modality to reinforce these good behaviors, making them a habit for learners to apply instinctively in real-life critical scenarios. Additionally, performing blindfolded simulations more frequently could potentially decrease the stress levels and challenge for the learners through experience and familiarity, with both the pediatric simulation setting and with having to wear a blindfold, based on the trends seen in summary statistics per year of training.
As seen in Table 1 and Table 2, the majority of learners had extensive adult emergency medicine and trauma leadership experience, but a very small amount of pediatric trauma experience and time spent on pediatric rotations, which is likely attributable to a large number of adult emergency medicine trained learners compared to pediatric emergency medicine trained learners. As management of critical pediatric patients is a known area of discomfort for many adult emergency providers, this may have had an impact on both how scenarios went overall and on the reported participant stress levels. Having had less exposure to pediatric medication weight-based doses, algorithms, and equipment sizes could theoretically cause a cognitive overload that detracted from focus on communication skills and techniques, thereby contributing to the overall poor CLC completion amongst learners. Conducting these simulations with two large cohorts, one of pediatric emergency medicine learners and one of emergency medicine learners, would be beneficial to examine this effect in future studies.
The trend towards improvement in CRM skills with years of medical training could make this a more beneficial teaching method for the more experienced. Their prior experience would facilitate focusing on the communication aspect of CRM specifically rather than being overwhelmed with the other CRM skills also required during the simulations. The majority of participants with more than three years of medical training (n=13) did not find the simulations to be too stressful regardless of whether they were blindfolded or not, presumably due to more familiarity with appropriate critical medical management and confidence in making decisions, supporting the theory that this might be a better teaching method at higher levels of training.
A larger, more longitudinal study would be beneficial to further examine these findings, and the study could be expanded by providing follow-up simulations at regular intervals to assess for persistent changes in communication behaviors as well as for retention of other CRM skills. This would additionally allow observation of trends in overall stress levels reported with subsequent simulations, helping to differentiate whether high-stress scores may be due to inexperience with pediatric simulation in general or with having to wear a blindfold. Additionally, an expansion of pediatric scenario types, rather than just trauma scenarios, would be ideal to help reduce future medical errors and increase competency in pediatric management across the board.
This study's largest limitation was its small sample size. Having only 28 participants meant that analysis of the data in terms of years of training was underpowered and statistical significance of results could not be assessed. This also limited the analysis of the primary objectives of CLC completion of critical actions. This study was also limited to a single-center study. Although there were EM residents from five different programs participating, the majority of them came from a single program where the residents do not have regular exposure to pediatric patients, which could skew results due to both inexperience and heightened stress levels in pediatric simulation scenarios. The three independent raters are local clinical faculty members and have worked with the majority of learners, potentially introducing some bias.
Another area of variance was the embedded participant nursing staff, as the same nurses could not participate during every scheduled simulation. As new embedded participant nurses had to be trained and learn the scenario before each day of simulation, there was an increased likelihood of them introducing errors or making the scenario artificially more challenging due to varying levels of experience. A last possible area that could have affected performance of the learners came from an attempt to minimize bias in the study design. All participants were forced to be seated five feet from the patient regardless of whether or not they were blindfolded, and those that did not have a blindfold in place found it stressful to be restricted from approaching the patient or standing up as they normally do in their day to day practice, which could have affected the way they would normally manage a pediatric trauma.

Conclusions
In this study, blindfolded simulations increased total instances of communication overall in comparison to traditional simulation scenarios and improved CLC in the critical action of monitor placement without significantly increasing perceived stress levels of participants. Although both groups considered the exercise to be too stressful to be a good learning technique, the overall weighted stress scores given were incredibly close to the optimal range for a teaching tool. EMS: "This is a 7-year-old girl that was involved in an MVC with her family about 20 minutes ago. She was unrestrained and ejected from the car. We found her unresponsive at the scene. She was hypotensive and we have failed all attempts at obtaining IV access. We have placed her in a C-collar and on a backboard. She has abrasions all over her lower extremities and back. VS at the scene were O2 Sat of 92%, BP of 95/60, HR of 145, RR of 28 with poor effort. She has been bagging easily en route. Trauma was activated from the scene."

Appendices
WEIGHT for the scenario: 25kg using Broselow (once resident asks for weight) PMHx: EM team will immediately take over care from paramedics. They will determine her GCS score of 8 or less and immediately attempt to secure an airway or continue to support with bagging. IO and/or 2 large bore IVs should be placed with normal saline bolus 20ml/kg ordered immediately. Ask for 2 units of trauma blood and activate massive transfusion protocol at either 1:1:1 or 2:1:1 ratio.
They should find the unstable pelvis, abdominal wall bruising, and additional suspected intra-abdominal hemorrhage (via positive FAST exam, although unnecessary if patient goes straight to OR due to suspected pelvic source of hemorrhage). A sheet should be tied around the pelvis. Patient will need aggressive fluid/blood resuscitation. Team will ask for labs. If US requested they will be described findings of fluid collection in Morrison's pouch.
TRANSITION: Once the patient has had breathing supported, has been given fluids/blood, and massive transfusion has been activated, and trauma surgery has been contacted, patient will need for transport to the operating room (OR) for definitive management.