Immersive Video Modeling Versus Traditional Video Modeling for Teaching Central Venous Catheter Insertion to Medical Residents

Background Central Venous Catheter (CVC) placement is a common critical care procedure. Simulated practice has been shown to reduce its iatrogenic complications. Video modeling (VM) is an instructional adjunct that improves the quality and success of CVC insertion. Immersive VM can improve recall and skill translation, but its role in teaching medical procedures is not established. Research question/hypothesis We hypothesized that, relative to traditional VM, immersive VM would decrease cognitive load and enhance ultrasound-guided CVC insertion skill acquisition. Methods Thirty-two resident physicians from four specialties were randomized into traditional (control) or immersive VM (intervention) groups for three CVC training sessions. Cognitive load was quantified via NASA Task Load Index (TLX). Mean (± standard deviations) values were compared using two-tailed t-tests. Skill acquisition was quantified by procedural time and the average 5-point [EM1] [TB2] entrustment score of three expert raters. Results Overall entrustment scores improved from the first (3.44±0.98) to the third (4.06±1.23; p<0.002) session but were not significantly different between the control and intervention groups. There were no significant differences between NASA TLX scores or procedural time. Conclusion We found no significant difference in entrustment, cognitive load, or procedural time. Immersive VM was not found to be superior to traditional VM for teaching CVC insertion.


Introduction
Central venous catheter (CVC) placement is a medical procedure that involves the insertion of a catheter into the venous system to facilitate the administration of medications, fluids, and blood products [1]. It has been associated with serious mechanical (pneumothorax, bleeding, thrombus formation, occlusion, extravasation, catheter embolism or breakage, fistula formation, air embolism, pericardial tamponade, cardiac aneurysm, or vein stenosis) and infectious (cellulitis, phlebitis, intracardiac abscess, or sepsis) complications. These complications vary in prevalence due to different definitions, reporting patterns, site selection, the catheter used, dressing standards, patient choice, and provider experience [2].
Expert video modeling is a common component of medical simulation training that originated in athletics training [3]. It allows trainees to witness the performance of a procedure to develop self-efficacy and confidence [4]. CVC insertion training that utilizes both video modeling (VM) and task trainers for procedural simulation has been shown to decrease instructor time [5,6], training time [5], equipment spoilage [6], and adverse events [5,7]. The merit of traditional VM has been demonstrated in procedural 1, 2 3 4 5 3 6 6 skills training, particularly in sports [3,8,9]. Recently, more immersive educational technologies, such as virtual reality headsets, have been made available at a reasonable cost, but their application has not been explored in this context.
Immersive VM uses video capture technology and a virtual reality headset that surrounds the user with a convincing replica of the same environment that they might expect in a real-life setting [10]. Immersive VM has been demonstrated to promote learner engagement in the tasks they observe [11][12][13][14]. This increased engagement could conceivably improve retention and learning. Additionally, immersive VM has a complicated impact on cognitive load. Frederiksen et al. found that performing procedures in immersive virtual reality increases cognitive load [15]. However, cognitive load decreased with repeated exposure. They hypothesized that through repeated exposure to higher cognitive load during training, trainees may be better prepared for complex real-world performance [15].
Building on this literature, we hypothesized that immersive VM would be acceptable to learners and that, relative to traditional VM, immersive VM would result in enhanced ultrasound-guided CVC insertion skill acquisition mediated by the decreased cognitive load during procedure performance.

Materials And Methods
The University of Saskatchewan Research Ethics Board deemed this study exempt from ethical review by (BIO# 18-46). The Regina Qu'Appelle Health Region provided operational approval.
All 32 first-year residents from four training programs (anesthesia, emergency medicine, general surgery, and internal medicine) in Saskatoon and Regina (Saskatchewan) were enrolled in a CVC insertion training program at our institution and were invited to participate in the study. Consent was provided by each participant prior to the first session and demographic information was collected (Appendix A).
Expert instructional procedural videos were created simultaneously in traditional (Video 1) and immersive (Video 2) video formats. The traditional format was a fixed, two-dimensional video viewed on the screen of a Samsung S6 32GB cell phone (Samsung Electronics, Suwon, South Korea). The immersive format was a video of a 180-degree wide field of view that was viewed on a Samsung S6 32GB cell phone placed in a Samsung Gear Virtual Reality Headset (2016 edition, Samsung Electronics). The headset allowed the user to raise, lower, and turn their head to change their visual focus and prevented visual input from their true surroundings. We recorded the instructional procedural videos using a CVC Insertion Kit (Teleflex, Wayne, USA) and a CVC Internal Internal Jugular Task Trainer (Simulab, Seattle, USA) with a Samsung S6 32GB cell phone (Video 1) and a 360fly 4K Video Camera (360fly, Canonsburg, USA) (Video 2). Both videos were edited with the same instructive audio narrative using Premiere Pro (Adobe, Inc., San Jose, USA). Equipment costs are outlined in Appendix B. The ultrasound-guided, CVC insertion training program spanned three three-hour sessions over a 12-week period. Each group contained three to four residents. All residents were provided with prereading material outlining the procedure. At the beginning of the first session, all residents received procedural instruction by a staff physician experienced in CVC insertion. The instructor taught a standardized, step-wise approach to CVC insertion based upon our institution's best practices. The resident groups were randomized to the control (traditional VM) or the interventional (immersive VM) groups. We aimed to maintain the composition of each instructional group throughout the study, but in cases where this was not possible because the residents changed instructional groups, they conducted their VM consistent with the study group that they were initially assigned. Following the viewing, both groups practiced the skill on task trainers. Individualized feedback was provided concurrently by the instructors throughout the practice session. The subsequent two training sessions followed the same video review and practice format but did not include dedicated instruction at the beginning. Concurrent feedback was provided by instructors through all of the sessions.

VIDEO 1: Traditional Video Model
At the end of each of the three training sessions, each participant completed a one-on-one testing session supervised by a staff physician instructor. These sessions were video recorded to include the procedural area, the participant's hands and arms, and the ultrasound screen. No feedback was provided during this test session. After each session, participants were asked to complete a NASA Task Load Index (TLX), a multidimensional scoring system that assesses cognitive workload (Appendix C), while the observing instructor completed an assessment form consisting of the O-Score entrustment scale [16] and narrative feedback. Participants were stopped by the instructor if they exceeded 15-minutes of procedural time. Time zero began with the first needle insertion into the skin and concluded with the placement of the Tegaderm™ (3M Company, Siant Paul, USA) dressing to secure the CVC. Task success was defined as the completion of the task within the time frame. Failure was defined as the failure to complete the procedure within the allotted time.
Following each participants' testing session, their procedural and ultrasound videos were combined into a single video ( Figure 1). These videos were saved on a password-protected hard drive with file names based on a computerized random number order generator from 100-400. Time from first needle insertion to Tegaderm™ placement was determined by a blinded study investigator. Two additional staff physician investigators, who were not involved in the participant's recorded teaching session but were familiar with the curriculum and its assessment, performed blinded assessments of each recorded CVC insertion using the O-Score entrustment scale completed via SurveyMonkey (SVML Inc., San Mateo, USA) (Appendix D).

FIGURE 1: A sample of the video created from a participant's test session, showing their performance with the video of the ultrasound matched in a picture-in-picture view
Procedural competence was assessed using the average of the three entrustment scores (one bedside rater and two video raters) and procedure length (from first needle insertion to Tegaderm™ placement). Cognitive load during simulated CVC placement was assessed using the NASA TLX score completed by each participant following each test session. These variables were compared between sessions within groups, as well as between groups using paired and unpaired two-tailed t-tests as appropriate. A power calculation conducted using an alpha of 0.05 and a beta of 0.8 seeking a one-point difference on the entrustment score between the groups required two groups of 16 participants.
Lastly, the acceptability of immersive and traditional VM was explored by comparing CVC insertion training program evaluations from residents in each group. The program was evaluated using a modified version of the evaluation of technology-enhanced learning materials learner perspective (ETELM-LP) survey [17]. Participants completed the survey online using SurveyMonkey (Appendix E) after all three sessions were completed and the results were compared by group using t-tests.

Results
All 32 first-year eligible for the study participated. Five additional third-year residents participated in the training program but were not eligible for the study. Group demographics are outlined in Table 1

TABLE 1: Demographic characteristics of the control and intervention groups
Cognitive load (as assessed by the NASA TLX) and procedural competence (as assessed by procedural time and entrustment score) are reported in Figures 2-4. There was no difference in cognitive load during the procedure in the immersive VM group. Further, there was no change in cognitive load through the three study sessions. There was a significant improvement in entrustment score (mean±SD) from the first (3.4±1.0) to the third (4.1±1.2; p<0.001) session across all participants. There were no statistical differences in entrustment scores between the control and intervention groups during the last session. Time to procedure completion did not significantly change from the first to the third session in either group.  Each item on the ETELM-LP (Appendix F) was rated from strongly disagree (1) to strongly agree (7). Significant differences between the groups are outlined in Table 2. Participants in the control group reported higher agreement that navigation of the technology-based components of the course was logical, consistent, and efficient; that the technology and media supported the learning objectives; that the course did not require inappropriately high technology skills; and that they did not have significant technical problems during the course. The control group also reported higher agreement that the quality of the video review helped them learn the skill, contributed to the achievement of the learning objectives, and was consistent with the instructor's teaching of the pre-course material.

ETELM Question Control Average
Intervention Average

pvalue
Navigation of the technology-based components of the course was logical, consistent, and efficient The quality of the video review of the procedure helped me to learn the skill 6.4 5.5 0.03 The video review of the procedure was consistent with the instructor's teaching and pre-course material 6.4 5.6 0.02 The video review of the procedure contributed to the achievement of course learning objectives 6.5 5.5 0.008

Discussion
Contrary to our hypotheses, immersive VM did not significantly reduce the procedural cognitive load or improve the competence of first-year resident physicians relative to traditional VM. Further, the results of the student evaluations suggest that traditional VM was preferred by learners.
Despite the strong literature base for the use of VM [3,9,18,19], we are unaware of any previous studies investigating its interplay with cognitive load. Early investigations of immersive video technologies [11][12][13][14][15] suggest that they are acceptable to learners and can be an effective educational tool [20]. Building on recent literature suggesting that immersive VM initially increases procedural cognitive load but that this cognitive load decreases with time [15], we hypothesized that incorporating immersive VM at the beginning of procedural training could decrease cognitive load during successive attempts at the procedure. However, our study suggests that this did not occur between sessions or the two groups. There are multiple potential reasons for this: the testing environment may have been persistently stressful, the introduction of a new VM technology may have resulted in an increased extraneous load that further complicated skill acquisition [21], or the three three-hour sessions may not have provided enough time to develop the complex task schema required for CVL insertion, thereby influencing cognitive load and working memory.
Previous research demonstrated increasing procedural competence over the course of multi-session CVC training programs [19][20][21]. Our findings of improving entrustment scores from the first to last session reflect this, but we did not find a significant difference between the two groups. There could be several reasons for this. Firstly, the concurrent feedback provided in both sessions allowed for the opportunity to facilitate deliberate practice, a method of teaching that depends upon focused, repetitive practice of skill improvement, with feedback [22,23]. The quality of our in-task, concurrent feedback may have overshadowed any positive effect immersive VM could have had on the scoring. This has been recently studied in novice medical students [24], wherein early procedural learning benefitted from VM, but in later procedural interventions students subjectively benefitted more from concurrent feedback. Secondly, we identified that the cognitive load remained higher in the immersive VM group, which may have adversely affected performance. Thirdly, the benefit of VM in the theoretical model of deliberate practice may be rooted in the learners' ability to observe and analyze the performance of experts at key decision points, with expert guidance [23]. Entrustment scores in simulated environments may be an indicator of competence in the clinical setting [25], suggesting that the improvements seen in this workshop may translate to enhanced patient care [26]. Procedural time is also a recognized surrogate of procedural mastery, with the level of experience being inversely related to time [21,[25][26][27]. However, we did not find this association in our study. This may have related to the heterogeneity of specialty programs participating in the study or the amount of time (four weeks) between the three training sessions.
Comparisons of the course evaluations between the control and intervention group raise additional concerns regarding the viability of immersive VM for procedural training. While support was available during the sessions for the residents, the results demonstrate significantly lower ratings for the immersive VM group. While statistically significant, this may not reflect a clinically relevant significance to learners and requires a more in-depth assessment as to the reasons behind these differences. Nevertheless, this suggests that despite the initial enthusiasm for this technology from learners [11][12][13][14][15], in the time frame and workshop format provided in this study, it may not have achieved the ease of use or familiarity to learners.
There were limitations to our study. First, we intentionally sought to conduct a pragmatic study by utilizing commercially available equipment. While we achieved this objective, it is possible that more advanced recordings and VM could have been incorporated with additional funds and equipment that may have resulted in an improved learner experience. Second, there were user-onboarding difficulties in the use of the commercial virtual reality (VR) headsets for the learners and instructors. Despite being a commercial product designed for recreational use, there were limitations related to internet connection speeds, user-interface challenges, and navigation within the VR software. As learners and instructors became familiar with the hardware, this became less of a barrier. Third, there was a delay in the completion of the NASA TLX questionnaire by a small minority of participants despite the time being provided for this during the session. This may have subjected these results to recall bias. Finally, 28% of the participants had previous formal training with CVC insertion. These were evenly distributed between groups, however, this study did not specifically investigate the extent of this formal training, and thus may have influenced the outcomes of interest.
Our study did not demonstrate the benefit of immersive VM over traditional VM, however, there will continue to be opportunities for further investigation in this field. In the short-term, it may be worth investigating the impact of immersive VM for teaching more complex procedures (e.g., intraoperatively) than ultrasound-guided CVC insertion as it may have a larger benefit in these contexts. In the long-term, the permeation of immersive experiences and technologies may become more commonplace, decreasing their cognitive load. Immersive technologies and augmented reality technologies will continue to rapidly evolve and become more intuitive. It will be important to continue to investigate the use of this and newer technology for medical procedural training as it becomes available.

Conclusions
Simulation using task trainers and video modeling is an effective way to teach medical procedures associated with iatrogenic complications in a low-risk environment. Increasingly sophisticated forms of video modeling have recently become readily available. We hypothesized that teaching ultrasound-guided CVC insertion using immersive VM technology would provide benefits over traditional video modeling. However, the use of immersive VM did not change the cognitive load or improve outcomes over traditional VM when used to augment simulation training for novice resident physicians learning this procedure.  Instructions provided a good introduction to the course 5.9

Appendices
Course objectives, expectations, and policies were clearly stated 6.2 The course was well organized 5.9 Course objectives were relevant to my needs 6.2 Navigation of the technology-based components of the course was logical, consistent, and efficient 5.8 The course technologies and media supported the learning objectives 5.9 This course did not require inappropriately high technology skills 6.2 I did not have significant technical problems during this course 5.9 The educational activities encouraged engagement with course material / content 6.0 The educational activities promoted achievement of the course objectives 6.1 There was a strong instructor presence / personal touch in the course 6.2 Educational activities encouraged interaction and collaboration with other participants 5.8 The course effectively blended online and face-to-face elements 5.7 Assessments were appropriate for the course objectives, content, and activities 6.0 I had sufficient opportunity to assess and reflect upon my learning progress 6 The video of an experienced clinician demonstrating the skill was effective to helping me learn 5.9 There was enough time to watch the video review of the procedure 6.1 The quality of the video review of the procedure helped me to learn the skill 5.9 I felt uneasy when watching the video review of the procedure 2.8 The video review of the procedure was consistent with the instructor's teaching and pre-course material 5.9 The video review of the procedure contributed to the achievement of course learning objectives 5.9 I was satisfied with the length of time spent on the course 5

Additional Information
Disclosures Human subjects: Consent was obtained or waived by all participants in this study. University of Saskatchewan Research Ethics Board issued approval BIO#18-46. In the opinion of the Research Ethics Board (REB) this submission is exempt from the requirement of the Research Ethics Board (REB) review and approval based on article 2.5 of the Tri-Council Policy Statement (TCPS2). Article 2.5 specifies "quality assurance and quality improvement studies, program evaluation activities, and performance reviews, or testing with normal educational requirements when used exclusively for assessment, management or improvement purposes do not constitute research for the purposes of this Policy, and do not fall within the scope of REB review". . Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: This work was supported by the University of Saskatchewan College of Medicine for program fees, as well as the Royal University Hospital Foundation (Saskatoon, Canada) Helping-Understanding-Giving (HUG) Grant, and the University of Saskatchewan's College of Medicine Dean's Summer Research Award. The authors have no other financial declarations. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.