"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead
Original article
peer-reviewed

Immersive Virtual Reality Medical Simulation: Autonomous Trauma Training Simulator



Abstract

Background

Medical and traumatic emergencies can be intimidating and stressful. This is especially true for early-career medical personnel. Training providers to respond effectively to medical emergencies before being confronted with a real scenario is limited by unnatural or high-cost training modalities that fail to realistically replicate the stress and gravity of real-world trauma management. Immersive virtual reality (IVR) may provide a unique training solution. 

Methods

We created a working group of 10 active duty or former military emergency medicine physicians and two technical experts. We hosted 10 meetings to facilitate the development process. The program was developed with financial support from the Telemedicine and Advanced Technology Research Center (TATRC), through the primary vendor Exonicus, Inc, with support from Anatomy Next Inc, and Kitware, Inc. Development was completed using an agile project management style, which allowed our team to review progress and provide immediate feedback on previous milestones throughout its completion. The working group completed the resulting four simulation scenarios to evaluate perceived realism and training potential. Finally, testing of the technology platform off the network in a deployed role 3 was conducted.

Results

Upon completion, we created four IVR scenarios based on the highest mortality battlefield injuries: hemorrhage, tension pneumothorax, and airway obstruction. The working group unanimously indicated a high level of realism and potential training usefulness. Throughout this process, there have been a number of lessons learned and we present those here to show what we have created as well as provide guidance to others creating IVR training solutions. 

Conclusion

Our team developed trauma scenarios that, to our knowledge, are the only IVR trauma scenarios to run autonomously without instructor input. Furthermore, we provide a potential template for the creation of future autonomous IVR training programs. This framework may offer a dynamic starting point as more teams seek to leverage the capabilities IVR offers. 

Introduction

Immersive virtual reality (IVR) can be highly effective as a medical simulation training platform [1-7]. Given recent advancements, this technology has become increasingly portable and visually realistic. While IVR technology appears to hold promise, there is a great deal to learn about the best way to functionally develop, implement, and share these training resources. Several commercial groups have created models that recreate current simulation lab environments with instructor input. While these systems increase training opportunities, decrease equipment needs, and offer broad potential, they still require a skilled trainer to ‘prompt the system’. Removing this limitation seems like a potential way to increase scalability. We are currently in the process of creating, to our knowledge, the only IVR simulator that would offer immediate autonomous feedback to users through both real-time patient physiologic responses and overall grading [8]. We will present phase 1 of this (multiphase) project. 

Materials & Methods

We created a working group of 10 active duty or former military emergency medicine physicians, and two technical experts who had programming and IVR project experience. We hosted 10 meetings to facilitate the development process (results). The project was completed at Madigan Army Medical Center in Tacoma, WA. The program was developed with financial support from the Telemedicine and Advanced Technology Research Center (TATRC), through the primary vendor Exonicus, Inc, with support from Anatomy Next Inc, and Kitware, Inc. Development was completed using an agile project management style, which allowed our team to review progress and provide immediate feedback on previous milestones throughout its completion. The working group completed the resulting four simulation scenarios to evaluate perceived realism and training potential. Finally, the technology platform was tested in a live, off the network, deployed environment in Iraq at a role 3 facility (deployed military hospital).

Results

Upon completion of phase 1, we have created four IVR scenarios based on the highest mortality battlefield injuries: hemorrhage, tension pneumothorax, and airway obstruction. Throughout this process, there have been a number of lessons learned. We present those here to show what we have created as well as provide guidance to others creating IVR training solutions (summary shown in Table 1). 

Steps (Approximate Order) Sample of Options
Select a training goal and simulation plan Procedural trainer or decision trainer? Guided process or free-for-all/sandbox mode? Instructor-less systems may fit more algorithmic processes/procedures
Select a virtual reality platform Microsoft Mixed Reality, Oculus Rift, HTC Vive, Magic Leap, phone-based system, many more
Select a physiology engine BioGears, Pulse, CAE, HuMoD, many more
Develop the case Based on prior decisions (above), available physiology engine, and specific capabilities of the selected immersive virtual reality platform, a case can be designed to meet your training goal
Create a master action list, grading scheme, and feedback plan Based on the case and learning objective, create a list of every potential action a user can make in the environment. These can be tracked for feedback/grading
Create a required room and 3D objects list Based on the case and required actions, the environment and three-dimensional objects list can be created
Visual, audio, and exam cues Based on the case, review audio/visual/exam cues the learner will need to make a decision or complete a procedure. Be aware, there are some limitations representing physical exam findings in virtual reality (for example, palpation)
Team members Determine what team members/additional characters will be required to complete the case/procedure. Nurse, assistant, medic, etc. Determine an interaction method, for example, point and click or verbal
User interface Determine how the learner will interact with their environment/objects/patient/lab results/orders. For example, clicking, direct movement, verbal, or using an object (tablet/computer)
User tutorial To work without an instructor, a guided walkthrough/tutorial is optimal
Gaze and location tracking Optional tracking systems are available to track gaze, distance traveled, and a multitude of other variables
Development and feedback plan Determine a method for providing the user with feedback on their performance. For example, ongoing during the case or in a summary at the end

Virtual reality platform

We reviewed the technical specifications of the current IVR platforms (Mobile, Microsoft Mixed Reality, HTC Vive), and augmented reality platforms (Hololens). A smaller focus group tried each platform and reviewed computing, graphics, network connectivity, and space requirements. Given its portability, graphics capabilities, and computing potential, we opted for primary development using Microsoft Mixed Reality.

Select a training goal/simulation plan

We initially sought to create a visually realistic training environment around one case. However, this seemed to limit immediate training benefit. Furthermore, the environment may be visually real, but most IVR platforms lack easy and scalable methods to change ambient temperature, moisture, or produce complex haptics. As we learned how to incorporate the physiology engine, we shifted towards a complex medical decision trainer. This could be placed in an endless number of environments and internally scaled to multiple patients in future iterations. The user is presented with an unstable trauma patient with a random injury. The physiology engine settings were selected to result in the patient’s death in 2-2.5 minutes if the player does not identify the injury and complete appropriate intervention. The player must keep the patient alive for a minimum of five minutes. Keeping the patient alive is the primary endpoint for the user. Additional factors, such as whether or not a complete assessment was performed, are tracked/graded, but do not affect the patient’s immediate survival. Standard trauma care actions are also available and tracked.

Selecting the case

Several individual cases, procedures, and environments were considered. However, selecting a single case or procedure seemed to drastically limit the scalability. We selected a generalizable trauma scenario for a few reasons. First, it allows multiple branch points to individual procedures (minisimulations) in future iterations. Second, several open-source physiology engines exist to run the physiology in these cases. The individual scenarios were further built to allow three progressive levels of consciousness if an injury is not addressed in time, and a failure state (death) that can be reversed if an injury is identified and treated.

The room and 3D objects

We sought to include every three-dimensional (3D) object that could be required in caring for a trauma patient, which totaled 36 items. The room was based on a 3D rendition of a standard military trauma bay.

Visual, audio, and exam cues

The selected injury scenario dictated visual/audio/exam cues (as in a traditional simulation).

Physiology engine

The incorporation of the physiology engine (Pulse) allowed us to develop a dynamic and more realistic simulation. Tying the simulation timeline to the physiology engine allows a player to see realistic vital sign changes, and complete the course of care as they would in a real patient scenario. It also allows for rapid case variations and randomization.

Team member interaction

Trauma management is a team sport. While a multiplayer/multidisciplinary approach is optimal, this again requires multiple skilled professionals to participate simultaneously. We sought to automate this process through two computer characters: a Nurse and a Medic. Commands are given vocally or through gaze-activated menus. 

Master action list/grading scheme/feedback system

A list of all the potential actions a player could make was developed (132), and each was tied to a specific outcome (injury treatment, lab availability, medication administration). These outcomes are injury and level of consciousness specific. Furthermore, each action was tied to a grade based on useful actions, neutral actions, and harmful actions. The Joint Trauma Committee Clinical Practice Guidelines and Advanced Trauma Life Support content were incorporated into the grading schematic.

User interface

The working group members have been end users of several computer training solutions, and sought to minimize technical frustration. The primary mode of interaction is using 3D objects to trigger animations. There are also multiple duplicate pathways, for example, starting intravenous access via a 3D object, voice command, or a menu selection.

User tutorial

It was readily apparent a thorough self-guided user tutorial would be necessary. We included key interventions and steps that would allow the user to ‘interact’ in the virtual environment.

Gaze/location tracking system

Given the user is completing actions in a digital world, it is quite easy to track multiple data points that may further relate to performance. We incorporated a gaze/location tracking system to allow for more analytics.

Upon completion of the trauma scenario creation, the working group unanimously indicated a high level of realism and potential training usefulness. The technology platform worked in a deployed environment without internet connectivity (Figure 1), further highlighting the capabilities of this autonomous IVR system for military training. 

Discussion

Our team developed four trauma scenarios that per our knowledge, after an extensive literature review, are the only IVR trauma scenarios to run autonomously without instructor input. Furthermore, we provide a potential template for the creation of future autonomous IVR training programs. This simulator, and IVR broadly, still has several limitations. First, the use of IVR can induce side effects (headaches, dizziness, and nausea). While the advancements in technology have greatly improved this, it is still an issue for some users. Second, it is challenging to create a seamless verbal feedback mechanism. Microsoft Mixed Reality does have voice recognition capabilities, but this produced mixed results when asking the simulated patient questions. Finally, the hardware and software are expensive. They both require initial development and upkeep. However, similar to computers, the cost of IVR systems and programming has been decreasing. Furthermore, these systems are still cheaper and more portable than most mannequin-based simulation systems. 

Conclusions

Overall, this pilot project helps reveal the broad potential IVR has for medical training through automated and instructorless scenarios. These and similar self-directed scenarios may create a scalable simulation platform, similar to flight simulation training. We feel this holds immense learning and training potential for medical education. Several challenges still exist including side effects and high costs for development/equipment. Furthermore, more research is needed around learning retention, optimal content, and delivery curriculums. The framework presented in this article may offer a dynamic starting point as more teams seek to leverage IVR for medical education.


References

  1. Chakravarthy B, ter Haar E, Bhat SS, McCoy CE, Denmark TK, Lotfipour S: Simulation in medical school education: review for emergency medicine. West J Emerg Med. 2011, 12:461-466. 10.5811%2Fwestjem.2010.10.1909
  2. Freeman KM, Thompson SF, Allely EB, Sobel AL, Stansfield SA, Pugh WM: A virtual reality patient simulation system for teaching emergency response skills to US Navy medical providers. Prehosp Disaster Med. 2001, 16:3-8. https://doi.org/10.1017/S1049023X00025462
  3. Hamza-Lup FG, Rolland JP, Hughes C: A distributed augmented reality system for medical training and simulation. Energy, Simulation-Training, Ocean Engineering and Instrumentation. 2018, 4:213-235. Accessed: August 9, 2019: https://arxiv.org/abs/1811.12815v1.
  4. McGrath JL, Taekman JM, Dev P, et al.: Using virtual reality simulation environments to assess competence for emergency medicine learners. Acad Emerg Med. 2018, 25:186-195. https://doi.org/10.1111/acem.13308
  5. Price MF, Tortosa DE, Fernandez-Pacheco AN, et al.: Comparative study of a simulated incident with multiple victims and immersive virtual reality. Nurse Educ Today. 2018, 71:48-53. https://doi.org/10.1016/j.nedt.2018.09.006
  6. Reis S, Guimarães P, Coelho F, Nogueira E, Coelho L: A framework for simulation systems and technologies for medical training. 2018 Global Medical Engineering Physics Exchanges/Pan American Health Care Exchanges (GMEPE/PAHCE). IEEE, Porto, Portugal ; 2018. 1-4. 10.1109/GMEPE-PAHCE.2018.8400757
  7. Sagalowsky ST, Wynter S-A, Auerbach M, Pusic MV, Kessler DO: Simulation-based procedural skills training in pediatric emergency medicine. Clin Pediatr Emerg Med. 2016, 17:169-178. https://doi.org/10.1016/j.cpem.2016.05.007
  8. Couperus K: 77 virtual reality trauma simulation: an immersive method to enhance medical personnel training and readiness. Ann Emerg Med. 2019, 74:30. https://doi.org/10.1016/j.annemergmed.2019.08.081
Original article
peer-reviewed

Immersive Virtual Reality Medical Simulation: Autonomous Trauma Training Simulator


Author Information

Kyle Couperus Corresponding Author

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Scott Young

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Ryan Walsh

Emergency Medicine, Vanderbilt University Medical Center, Nashville, USA

Christopher Kang

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Emergency Medicine, University of Washington, Seattle, USA

Military/Emergency Medicine, Uniformed Services University of the Health Sciences, Bethesda, USA

College of Medicine, Baylor University, Houston, USA

Carl Skinner

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Robyn Essendrop

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Kristin Fiala

Emergency Medicine, Darnall Army Medical Center, Fort Hood, USA

Jillian F. Phelps

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Zachary Sletten

Emergency Medicine, San Antonio Military Medical Center, San Antonio, USA

Matthew T. Esposito

Emergency Medicine, San Antonio Military Medical Center, San Antonio, USA

Jason Bothwell

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA

Chad Gorbatkin

Emergency Medicine, Madigan Army Medical Center, Tacoma, USA


Ethics Statement and Conflict of Interest Disclosures

Human subjects: Consent was obtained by all participants in this study. Madigan Army Medical Center IRB issued approval 901816. The investigators have adhered to the policies for protection of human subjects as prescribed in 45 CFR 46. Local IRB approval was obtained from the Madigan Army Medical Center IRB, protocol number 901816. . Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: This work was funded by the AMEDD Advanced Technology Initiative (AAMTI), through the Telemedicine and Advanced Technology Research Center (TATRC). Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Acknowledgements

The views expressed are those of the author(s) and do not reflect the official policy of the Department of the Army, the Department of Defense or the U.S. Government. This work was supported by the Telemedicine and Advanced Technology Research Center (TATRC) at the US Army Medical Research and Materiel Command. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the US Army


Original article
peer-reviewed

Immersive Virtual Reality Medical Simulation: Autonomous Trauma Training Simulator


Figures etc.

SIQ
6.5
RATED BY 2 READERS
CONTRIBUTE RATING

Scholary Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.