The COVID-19 Elective for Pediatric Residents: Learning About Systems-Based Practice During a Pandemic

Background: The coronavirus disease 2019 (COVID-19) pandemic has prompted pediatric residency programs to adjust the delivery of educational curricula and to update content relevant to the pandemic. Objective: In this descriptive paper, we present how we rapidly developed and implemented a COVID-19 pandemic elective for pediatric residents. Methods: This curriculum was established at a single tertiary care children’s hospital in June 2020. We used the ADDIE (analysis, design, development, implementation, evaluation) framework to develop a two-week elective (30 hours) consisting of six flexibly scheduled modules. We administered post-elective surveys and exit interviews to solicit feedback to improve the elective and obtain effectiveness of our educational interventions. Results: We developed an asynchronous online COVID-19 Elective for Pediatric Residents. The curriculum modules focus on pathophysiology of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the disaster management ecosystem, simulation of clinical care, mental health ramifications, and public health consequences. We also include six in-situ experiences (visits to a drive-through COVID-19 testing site, testing laboratory and local public health department, a simulation of a critically ill child, and meetings with emergency managers and social workers) to solidify learning and allow for further reflection. To date, eight participants have taken the elective. All participants strongly agreed on a five-point Likert item survey that the elective enhanced their knowledge in current evidence-based literature for COVID-19, disaster preparedness, hospital response, management of the critically ill child, and mental and public health ramifications. All participants agreed this curriculum was relevant to and will change their practice. Conclusions: We demonstrate how a COVID-19 elective for pediatric residents could be quickly developed and implemented. The pilot results show that pediatric trainees value asynchronous learning, supplemented by relevant in-situ experiences. Moreover, these results suggest that this curriculum provides needed disaster response and resiliency education for pediatric residents.


Introduction
Medical education must adapt and restructure in response to disasters [1,2]. Pediatricians are infrequently frontline providers for such events and rarely receive training in disaster response and resiliency [3]. However, the coronavirus disease 2019  pandemic has forced some pediatricians to care for patients outside the traditional pediatric age range and try to learn and incorporate these concepts rapidly in real-time [4]. The Accreditation Council for Graduate Medical Education (ACGME) for Pediatrics Residency Programs states, "Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care…as well as the ability to call effectively on other resources to provide optimal health care" but does not directly address disaster response [5]. Recent literature suggests that "given the limited resources during the COVID-19 outbreak, trainees [must] learn creative problem-solving skills and gain a practical understanding of the relationships of value, cost and quality to medical care [6]." The concept of disaster preparedness, in the setting of general public concern as well as the current COVID-19 pandemic, is not directly addressed. This suggests that disaster training was not previously a priority for pediatricians and that there may be a lack of relevant education in pediatric residency programs.

Interventions
We created and launched an asynchronous online pandemic elective, supplemented by in-situ experiences. Due to the sparsity of publicly available pediatric disaster curricula, our elective presents an adaptive example of concepts borrowed from emergency medicine education and reframed with a pediatric focus [8,[13][14]. We used the ADDIE (analysis, design, development, implementation, evaluation) framework to inform our curriculum [15]. ADDIE is a systematic framework for creating effective instructional experiences targeting knowledge gaps. We specifically selected the ADDIE framework, not only because of its simplicity, but also because ADDIE models work best for curriculum geared towards producing specific learning outcomes and behavior changes, such as this curriculum. The ADDIE model has been used effectively in the military, health care, and education industries [16].

Analysis
Given ASPR's request to rapidly develop and implement a curriculum, we conducted an informal needs assessment among our research team and pediatric trainees, including pediatric emergency medicine fellows, which confirmed disaster medicine and resiliency concepts are not emphasized in pediatric residency curricula. The curriculum development team consisted of the subspecialty pediatric emergency medicine and critical care physicians, a disaster manager, a pediatric emergency medicine fellow, an associate residency program director, and an epidemiologist. This team met weekly during the ADDIE process to review learning objectives, assessment methods, and learning outcomes.
The curriculum development team decided that all curricular components would focus on treating the pediatric patient both medically and psychosocially during a crisis. To elicit changes in knowledge and future behavior, we developed an online post-elective survey for participants. In addition, we administered inperson exit interviews to solicit feedback to improve the elective.

Design
We adapted the content for this elective from disaster curriculums available for medical students and emergency medicine residents [8,[13][14]. In addition, we solicited input from a local physician disaster expert. The two-week elective (30 hours) consists of six modules addressing the needs assessment topics (Supplemental Content 1). Module 1 focuses on the pathophysiology of SARS-CoV2 in children and pregnant women. We included literature on pregnant women because upon delivery, their medical history is crucial to managing ill neonates. Module 2 introduces the disaster management ecosystem, resilience lifecycle components, and emergency manager's roles. Module 3 highlights the importance of incorporating the resiliency framework into disaster preparedness plans. Module 4 rehearses safe personal protective equipment donning and doffing and increases familiarity with management of COVID-19 related Multisystem Inflammatory Syndrome (MIS-C). Module 5 addresses mental health ramifications of disasters on children, healthcare providers, and the community. Module 6 focuses on the public health consequences of global disease.
Each module has specific objectives and includes multiple modes of content delivery, including videos, readings, lectures and podcasts. In addition, we include six in-situ experiences to solidify learning and allow for further reflection according to adult learning theories and principles [17,18]. These experiences are visits to a drive-through COVID-19 testing site, testing laboratory, and local public health department, along with high-fidelity simulation of a critically ill child, and meetings with emergency managers and social workers.

Development
Our curricular content is in a secure-online platform called The Learning Center (TLC). TLC is Nationwide Children's Hospital's learning management system. This system is where employees will access their yearly required curriculum as well as sign up for various instructor led trainings. TLC can be accessed remotely, hosts multiple resources in a centralized location, allows for administration of evaluations, and tracks progress.
A pediatric chief resident and physician with disaster medicine expertise reviewed each module for face and content validity. The curriculum development team met weekly during the ADDIE process to review learning objectives, assessment methods, and learning outcomes.
We developed online surveys examining the knowledge, attitudes, and future behavior changes of participants. We also developed an in person exit interview for participants to provide further feedback. The online survey and interview questions were piloted among an emergency medicine resident and a medicinepediatrics resident.
Results from these instruments are aggregated every three to four months and then the curriculum is updated on TLC based on recent literature and trainee feedback.

Implementation
Each resident had an orientation with the curriculum course director (MSI) and the curriculum administrator (DM) to go over the contents, expectations, deliverables/assignments and also how to use TLC. The curriculum went live in June 2020.

Outcomes measured
Throughout the course, pediatric residents completed knowledge-based tests and reflective writing assignments. After completion of all the modules, a one-on-one exit interview was conducted by the authors (DM, CBL). This information, in conjunction with a post-elective survey on changes in knowledge of disaster response and resiliency (Supplemental Content 2) were utilized in the evaluation framework for this elective. These instruments were internally developed but provides feedback on the elective in real time, allowing the authors to implement relevant changes in a timely manner.

Analysis of outcomes
For this small descriptive study, we analyzed survey data using counts and percentages, and conducted an in-depth thematic analysis of exit interview data [19].

IRB statement
This study was deemed exempt by the IRB at Nationwide Children's Hospital.

Results
To date, eight pediatric residents participated in this elective and no resident declined the exit interview. Participating residents were randomly split equally between postgraduate year (PGY) 2 and PGY3. Half of the participants identified as female. All participants completed the online portions of the elective and submitted written assignments and certifications. In addition, all participants strongly agreed on a five-point Likert scale that the elective enhanced their medical knowledge on current evidence-based literature for COVID-19, disaster preparedness, hospital response, management of the critically ill child, mental and public health ramifications. Every participant strongly agreed that the in-situ activities reinforced their knowledge. Finally, all participants strongly agreed that the elective enhanced their knowledge overall about pandemics and was relevant to and will change their clinical practice.
Three prominent themes emerged from the thematic analysis of exit interviews: the curriculum addressed core disaster medicine concepts, the curriculum provided practical knowledge on local resources to call upon during crises, and the multimodal learning allowed for flexibility in participating in educational activities. "I was in the dark, if (disaster resilience and how hospitals and public organizations respond to pandemics) was mentioned, it was briefly mentioned in PowerPoint that a guest speaker talked about. This is the first time that we talked about it. The FEMA courses were awesome -those were well structured and broke everything down. FEMA and emergency preparedness were some of the most valuable parts of the course." "I didn't know a whole lot about what went into (disaster resilience and response) -we had one day in med school -I didn't really know who all was in charge of responding to disasters -how they plan ahead of time -I definitely think I know a lot more now than I did before." "I think it's really shocking that we don't get more education about this type of thing in general in residency. I think it's important for us to know how the hospital responds to a pandemic and who/what goes into it. For example, the Emergency Coordinator told me that the Incident Commander for COVID-19 is a physician who wasn't aware of these things and some terminology before the pandemic. We don't get that sort of education and it is now more relevant than ever."

Knowledge on
Local Resources "The highlight was the Public Health Department [visit]. I was frustrated with myself for how long it took me to go there. The Public Health Department is astounding. The best way to say it -is that it is all inclusive. They have so many resources.
During the tour I saw so many people in the waiting room that are part of the patient base. I felt that I should have reached out sooner -to patients and families." "Parents' health is so important for the child's health -I could have directed them to the Public Health Department for resources. It is interesting how complex their system is -it works so well and is so embedded in the community. Once you get it, it's like, wow, you guys do so much." "I enjoyed talking to X from You Matter. How her practice has changed doing things remotely -along with coping skills she has been encouraging. (It was a) good experience."

Multimodal
Learning/Flexibility "The resources that were presented were very valuable -the disaster game -program through Hopkins -simulation from the ventilator. Learning FEMA's courses were good -interactive. Resources were good -nothing was too long -everything that was bulk reading was broken up which was good." "Liked the different modalities introduced like reading, watching videos, listening to podcasts. Definitely liked the number of resources, alternatives to reading articles which he liked." "Flexibility was nice, my schedule was different than what was reported, that was good to shift stuff around. Being able to pick the time of day."

Discussion
We demonstrated that a multimodal asynchronous electronic platform, in conjunction with targeted in-situ activities, leads to a reported understanding of core disaster medicine concepts for pediatric residents. Our curriculum incorporated adult learning theory principles of experiential and self-directed learning and delivers content in accordance with the ACGME's Systems-Based Practice Core Competency. The modular curriculum incorporates unique in-situ experiences that complement and enhance foundational knowledge and allows for real and simulated application of these knowledge gains [9,10,20].
Trainees provided positive feedback on the flexibility of scheduling in-person experiences to work around the quarantine limitations. The trainees completed the online modules and, when necessary, scheduled the in-person activities at a later time. The COVID-19 pandemic has required considerable flexibility in the provision of education and training, particularly for residents who must self-quarantine if ill or exposed to high-risk patients [21]. The pandemic has also impacted pediatric volumes at most hospitals which has implications for education, making this asynchronous novel curriculum particularly suitable in these settings. Furthermore, providing engaging curricular access during a time of isolation decreases stress and anxiety [22].
The primary limitation of this study is the small sample size as the elective is new and limited to one institution. However, our curriculum can be easily adapted for other residency programs and disasters. Modules 2, 3, 5, and 6 address disaster curricula, not specific to COVID-19, and Modules 1 and 4 could easily be interchanged for other illnesses/disasters. Dissemination of these results may inform other programs trying to implement similar educational interventions.

Conclusions
This descriptive study shows how The COVID-19 Elective for Pediatric Residents allows for asynchronous learning, supplemented by relevant in-situ experiences, and provides needed disaster response and resiliency education for pediatric residents. Our results highlight the lack of disaster medicine and hospital response in pediatric residency curriculum, particularly during a time when all physicians need to be knowledgeable and prepared to act on these concepts. This unique curriculum can be modified to any pandemic or global disaster and moreover, to any residency training program. In addition, our continual assessment of the curriculum has enabled real-time modifications and adaptations of course content to learner's needs. We plan to frequently improve this course and assess long-term knowledge retention. Although further research in teaching pediatric residents disaster response and resilience in a systems-based practice manner is warranted, this well-received curriculum may be a first step in addressing this knowledge gap for pediatric trainees. 1. Understand the transmission of coronavirus and be able to describe three detection methods.
2. Describe the five key clinical characteristics of COVID-19 in newborns, infants and children.
3. Understand the management of infants born to mothers with COVID-19 in the hospital setting.

OBJECTIVE 1: OVERVIEW OF CORONAVIRUS (2 hours)
Welcome to OpenWHO, the World Health Organization's first online, interactive platform. For this portion of Module, you will complete four parts in the course titled "Emerging respiratory viruses, including COVID-19: methods for detection, prevention, response and control.
You will need to create an account (free) and complete the "required" components of the registration. This can be done at the following link: https://openwho.org/account/new The four educational modules are as follows and can be accessed through the following link:  (2 hours) Read the following articles and complete the assessment quiz that is located in The Learning Center.

Module 2 Objectives
The goal of this module is to provide the learner with an understanding of the disaster management strategies utilized to identify and manage risks in both the public and private sectors, in addition to creating a resilience foundation that will serve the learner throughout their career.
1. Understand the major components of the resilience life-cycle and the associated techniques used by emergency managers to identify and manage risks for an organization/jurisdiction.

2.
Recognize the mechanisms within the preparedness cycle and the methods used to manage risks.
3. Understand the fundamentals of the response system and structure utilized by the public and private sectors to manage incidents, emergencies, crises and disasters.

Objective
This module focuses on reviewing and rehearsing safe personal protective equipment (PPE) donning and doffing, increasing familiarity with mechanical ventilators, and working through the recognition and stabilization of patients presenting with COVID-19 related Multi-system Inflammatory Syndrome (MIS-C). By the end of Module #4, the trainee should be able to do the following: 1. Demonstrate safe donning and doffing of PPE.
2. Describe five key features of a mechanical ventilator. The cognitive component of this objective has 3 modules. The first two combined will take you about an hour. Please allow yourself 4 hours for the third module involving the virtual ventilator simulator. This is an animated interactive tutorial that includes self-paced exercises and tests. You may need to be on the NCH campus to access the ventilator simulator or use Google Chrome.

Module 5 Objective
This module focuses on the mental health ramifications of COVID-19 and similar disasters on the children, healthcare providers and the community at large. This module also lists various mental health support resources. By the end of this module, the trainee should be able to: 1. Describe three way in which global disasters and pandemics impact community mental health.
2. Describe three ways in which global disasters and pandemics impact physician mental health.
3. Identify resources available in Central Ohio to provide mental health support for the community.
4. Access resources available at NCH that support health care providers with regards to mental health concerns.

OBJECTIVE 1: GLOBAL MENTAL HEALTH IMPACT OF THE COVID-19 PANDEMIC (4-5 hours)
Although it is only starting to be explored, it is clear that COVID-19 and the responses to it have caused major disruptions to the lives of billions of people around the world. It is also evident that the mental health impact of this virus will be broad and long-lived. Below are some readings that will provide you with an overview of this topic. We also encourage you to find additional relevant literature, as this topic will be evolving rapidly for months (and possibly years). It is well documented that health care providers may experience psychological trauma from their experiences at work, especially while caring for patients with COVID-19. During times where stressors are severe and prolonged, the risk of mental health deterioration for providers increases. Resources are available, but may not be widely known. Furthermore, providers may not take advantage of the resources that exist. We have provided readings below and again, this is an evolving topic, so we encourage you to find additional resources.

Interagency Standing
Reading/Video There has been an increasing emphasis on pediatric mental health over recent years leading to the development of some pediatric-specific mental health resources. However, awareness and access are still considerably less than they should be, and as demonstrated above, mental health issues are becoming even more pressing during the COVID-19 pandemic.

Final Assignment
There are four options to complete the assessment for this module. Please complete one of them and email it to the course directors.
Option 1: Complete Psychological First Aid training at https://learn.nctsn.org/enrol/index.php?id=38. You will need to create an account and the training takes about 6 hours. This is for learners particularly interested in participating in the mental health response to a crisis. Proof of completion will be accepted as the assessment for this module.
Option 2: Design a mental health crisis response plan for either patients or providers in a particular context (for example, for patients in an NCH primary care clinic). As a guideline, this should be about 1 page, but use what space you need.
Option 3: Select an available mental health resource and design an informative means of increasing awareness of it. This may take the form of a poster, press release, podcast, etc. Be creative.

Please answer the following questions on your beliefs about the elective.
a. The Pandemic Elective enhanced my knowledge overall about pandemics.
b. The learning center online platform was easy to navigate.
c. Information presented in this curriculum was relevant to my clinical practice.
d. Information presented in this curriculum will change my clinical practice.
(Respondents answered using the following options: Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree) 6. Please list two strengths of this elective. (Forced free response)

Please list two ways in which this elective can be improved. (Forced free response)
Please answer these basic demographic questions.

What gender do you identify with?
Female

Male
Prefer not to answer 9. What is your training program at Nationwide Children's Hospital?