Early identification and successful remediation of unachieved emergency medicine (EM) milestones are challenging for program directors. Residents who fail to achieve milestones in the expected time frame will have varied educational needs to course correct, dependent on the year of training, as well as the specific deficiencies to resolve. Experts from the Council of Residency Directors in Emergency Medicine (CORD-EM) Remediation Task Force (RTF) collaborated with the objective to create tools for identifying and remediating residents with deficiencies in patient care milestones (PCMs).
As described by the Accreditation Council for Graduate Medical Education (ACGME), “Milestones are descriptors and targets for resident performance as a resident moves from entry into residency through graduation” [1-2]. As milestones are incorporated into resident assessment, some trainees may not achieve specific milestone levels in the expected time frame. These residents will require remediation to help them achieve training goals. Educators need tools to effectively identify these trainees so that any deficiencies may be addressed as soon as possible. Yet, there are very few assessment tools for emergency medicine (EM) milestones that have been previously validated, leaving residency leadership searching for appropriate tools and strategies. Once a resident is identified, a remediation plan must be developed and implemented; however, the best practices to remediate a deficiency are complex. Best practices differ based on the resident's year of training as well as the actual deficiency. Therefore, the remediation plans that are developed must be tailored to the individual needs of the resident based on the assessment of the program leadership and the Clinical Competency Committee (CCC) before they can be implemented.
Consider clinical scenarios that faculty might encounter: Resident X is a second-year emergency medicine (EM) resident in a three-year training program working in the Emergency Department (ED) in July. He presents a narrow differential diagnosis (DDx) without an appropriate treatment plan and reports that the patient may be discharged. When the faculty member evaluates the patient, he uncovers major discrepancies in the history and key physical examination (PE) findings that necessitate further evaluation and hospital admission. During the program’s CCC meeting, other faculty note similar and consistent deficiencies in the resident’s patient care. The resident falls short in several expected levels of the patient care milestones (PCMs), leaving the CCC wondering about effective remediation strategies to implement at this resident’s stage of training, as well as opportunities for earlier identification of deficiencies during training.
A second-year EM resident (Resident Y) has just evaluated a patient with a history of coronary artery disease, hypertension, and myocardial infarction. The patient presented to the ED with chest pain, marked hypertension, and new T-wave inversions. The nurse asks why the patient has been admitted to an unmonitored bed. The resident says it is because “his vital signs were stable.” Later that day, this resident tells you that he wants to discharge a patient with new-onset diabetes. However, the patient has no health insurance, no means of obtaining medication, and no follow-up physician.
Although Hauer et al. described a general approach to the remediation of physician performance deficits, it can be difficult to translate the deficient skill into a specific behavior to be targeted in a remediation plan . With the adoption of the milestones, new remediation tools are required to address milestone-based deficiencies and to craft milestone-based remediation plans.
The objective of this project was to determine the best practices for remediation and create tools for identification, assessment, and strategies for remediation of deficiencies in patient care milestones.
Materials & Methods
The Council of Residency Directors in Emergency Medicine-Remediation Task Force (CORD-EM RTF) is comprised of residency leadership from EM training programs all across the United States. The CORD-EM RTF was divided into four working subgroups based on the six core-competencies: (1) patient care, (2) medical knowledge, (3) professionalism and interpersonal communication skills, and (4) practice-based learning and improvement and systems-based practice. Each of the subgroups was tasked with focusing on the stated objective: determination of best practices for remediation for each of the given competencies and creation of a toolkit that program directors can utilize to identify and remediate residents.
The authors’ group, comprised of EM residency program leadership with over 60 years of collective graduate medical education (GME) experience, focused on the PCMs. The PCM subgroup had telephone and email correspondence, as well as face-to-face meetings twice per year over a two-year period, to discuss the objectives and collaborate.
The first step was a literature search of the best practices for remediation of patient care (PC). PubMed and MEDLINE databases were used to search for literature pertaining to remediation of patient care milestones. For the PubMed database search, the following medical subject heading (MeSH) terms were used: "Education, Medical, Graduate" OR "Internship and Residency," OR "Clinical Competence," AND "Emergency Medicine," AND "Curriculum," AND "last 10 years." The following additional MeSH terms were used to search for literature pertaining to specific PCMs: "Resuscitation," "Diagnostic Imaging," "Diagnosis," "Physiologic Phenomena," "Airway Management," "Therapeutics," "Wounds and Injuries," and "Catheters." Similar terms were used in the MEDLINE data search. Only articles in the English language were considered for review. In addition, only articles pertaining to methods of improving patient care skills in postgraduate physician learners were considered for review and utilization. Since there is significant overlap in many aspects of patient care across all specialties, the articles were not limited to those pertaining to only emergency medicine. The search yielded 38 articles in total and the articles were screened for content that was focused on curricula for EM procedures, assessment tools, and resources used for the education of PCMs. A total of 17 articles were selected for review from the literature search; 14 of these articles were utilized in the creation of the PCM rubric. These 14 articles were divided into the following categories: simulation/task trainers (8), curriculum (2), assessment tools (3), and free open access medical education (1).
Next, the group worked to create a tool that would assist in the identification of residents in need of remediation of PCMs. The EM milestones list the standardized direct observation tool (SDOT) to assess milestone achievement. However, the previously developed emergency medicine SDOT is not milestone-based and may be difficult to translate when performing milestone evaluations. Therefore, the Patient Care Milestone Standardized Direct Observation Tools (PC-mSDOTs) was created to reflect the influence of the new milestones (m) on the SDOT.
Seven mSDOTs were developed for each EM training year as depicted in Table 1. The EM-3 and EM-4 years were combined into one mSDOT for use in either three- or four-year programs, as residents in their final year of training are expected to perform at the higher milestone levels. The evaluator is expected to indicate whether the level has been achieved, needs improvement, or was not observed. Faculty comments and review of the mSDOT with the resident in real time is expected. Residents may also provide comments.
The final step was the development of the PCM rubric. The ACGME EM Milestones were used as a guide to approaching remediation of each PCM at each level. Using the results of the literature search, as well as the combined program director experience with successful remediation practices from the subgroup, recommendations for potential remediation strategies were collated into the rubric. Only proficiency levels 1-4 were targeted, as level 5 represents a post-residency aspirational achievement.
Assessment and remediation of patient care milestones
The creation of the PC-mSDOT in conjunction with the remediation rubric provides a new resource for the early identification of residents in need of remediation, as well as strategies for the development and the implementation of a plan based on the CORD-EM RTF's best practices and expert consensus.
Early Identification: A New Assessment Tool – The PC-mSDOT
The assessment of resident performance can occur in or out of the clinical setting and can utilize various assessment methods, the combination of which provides different degrees of standardization . Direct observation can provide valuable information regarding a resident’s performance of PC. The SDOT was developed to obtain partial standardization via a structured observer assessment in the clinical setting and has been shown to have good inter-rater reliability [5-6]. By incorporating the PCMs into the SDOT and thus creating the PC-mSDOT, it is our hope that residency leadership may have a tool that will more readily indicate when a resident has deficiencies in these areas. The evaluator is expected to indicate whether the level has been achieved, needs improvement, or was not observed. Faculty comments and review of the mSDOT with the resident in real time is expected. Residents may also provide comments.
An example of the PC-mSDOT is depicted in Figure 1. The full complement of PC-mSDOTs have been posted on the Council of Residency Directors (CORD) website for use by all EM residency programs and may be accessed through the following link: http://www.cordem.org/resources/residency-management/cord-standardized-assessment-methods/
Formulating a Remediation Plan: Consensus Recommendations
Each resident has distinct strengths and weaknesses and some may struggle to progress to expected milestone levels at different points during training. Recognizing that there is no “one-size-fits-all” remediation curriculum, plans that are unique to the individual resident’s deficiencies are needed. The literature search supported the use of simulation and task trainers in the education and evaluation of many of the PCMs [7-14]. Three articles had assessment tools for procedure-based PCMs [15-17]. There were two articles that were curriculum-based and one article that supported the use of free open access medical education as a means of increasing knowledge base for emergency procedures [18-20]. The Patient Care Remediation Task Force (PC-RTF) created a compilation of consensus remediation practices utilizing documented methodology from our literature search and personal experience that is linked to levels 1 - 4 of 14 (Table 2).
Implementing a Remediation Plan: Combining Toolkit Options
The PC-mSDOT may be administered to all residents or residents who have been identified as having or potentially having PCM deficiencies. Administering the PC-mSDOT to all residents early in the academic year establishes a baseline, and deficiencies may be identified expeditiously. A second PC-mSDOT may then be administered later in the academic year to track the progress of the remediation. The PC-mSDOT may be used to assess performance in both the clinical and extra-clinical settings.
Using the PCM remediation rubric in Table 2, residency leadership may readily access suggested remediation methods when it is discovered that there are specific deficiencies that require additional resources. There are also suggested assessment methods linked to each PCM that may be used to identify deficiencies and track progress. Below each PCM level, there are remediation strategies and tools which may be incorporated into individualized plans. This may be used for residents with deficiencies at any point in their training.
Referring back to the resident scenarios presented in the Introduction and based on the RTF-PC toolkit, we offer sample remediation plans.
Resident X has deficiencies in PC2, PC4, and PC7 and does not meet level 1 for these PCMs. Applying the remediation rubric, the PD would refer to level 1 of the three individual PCMs and may develop a remediation plan to include:
During clinical shifts for the EM block, Resident X will be required to:
▫ Work one-on-one with faculty who will review every history and PE with the resident;
▫ Have direct and immediate feedback from faculty with particular attention on the history and focused PE skills;
▫ Shadow the senior resident on shift while s/he performs a history and PE (to establish a successful frame of reference);
▫ Develop a list of at least five differential diagnoses for each patient encounter based on the likelihood of occurrence;
▫ Include clinical reasoning for the differential diagnoses during the oral case presentation;
▫ Review available resources in the department and describe the discharge rationale and plan for each patient.
At the end of the EM block, Resident X will:
▫ Be reevaluated by faculty using the PC-mSDOT either in the clinical setting or extra-clinical setting with an objective structured clinical exam (OSCE) or a simulation exercise.
Have shift evaluations been reviewed with the PD or another member of the residency leadership?
Resident Y has deficiencies with at least three PCMs (PC4, PC6, and PC7). The remediation plan for this resident may include:
During clinical shifts, Resident Y will be required to:
▫ Describe the reevaluation and discharge plan for all patients prior to discharge;
▫ “Check-in” with supervising faculty and residents at defined intervals during shifts to report the progress of therapeutic interventions and the ED course;
▫ Perform frequent rounds on patients to ensure that therapeutic interventions and care plans are executed in a timely fashion;
▫ Discuss clinical reasoning for differential diagnosis during case presentations;
▫ Describe discharge plans for patients, including acknowledgment of patient and/or family concerns, safety issues, financial, or compliance barriers;
▫ Personally discharge 10 patients and review medications, follow-up information, and return precautions under direct faculty observation;
▫ Personally schedule follow-up appointments for a specific number of discharged patients.
Outside of the clinical setting, Resident Y will be required to:
▫ Attend biweekly simulation and oral board sessions that require a reassessment of interventions, disposition, and discharge planning. Designated faculty will supervise these sessions and immediate direct feedback will be given to the resident.
At the end of the remediation period, the resident will:
▫ Be reevaluated by faculty using the PC-mSDOT either in the clinical setting or extra-clinical setting with an OSCE or a simulation exercise;
▫ Have shift evaluations reviewed with the PD or another member of the residency leadership;
▫ Have in-training exam scores closely monitored to assess medical knowledge.
Resident remediation is prevalent in EM. In a survey of ACGME-accredited EM programs, Silverberg et al. found that 90% of program respondents had at least one resident on remediation within the previous three years . The same study demonstrated that the prevalence of remediation in EM residencies is 4.4% with deficiencies in patient care being the second most common competency being remediated (46.6%) .
Among the challenges of remediation, PDs have difficulty with identifying residents in need of remediation, diagnosing the cause of their underlying deficiencies, and remediating them . Residents failing to meet expectations may be identified in several ways, including the review of end-of-rotation evaluations, CCC meeting assessments, or a resident’s semi-annual review. However, waiting to uncover issues during infrequently scheduled evaluations may lead to a delay in the identification of deficiencies. The literature supports that post-rotation assessments completed by faculty are not helpful in identifying those residents who are struggling . Moreover, several studies have shown that informal emails, telephone calls, and hallway/“curbside conversations” (rather than standardized assessments) are more common methods to raise concerns about resident competency [24-25]. Our proposed PC-mSDOT provides a resource for the early identification of residents who are not achieving appropriate milestone levels for their year of training. Moreover, the PC-mSDOT may be utilized to assess the progress of a resident who is undergoing remediation.
Studies have also demonstrated that resident remediation requires substantial resources [26-27]. Many PDs recognize the growing need for remediation toolkits, resources, and best practices. Katz et al. published a novel approach to remediation using actual resident cases presented to a multidisciplinary panel of current and former program directors. This panel utilized a four-step approach to create an expert consensus to develop a remediation plan of action . However, with the development and implementation of milestones, the need for specific tools for the assessment and remediation of milestone deficiencies has arisen. There have been several remediation strategies that have been published in the recent literature. Williamson et al. published remediation strategies for systems-based practice (SBP) and practice-based learning and improvement (PBLI) milestones that may be applied across all specialties . Similarly, Regan et al. published remediation methods for deficiencies in the interpersonal and communication skills (ICS) and professionalism milestones that may be utilized by all specialties . The milestones that are focused on SBP, PBLI, ICS, and professionalism are more easily generalized across specialties than the PCMs. Although there is some commonality to various aspects of PC, such as history and physical examination skills, there are many more facets that are specialty-specific. There are currently no published tools or strategies for remediation of EM PCMs. It is the authors’ hope that the PC-mSDOT and the PCM remediation rubric offered by the CORD-RTF will be instrumental in assisting PDs in successful resident remediation.
The authors acknowledge that there are limitations to these remediation tools. First, the PC-mSDOT has yet to be validated. This is an ongoing process that clinician educators are currently working towards. Since there are currently no specific evidence-based “best practices” for remediation of PC, our toolkit was based on the expert consensus of the CORD RTF. Further review of the remediation outcomes will need to be tracked over time to establish best practices.
EM program leadership can use the PCM-mSDOTs to identify resident strengths and areas for improvement, track resident progress, and initiate remediation plans. The PCM remediation rubric may a useful tool to formulate an individualized remediation plan for any resident with deficiencies at various milestone levels.
- The Emergency Medicine Milestones Project. (2015). Accessed: May 6, 2017: http://www.acgme.org/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf .
- Beeson MS, Carter WA, Christopher TA, et al.: The development of the emergency medicine milestones. Acad Emerg Med. 2013, 20:724-29. 10.1111/acem.12157
- Hauer KE, Ciccone A, Henzel TR, et al.: Remediation of the deficiencies of physicians across the continuum from medical school to practice: a thematic review of the literature. Acad Med. 2009, 84:1822-32. 10.1097/ACM.0b013e3181bf3170
- Ilgen JS, Humbert AJ, Kuhn G, et al.: Assessing diagnostic reasoning: a consensus statement summarizing theory, practice, and future needs. Acad Emerg Med. 2012, 19:1454-61. 10.1111/acem.12034
- Shayne P, Gallahue F, Rinnert S, Anderson CL, Hern G, Katz E; CORD SDOT Study Group: Reliability of a core competency checklist assessment in the emergency department: the Standardized Direct Observation Assessment Tool. Acad Emerg Med. 2006, 13:727-32. 10.1197/j.aem.2006.01.030
- LaMantia J, Kane B, Yarris L, et al.: Real-time inter-rater reliability of the Council of Emergency Medicine residency directors standardized direct observation assessment tool. Acad Emerg Med. 2009, 16:S51-57. 10.1111/j.1553-2712.2009.00593.x
- Dull KE, Bachur RG: Simulation in the pediatric emergency department. Clin Pediatr (Phila). 2012, 51:711-17. 10.1177/0009922811420716
- Lee MO, Brown LL, Bender J, Machan JT, Overly FL: A medical simulation-based educational intervention for emergency medicine residents in neonatal resuscitation. Acad Emerg Med. 2012, 19:577-85. 10.1111/j.1553-2712.2012.01361.x
- Auerbach M, Kessler D, Foltin JC: Repetitive pediatric simulation resuscitation training. Pediatr Emerg Care. 2011, 27:29-31. 10.1097/PEC.0b013e3182043f3b
- Girzadas DV Jr, Antonis MS, Zerth H, Lambert M, Clay L, Bose S, Harwood R: Hybrid simulation combining a high fidelity scenario with a pelvic ultrasound task trainer enhances the training and evaluation of endovaginal ultrasound skills. Acad Emerg Med. 2009, 16:429-35. 10.1111/j.1553-2712.2009.00399.x
- Petrosoniak A, Ryzynski A, Lebovic G, Woolfrey K: Cricothyroidotomy In Situ Simulation Curriculum (CRIC Study): Training residents for rare procedures. Simul Healthc. 2017, 12:76-82. 10.1097/SIH.0000000000000206
- Pywell MJ, Evgeniou E, Highway K, Pitt E, Estela CM: High fidelity, low cost moulage as a valid simulation tool to improve burns education. Burns. 2016, 42:844-52. 10.1016/j.burns.2015.12.013
- Wadman MC, Lomneth CS, Hoffman LH, Zeger WG, Lander L, Walker RA: Assessment of a new model for femoral ultrasound-guided central venous access procedural training: a pilot study. Acad Emerg Med. 2010, 17:88-92. 10.1111/j.1553-2712.2009.00626.x
- Werner HC, Vieira RL, Rempell RG, Levy JA: An educational intervention to improve ultrasound competency in ultrasound-guided central venous access. Pediatr Emerg Care. 2016, 32:1-5. 10.1097/PEC.0000000000000664
- Hartman N, Wittler M, Askew K, Hiestand B, Manthey D: Validation of a performance checklist for ultrasound-guided internal jugular central lines for use in procedural instruction and assessment. Postgrad Med J. 2017, 93:67-70. 10.1136/postgradmedj-2015-133632
- Schmidt JN, Kendall J, Smalley C: Competency assessment in senior emergency medicine residents for core ultrasound skills. West J Emerg Med. 2015, 16:923-26. 10.5811/westjem.2015.9.28587
- Rice J, Crichlow A, Baker M, Regan L, Dodson A, Hsieh YH, Omron R: An assessment tool for the placement of ultrasound-guided peripheral intravenous access. J Grad Med Educ. 2016, 8:202-207. 10.4300/JGME-D-15-00298.1
- Motov SM, Marshall JP: Acute pain management curriculum for emergency medicine residency programs. Acad Emerg Med. 2011, 18:S87-91. 10.1111/j.1553-2712.2011.01069.x
- Friedman D, Zaveri P, O'Connell K: Pediatric mock code curriculum: improving resident resuscitations. Pediatr Emerg Care. 2010, 26:490-94. 10.1097/PEC.0b013e3181e5bf34
- Joshi N, Morley EJ, Taira T, Branzetti J, Grock A: ALiEM Blog and Podcast Watch: procedures in emergency medicine. West J Emerg Med. 2017, 18:1128-34. 10.5811/westjem.2017.8.34844
- Silverberg M, Weizberg M, Murano T, Smith JL, Burkhardt JC, Santen SA: What is the prevalence and success of remediation of emergency medicine residents?. West J Emerg Med. 2015, 16:839-44. 10.5811/westjem.2015.9.27357
- Torbeck L, Wrightson AS: A method for defining competency-based promotion criteria for family medicine residents. Acad Med. 2005, 80:832-39. 10.1097/00001888-200509000-00010
- Guerrasio J, Cumbler E, Trosterman A, Wald H, Brandenburg S, Aagaard E: Determining need for remediation through postrotation evaluations. J Grad Med Ed. 2012, 4:47-51. 10.4300/JGME-D-11-00145.1
- Yao DC, Wright SM: National survey of internal medicine residency program directors regarding problem residents. JAMA. 2000, 284:1099-104. 10.1001/jama.284.9.1099
- Sullivan C, Murano T, Comes J, Smith JL, Katz ED: Emergency medicine residency directors’ perceptions on professionalism: a Council of Emergency Medicine Residency Directors survey. Acad Emerg Med. 2011, 18:S97-103. 10.1111/j.1553-2712.2011.01186.x
- Dupras DM, Edson RS, Halvorsen AJ, Hopkins RH Jr, McDonald FS: "Problem residents": prevalence, problems and remediation in the era of core competencies. Am J Med. 2012, 125:421-25. 10.1016/j.amjmed.2011.12.008
- Guerrasio J, Garrity MJ, Aagaard EM: Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006-2012. Acad Med. 2014, 89:352-58. 10.1097/ACM.0000000000000122
- Katz ED, Goyal DG, Char D, Coopersmith CM, Fried ED: A novel concept in residency education: case-based remediation. J Emerg Med. 2013, 44:493-98. 10.1016/j.jemermed.2012.09.031
- Williamson K, Moreira M, Quattromani E, Smith JL: Remediation strategies for system-based practice and practice-based learning and improvement milestones. J Grad Med Ed. 2017, 9:290-93. 10.4300/JGME-D-16-00334.1
- Regan L, Hexom B, Nazario S, Chinai SA, Visconti A, Sullivan C: Remediation methods for milestones related to interpersonal and communication skills and professionalism. J Grad Med Ed. 2016, 8:18-23. 10.4300/JGME-D-15-00060.1
Remediation Strategies for Emergency Medicine Patient Care Milestones
Ethics Statement and Conflict of Interest Disclosures
Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.
We would like to acknowledge and thank Drs. Bonnie Kaplan, Sara Krzyzaniak, and David Gordon for their contribution to the development of the PC-mSDOTs.
Cite this article as:
Murano T, Smith J L, Weizberg M (November 07, 2018) Remediation Strategies for Emergency Medicine Patient Care Milestones. Cureus 10(11): e3557. doi:10.7759/cureus.3557
Received by Cureus: October 19, 2018
Peer review began: October 26, 2018
Peer review concluded: November 05, 2018
Published: November 07, 2018
© Copyright 2018
Murano et al. This is an open access article distributed under the terms of the Creative Commons Attribution License CC-BY 3.0., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.