Remediation Strategies for Emergency Medicine Patient Care Milestones

Early identification and successful remediation of unachieved emergency medicine (EM) milestones are challenging for program directors. Residents who fail to achieve milestones in the expected time frame will have varied educational needs to course correct, dependent on the year of training, as well as the specific deficiencies to resolve. Experts from the Council of Residency Directors in Emergency Medicine (CORD-EM) Remediation Task Force (RTF) collaborated with the objective to create tools for identifying and remediating residents with deficiencies in patient care milestones (PCMs).


Introduction
As described by the Accreditation Council for Graduate Medical Education (ACGME), "Milestones are descriptors and targets for resident performance as a resident moves from entry into residency through graduation" [1][2]. As milestones are incorporated into resident assessment, some trainees may not achieve specific milestone levels in the expected time frame. These residents will require remediation to help them achieve training goals. Educators need tools to effectively identify these trainees so that any deficiencies may be addressed as soon as possible. Yet, there are very few assessment tools for emergency medicine (EM) milestones that have been previously validated, leaving residency leadership searching for appropriate tools and strategies. Once a resident is identified, a remediation plan must be developed and implemented; however, the best practices to remediate a deficiency are complex. Best practices differ based on the resident's year of training as well as the actual deficiency. Therefore, the remediation plans that are developed must be tailored to the individual needs of the resident based on the assessment of the program leadership and the Clinical Competency Committee (CCC) before they can be implemented.
Consider clinical scenarios that faculty might encounter: Resident X is a second-year emergency medicine (EM) resident in a three-year training program working in the Emergency Department (ED) in July. He presents a narrow differential diagnosis (DDx) without an appropriate treatment plan and reports that the patient may be discharged. When the faculty member evaluates the patient, he uncovers major discrepancies in the history and key physical examination (PE) findings that necessitate further evaluation and hospital admission. During the program's CCC meeting, other faculty note similar and consistent deficiencies in the resident's patient care. The resident falls short in several expected levels of the patient care The authors' group, comprised of EM residency program leadership with over 60 years of collective graduate medical education (GME) experience, focused on the PCMs. The PCM subgroup had telephone and email correspondence, as well as face-to-face meetings twice per year over a two-year period, to discuss the objectives and collaborate.
The first step was a literature search of the best practices for remediation of patient care (PC). PubMed and MEDLINE databases were used to search for literature pertaining to remediation of patient care milestones. For the PubMed database search, the following medical subject heading (MeSH) terms were used: "Education, Medical, Graduate" OR "Internship and Residency," OR "Clinical Competence," AND "Emergency Medicine," AND "Curriculum," AND "last 10 years." The following additional MeSH terms were used to search for literature pertaining to specific PCMs: "Resuscitation," "Diagnostic Imaging," "Diagnosis," "Physiologic Phenomena," "Airway Management," "Therapeutics," "Wounds and Injuries," and "Catheters." Similar terms were used in the MEDLINE data search. Only articles in the English language were considered for review. In addition, only articles pertaining to methods of improving patient care skills in postgraduate physician learners were considered for review and utilization. Since there is significant overlap in many aspects of patient care across all specialties, the articles were not limited to those pertaining to only emergency medicine. The search yielded 38 articles in total and the articles were screened for content that was focused on curricula for EM procedures, assessment tools, and resources used for the education of PCMs. A total of 17 articles were selected for review from the literature search; 14 of these articles were utilized in the creation of the PCM rubric. These 14 articles were divided into the following categories: simulation/task trainers (8), curriculum (2), assessment tools (3), and free open access medical education (1).
Next, the group worked to create a tool that would assist in the identification of residents in need of remediation of PCMs. The EM milestones list the standardized direct observation tool (SDOT) to assess milestone achievement. However, the previously developed emergency medicine SDOT is not milestone-based and may be difficult to translate when performing milestone evaluations. Therefore, the Patient Care Milestone Standardized Direct Observation Tools (PC-mSDOTs) was created to reflect the influence of the new milestones (m) on the SDOT.
Seven mSDOTs were developed for each EM training year as depicted in Table 1. The EM-3 and EM-4 years were combined into one mSDOT for use in either three-or four-year programs, as residents in their final year of training are expected to perform at the higher milestone levels. The evaluator is expected to indicate whether the level has been achieved, needs improvement, or was not observed. Faculty comments and review of the mSDOT with the resident in real time is expected. Residents may also provide comments.  The final step was the development of the PCM rubric. The ACGME EM Milestones were used as a guide to approaching remediation of each PCM at each level. Using the results of the literature search, as well as the combined program director experience with successful remediation practices from the subgroup, recommendations for potential remediation strategies were collated into the rubric. Only proficiency levels 1-4 were targeted, as level 5 represents a post-residency aspirational achievement.

Assessment and remediation of patient care milestones
The creation of the PC-mSDOT in conjunction with the remediation rubric provides a new resource for the early identification of residents in need of remediation, as well as strategies for the development and the implementation of a plan based on the CORD-EM RTF's best practices and expert consensus.

Early Identification: A New Assessment Tool -The PC-mSDOT
The assessment of resident performance can occur in or out of the clinical setting and can utilize various assessment methods, the combination of which provides different degrees of standardization [4]. Direct observation can provide valuable information regarding a resident's performance of PC. The SDOT was developed to obtain partial standardization via a structured observer assessment in the clinical setting and has been shown to have good inter-rater reliability [5][6]. By incorporating the PCMs into the SDOT and thus creating the PC-mSDOT, it is our hope that residency leadership may have a tool that will more readily indicate when a resident has deficiencies in these areas. The evaluator is expected to indicate whether the level has been achieved, needs improvement, or was not observed. Faculty comments and review of the mSDOT with the resident in real time is expected. Residents may also provide comments.
An example of the PC-mSDOT is depicted in Figure 1.

Formulating a Remediation Plan: Consensus Recommendations
Each resident has distinct strengths and weaknesses and some may struggle to progress to expected milestone levels at different points during training. Recognizing that there is no "one-size-fits-all" remediation curriculum, plans that are unique to the individual resident's deficiencies are needed. The literature search supported the use of simulation and task trainers in the education and evaluation of many of the PCMs [7][8][9][10][11][12][13][14]. Three articles had assessment tools for procedure-based PCMs [15][16][17]. There were two articles that were curriculum-based and one article that supported the use of free open access medical education as a means of increasing knowledge base for emergency procedures [18][19][20] Require the resident to describe the discharge plan, including patient or family concerns, safety issues, financial barriers, or reliability of compliance prior to discharging patients from the ED.
Require the resident to complete oral board cases that provide a range of acuity levels for disposition.
Engage in oral board review cases that involve communicating with a surrogate.
Require the resident to discharge standardized patients with a variety of issues while being observed.
PC8: Multi-tasking (Task-switching): Employs task switching in an efficient and timely manner in order to manage the ED.
Manages a single patient amidst distractions.
Task switches between different patients.
Employs task switching in an efficient and timely manner in order to manage multiple patients.
Employs task switching in an efficient and timely manner in order to manage the ED. Have the resident run multiple patient scenarios in the simulation lab.

PC8 SUGGESTED REMEDIATION METHODS
Have the resident engage in simulated encounters of multiple patients, give immediate feedback and monitor progress at regular intervals.
Engage in oral board style case review with multiple patient encounters.
Give the resident the opportunity to function as a "pre-attending," requiring them to manage the entire ED.
Utilize direct observation and redirect the resident as needed to prioritize correctly.
Engage in oral board style case review with multiple patient encounters.
Utilize direct observation and redirect the resident as needed to prioritize correctly.
Encourage direct observation and evaluation by faculty members.
Obtain and review productivity data (i.e., patients/hr) and compare this to peers or local/national expectations.
Utilize direct observation and redirect the resident as needed to prioritize correctly. Set a specific expectation with the resident that they will see "X" patients per hour or per shift.

PC9: General Approach to Procedures:
Performs the indicated procedure on all appropriate patients (including those who are uncooperative, at the extremes of age, hemodynamically unstable, and those who have multiple co-morbidities, poorly defined anatomy, high risk for pain or procedural complications, sedation requirement), takes steps to avoid potential complications, and recognizes the outcome and/or complications resulting from the procedure. The resident must then present the information to the attending prior to performing the procedure.
Have the resident prepare a summary of common procedures, indications, contraindications, anatomic landmarks, equipment, anesthetic, and procedural technique, and potential complications for common ED procedures to review with a mentor.
Have the resident prepare and present a summary of the backup strategies to employ for unsuccessful common ED procedures.
Have the resident review and present the interpretation of results of common diagnostic procedures.

PC10: Airway Management:
Performs airway management on all appropriate patients (including those who are uncooperative, at the extremes of age, hemodynamically unstable, and those who have multiple co-morbidities, poorly defined anatomy, high risk for pain or procedural complications, sedation requirement), takes steps to avoid potential complications, and recognize the outcome and/or complications resulting from the procedure.  Performs local anesthesia using appropriate doses of local anesthetic and appropriate technique to provide skin to subdermal anesthesia for procedures.
Knows the anatomic landmarks, indications, contraindications, potential complications and appropriate doses of local anesthetics used for regional anesthesia.
Performs patient assessment and discusses with the patient the most appropriate analgesic/sedative medication and administers in the most appropriate dose and route.
Performs pre-sedation assessment, obtains informed consent and orders appropriate choice and dose of medications for procedural sedation.
Obtains informed consent and correctly performs regional anesthesia.
Ensures appropriate monitoring of patients during procedural sedation.

PC11 SUGGESTED REMEDIATION METHODS:
Procedural competency forms, checklist assessment of procedure and simulation lab performance, global ratings, patient survey, chart review Utilize direct supervision of the resident discussing the administration of local anesthesia with a specified number of patients.
Have the resident review and discuss common analgesic and sedative medications.
Have the resident review and discuss common medications for procedural sedation.
Utilize direct supervision of the resident performing procedural sedation on a specified number of patients. Have the resident spend extra time in a suture lab.
Have the resident review procedure videos on regional anesthesia.
Observe the resident in the simulation lab practicing informed consent and pre-sedation conversations with mock patients.
Utilize direct supervision of the resident performing pre-sedation assessment on a specified number of patients and provide appropriate feedback.
Utilize direct supervision of the resident performing procedural sedation on a specified number of patients. Have the resident review procedural videos on emergency ultrasound.
Require the resident to perform a specified number of ultrasound exams under direct supervision.
Require the resident to perform the required number of studies.
Require the resident to perform a specified number of eFAST exams under direct supervision. Determines which wounds should not be closed primarily.
Describes the indications for and steps to perform an escharotomy. Identifies wounds that require antibiotics or tetanus prophylaxis.
Demonstrates appropriate use of consultants. Identifies wounds that may be high risk and require more extensive evaluation (example: x-ray, ultrasound, and/or exploration). Require the resident to review procedure videos on complex wound management.
Require the resident to review and discuss basic burn assessment with a mentor.
Require the resident to watch procedure videos on complex wound management.
Require the resident to review procedure videos on escharotomy.
Require the resident to watch procedure videos on wound management.
Require the resident to review and discuss burn management with a mentor.  Require the resident to review procedure videos on vascular access.
Require the resident to review procedure videos on the relevant procedures.
Require the resident to practice vascular access skills with task trainers and mannequins.
Require the resident to practice vascular access skills with task trainers and mannequins.
Require the resident to practice vascular access skills with task trainers and mannequins. The copyright owners grant third parties the right to use the Emergency Medicine Milestones on a non-exclusive basis for educational purposes.

Implementing a Remediation Plan: Combining Toolkit Options
The PC-mSDOT may be administered to all residents or residents who have been identified as having or potentially having PCM deficiencies. Administering the PC-mSDOT to all residents early in the academic year establishes a baseline, and deficiencies may be identified expeditiously. A second PC-mSDOT may then be administered later in the academic year to track the progress of the remediation. The PC-mSDOT may be used to assess performance in both the clinical and extra-clinical settings.
Using the PCM remediation rubric in Table 2, residency leadership may readily access suggested remediation methods when it is discovered that there are specific deficiencies that require additional resources. There are also suggested assessment methods linked to each PCM that may be used to identify deficiencies and track progress. Below each PCM level, there are remediation strategies and tools which may be incorporated into individualized plans. This may be used for residents with deficiencies at any point in their training.
Referring back to the resident scenarios presented in the Introduction and based on the RTF-PC toolkit, we offer sample remediation plans.
Resident X has deficiencies in PC2, PC4, and PC7 and does not meet level 1 for these PCMs. Applying the remediation rubric, the PD would refer to level 1 of the three individual PCMs and may develop a remediation plan to include: During clinical shifts for the EM block, Resident X will be required to: ▫ Work one-on-one with faculty who will review every history and PE with the resident; ▫ Have direct and immediate feedback from faculty with particular attention on the history and focused PE skills; ▫ Shadow the senior resident on shift while s/he performs a history and PE (to establish a successful frame of reference); ▫ Develop a list of at least five differential diagnoses for each patient encounter based on the likelihood of occurrence; ▫ Include clinical reasoning for the differential diagnoses during the oral case presentation; Resident remediation is prevalent in EM. In a survey of ACGME-accredited EM programs, Silverberg et al. found that 90% of program respondents had at least one resident on remediation within the previous three years [21]. The same study demonstrated that the prevalence of remediation in EM residencies is 4.4% with deficiencies in patient care being the second most common competency being remediated (46.6%) [21].
Among the challenges of remediation, PDs have difficulty with identifying residents in need of remediation, diagnosing the cause of their underlying deficiencies, and remediating them [22]. Residents failing to meet expectations may be identified in several ways, including the review of end-of-rotation evaluations, CCC meeting assessments, or a resident's semi-annual review. However, waiting to uncover issues during infrequently scheduled evaluations may lead to a delay in the identification of deficiencies. The literature supports that post-rotation assessments completed by faculty are not helpful in identifying those residents who are struggling [23]. Moreover, several studies have shown that informal emails, telephone calls, and hallway/"curbside conversations" (rather than standardized assessments) are more common methods to raise concerns about resident competency [24][25]. Our proposed PC-mSDOT provides a resource for the early identification of residents who are not achieving appropriate milestone levels for their year of training. Moreover, the PC-mSDOT may be utilized to assess the progress of a resident who is undergoing remediation.
Studies have also demonstrated that resident remediation requires substantial resources [26][27]. Many PDs recognize the growing need for remediation toolkits, resources, and best practices. Katz et al. published a novel approach to remediation using actual resident cases presented to a multidisciplinary panel of current and former program directors. This panel utilized a four-step approach to create an expert consensus to develop a remediation plan of action [28]. However, with the development and implementation of milestones, the need for specific tools for the assessment and remediation of milestone deficiencies has arisen. There have been several remediation strategies that have been published in the recent literature. Williamson et al. published remediation strategies for systems-based practice (SBP) and practice-based learning and improvement (PBLI) milestones that may be applied across all specialties [29]. Similarly, Regan et al. published remediation methods for deficiencies in the interpersonal and communication skills (ICS) and professionalism milestones that may be utilized by all specialties [30]. The milestones that are focused on SBP, PBLI, ICS, and professionalism are more easily generalized across specialties than the PCMs. Although there is some commonality to various aspects of PC, such as history and physical examination skills, there are many more facets that are specialty-specific. There are currently no published tools or strategies for remediation of EM PCMs. It is the authors' hope that the PC-mSDOT and the PCM remediation rubric offered by the CORD-RTF will be instrumental in assisting PDs in successful resident remediation.
The authors acknowledge that there are limitations to these remediation tools. First, the PC-mSDOT has yet to be validated. This is an ongoing process that clinician educators are currently working towards. Since there are currently no specific evidence-based "best practices" for remediation of PC, our toolkit was based on the expert consensus of the CORD RTF. Further review of the remediation outcomes will need to be tracked over time to establish best practices.

Conclusions
EM program leadership can use the PCM-mSDOTs to identify resident strengths and areas for improvement, track resident progress, and initiate remediation plans. The PCM remediation rubric may a useful tool to formulate an individualized remediation plan for any resident with deficiencies at various milestone levels.