Abstract
Background
The Accreditation Council for Graduate Medical Education (ACGME) initiated Resident Subspecialty Milestones in 2015 to assess personal development of subspecialty residents (fellows). Many of the advanced level training milestones are targeted at professionalism and team communication, which can be challenging to assess on a daily basis due to limitations such as clinical demands and time. As a result, feedback is often sporadic or overlooked. (1) The use of simulation for objective structured clinical exams (OSCEs) is a well-developed tool in undergraduate medical education. However, there is limited data on the use of OSCEs at the fellow level to assess professionalism. (2)
Objective
Assess the ability of an OSCE to evaluate ACGME milestones objectively for Pediatric Critical Care and Pediatric Anesthesia fellows.
Design/ Methods:
Each sub-specialty group developed 1 specialty-specific scenario per milestone. The following milestones were chosen for the Pediatric Critical Care fellows: Transfer of care with seamless transitions, Work in inter-professional teams, High standards of ethical behavior. The Pediatric Anesthesia fellows were evaluated on the following Milestones: Communication with patients and families, Giving and receiving feedback, and Interdisciplinary and Transition of Care. Using a checklist and global score mapped to each specific milestone, each fellow was assessed during the simulations by two faculty per station. Post-surveys were completed by faculty and fellows.
Results
Combined survey data were analyzed from all scenarios. Surveys were reported based on a 5-point Likert scale. Data reported as means and range. (M; Min:Max). Sample faculty (n=21) responses: It is important to have a milestone simulation program (4.7; 3:5); It is useful to observe these scenarios specifically (4.66; 3:5); The case was appropriate for the objective (4.65; 4:5); Ease of using the grading rubric (4.39; 3:5). Sample fellow (n=10) responses include: These scenarios helped to understand potential communication issues (4.53; 4:5); These were believable scenarios (4.72; 4:5); I understood the purpose of this activity before participating (4.33; 3:5). Data regarding inter-rater reliability and comparison of faculty evaluation to self-evaluation is ongoing.
Conclusions
Based on surveys, the simulation exercises were well received by both faculty and fellows. Faculty agreed that it was useful to observe fellows objectively with the intention of milestone evaluations. All fellows within the same program and discipline were evaluated in a consistent manner, allowing for an equal comparison of skill and knowledge across identical scenarios and situational variables. Ongoing validation of the evaluation tools are necessary to ensure reproducibility of observations.
References:
1. Cheng A, Grant V, Dieckmann P, Arora S, Robinson T ,Eppich W. Faculty Development for Simulation Programs; Five Issues for the Future of Debriefing Training. Simul Healthc 2015; 4:217-22.
2. Bromley LM The Objective Structured Clinical Exam- practical aspects. Curr Opin Anaesthesiol. 2000 Dec;13(6):675-8
