"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead

The Validity of a Global Rating Scale and Checklist for Evaluation of Pediatric Laceration Repair


Abstract

Background: The Accreditation Council for Graduate Medical Education has developed milestones including procedural skills under the core competency of patient care. Laceration repair is a procedure the Pediatric Residency Review Committee states residents should have training in, and progress in that training should be monitored by residency programs. There exists no validated tool to evaluate pediatric resident laceration repair performance.

Research Question: Does a novel tool for the evaluation of resident laceration repair performance in a simulated laceration repair in the Pediatric Emergency Department (PED) demonstrate validity and reliability as measured by concordance in scores across reviewers and alignment of scores with proxies for procedural skill?

Methodology: A novel tool was developed to evaluate PED resident laceration repair performance. It is based on the validated Objective Structured Assessment of Technical Skills, & consists of two components: a global rating scale (GRS) and a checklist. 30 pediatric and family medicine residents were filmed performing simulated laceration repair. Each video was evaluated by five Emergency or Pediatric Emergency Medicine physicians. Evaluators were blinded to the identity of the proceduralist. To estimate agreement between the evaluators, concordance correlation coefficients (CCC) were calculated for both the GRS and checklist scores. Average scores were calculated for each resident and compared across levels of training and laceration repair experience. For each item, score ranges were calculated for each proceduralist, and then the median range was calculated for each item.

Results: 13 interns, 4 second year, and 13 third year residents were filmed performing laceration repair procedures. The CCC showed fair concordance across reviewers for both the checklist (0.55, 95% CI 0.38-0.69) and the GRS (0.53, 95% CI 0.36-0.67). Scores for both tools improved with increased years of training and reported procedural experience (figures 1 &2). The median range of scores for each 5-point item on the GRS varied between 0.0-2.0.

Conclusions: A novel tool to evaluate resident laceration repair performance in a PED showed fair agreement across reviewers. Proceduralist scores improved by training and experience. The study tool is not precise enough for summative evaluation, but it may be useful to distinguish between trainees who have and have not attained competence in laceration repair for formative feedback. While there was fair variability between scores of reviewers, the gradient scoring of the GRS allowed item scores for each reviewer to provide a more refined evaluation than the simple binary checklist item scores.

Poster
non-peer-reviewed

The Validity of a Global Rating Scale and Checklist for Evaluation of Pediatric Laceration Repair


Author Information

Neil G. Uspal Corresponding Author

Pediatrics, Seattle Children's Hospital - University of Washington School of Medicine, Seattle, USA

Anita A. Thomas

Pediatrics, Seattle Children's Hospital- University of Washington School of Medicine, Seattle, USA

Rebekah Burns

Pediatrics, Seattle Children's Hospital - University of Washington School of Medicine, Seattle, USA

Maya Jones

Pediatrics, Seattle Children's Hospital - University of Washington School of Medicine, Seattle, USA

Isabel T. Gross

Pediatrics, Yale University School of Medicine, New Haven, USA

Ryan D. Kearney

Pediatrics, Seattle Children's Hospital - University of Washington School of Medicine, Seattle, USA

Rachel E. Whitney

Pediatrics, Yale University School of Medicine, New Haven, USA

Julie E. Uspal

Emergency Medicine, University of Pennsylvania School of Medicine

Nancy Gove

Miscellaneous, Seattle Children's Hospital - University of Washington School of Medicine, Seattle, USA

Jennifer Reid

Emergency Medicine, Seattle Children's Hospital - University of Washington School of Medicine, Seattle, USA


PDF Share
Converted 273a910040d911e79a156f0a4949e39a ipssw2017 lac repair poster 052417