"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead
Original article
peer-reviewed

Use of a Refresher Course Increases Confidence in Point-of-Care Ultrasound Skills in Emergency Medicine Faculty



Abstract

Introduction

All practicing emergency medicine (EM) physicians need to maintain a skillset in emergency ultrasound (US) after their initial training. EM physicians in academic practice may be supervising trainees performing ultrasound applications that they aren’t comfortable with. This study investigates the effectiveness of a US refresher course. The hypothesis was that a series of short courses would increase confidence in performing and supervising US applications.

Methods

Nine basic emergency ultrasound applications were taught over the course of one year by ultrasound fellowship-trained EM faculty in a simulation center at a single academic institution. Each session included 30-minutes of didactics/image review and 30-minutes of hands-on practice on normal volunteers and was followed by an anonymous questionnaire evaluating comfort level performing and supervising the ultrasound application before and after the course using a Likert scale from 1 “not at all confident” to 5 “very confident”.

Results

Thirty-six of 60 EM physicians participated in at least 1 of the 9 sessions (median 3, interquartile range 2-4). Faculty who attended had a median of 10 (interquartile range 7-15) years in practice and 61% work at both academic and community sites.

For all sessions combined, confidence in performing US increased from a mean score on the Likert scale of 3.3 to 4.4 (difference 1.1, confidence interval (CI) (0.94, 1.29), p < 0.001) and confidence in supervising trainees increased from a mean of 3.4 to 4.5 (difference 1.1, CI (0.88, 1.23), p < 0.001). The largest increases were seen in musculoskeletal (MSK), nerve, and pelvic applications and the least increase was seen with the session focused on intravenous access, but confidence was increased in all sessions.

Physicians in practice ≥10 years increased in confidence in performing and supervising the applications by 1.4 (CI (1.11, 1.60), p < 0.001) and 1.3, (CI (1.01, 1.49), p < 0.001), respectively. Physicians in practice <10 years increased 0.8 (CI (0.57, 1.03), p < 0.001) and 0.8 (CI (0.55, 1.05), p < 0.001), respectively.

Conclusion

An emergency ultrasound refresher course for EM physicians at a single institution improved self-reported confidence in both performing and supervising trainees in all applications reviewed. Those in practice ≥10 years showed the largest increases.

Introduction

Point-of-care ultrasound (POCUS) has become an essential tool in emergency medicine (EM) [1]. Ultrasound (US) is user-dependent and requires targeted training to ensure and maintain competence [2]. POCUS training during EM residency has recently been integrated into the Accreditation Council for Graduate Medical Education (ACGME) requirements for graduation [3]. Pathways for credentialing during residency and as a practicing EM physician have been established by the American College of Emergency Physicians (ACEP). Per policy, ongoing education after credentialing is also required for practicing physicians [4], but the exact nature of this education is left to the discretion of individual departments [5]. Low confidence in supervising physicians can be a potential source of suboptimal outcomes in the department and residency training [6-7]. Focused education beyond just clinical practice is necessary for effective learning, teaching, and patient safety [8].

Current literature has investigated different teaching models for introducing POCUS competency at various levels of training [9-12]. At the attending level, there has been a little investigation into the ideal method of maintaining US skills and comfort with the supervision of trainees. Despite efforts to standardize POCUS training during EM residency, a large variety of skills and comfort level persists after graduation [13-15].

The purpose of this study is to investigate a longitudinal US refresher course utilizing a blended model of didactic and image review with hands-on practice as a potential way for practicing EM faculty improve confidence. The hypothesis is that a series of short courses would increase confidence in performing POCUS applications in attending EM physicians. Changes in confidence for supervising trainees and the differences observed in participants based on the number of years in practice was also investigated.

Materials & Methods

This study was performed at a single university-affiliated ED with an active residency program that is responsible for staffing a level 1 tertiary care ED, community ED, and off-site urgent care facility with a total of 60 EM faculty. The institution also has an emergency US division with a US fellowship and dedicated emergency ultrasonographer. All EM attendings had been trained in ultrasound either during residency or in the practice pathway per ACEP guidelines. After surveying faculty for a needs assessment, nine basic emergency US applications (see Figure 1) were reviewed over the course of a year by US fellowship-trained EM faculty in a simulation center. Each session included 30-minutes of didactics/image review where technique, normal, and abnormal findings were reviewed. The next 30-minutes were used for hands-on practice on normal volunteers. Sessions were optional and scheduled immediately after faculty meetings in order to maximize attendance. An anonymous questionnaire filled out at the end of each session evaluated comfort level performing and supervising the ultrasound application before and after the course using a Likert scale from 1 “not at all confident” to 5 “very confident”. The survey also investigated years of practice, moonlighting, personal use of US for the application over the last year, and supervision of trainees over the last year.

The results were compiled anonymously by a blinded reviewer. Statistical analyses were done using Microsoft Excel (version 16.16.1). Confidence scores were reported using means, (using 95% confidence intervals (CI)), demographic data were reported using medians, (using interquartile ranges (IQR)), and percentages were used for descriptive data. Differences between two groups were compared using the Student’s t-test with p < 0.05 representing a statistically significant difference.

Results

Thirty-six EM physicians (60%) participated in at least 1 of the 9 sessions (median 3, IQR 2-4). A total of 84 surveys were completed. Faculty who attended had a median of 10 (IQR 7-15) years in practice. Sixty-one percent of the faculty who attended work at both the academic and community sites. Eighty percent of participants had performed the specific US application within the last year and 81% had supervised the application.

For all sessions combined, confidence in performing emergency US applications increased from a mean score of 3.3 to 4.4 on the 5 point-Likert scale (difference 1.1, CI (0.94, 1.29), p < 0.001) and confidence in supervising trainees increased from a mean of 3.4 to 4.5 (difference 1.1, CI (0.88, 1.23), p < 0.001). Prior to the sessions 44% (37/84) rated themselves a 4 or 5 on performing and 50% (42/84) rated themselves a 4 or 5 at supervising; after the sessions those numbers rose to 96% (81/84) (difference 52%, CI (40.83, 62.78), p < 0.001) and 93% (78/84) (difference 43%, CI (31.32, 53.01), p < 0.001). For individual sessions, confidence in both performance and application increased for all applications, see Figure 1.

Physicians in practice ≥10 years, (55% of respondents), showed an increase in confidence of performing and supervising US by 1.4 (CI (1.11, 1.60), p < 0.001) and 1.3, (CI (1.01, 1.49), p < 0.001), while physicians in practice <10 year, (45% of respondents), increased by 0.8 (CI (0.57, 1.03), p < 0.001) and 0.8 (CI (0.55, 1.05), p < 0.001), respectively, see Figure 2.

Discussion

This study demonstrated that confidence in both performing and supervising POCUS applications improved in EM physicians after participation in the refresher course. Confidence in performing and supervising was similar both before and after the sessions. This correlation suggests that by improving confidence in performing POCUS, confidence in supervising trainees also increases. This is important because supervising mid-level providers and trainees is a large part of many practicing EM physician’s roles and increasing supervisor confidence may help to improve learner's experience.

Education literature has shown that competency for learned skills, such as ultrasound and other procedures, can rapidly decay after initial training and that a major influence on this decay is the frequency of practice [16]. It has been demonstrated that learned skills for procedures, even those that are rarer, can be maintained more effectively through targeted practice and repetition [17]. Use and frequency of POCUS in the ED across the United States vary widely based on application, practice site, and provider [18]. Our study supports these findings with confidence varying widely, correlating to the frequency the exam is performed but suggests that confidence can be increased rapidly with a targeted refresher.

This study showed improvement in confidence across applications with some variation. Musculoskeletal (MSK) and nerve ultrasound were the applications that the least number of participants had performed or supervised over the last year and were also the applications that the fewest participants felt comfortable with prior to the sessions. These applications also showed the largest improvement after the sessions, suggesting that comfort is lower in applications that are performed less, but that a refresher has the potential to increase confidence to a higher extent. The application with the least change in confidence was intravenous (IV) access. This may be because this is an essential skill that providers are required to perform on a regular basis. In more technically challenging applications that are also commonly used, such as the cardiac and FAST (focused assessment with sonography in trauma) exam, although most participants had performed or supervised this exam recently, there were still many participants who were not confident prior to the session.

The widespread use of POCUS in the ED has been a more recent phenomenon and integration into the EM residency curriculum has occurred within the last decade [3]. Given the differences in the training environment, physicians may have different exposure to POCUS techniques. In this study, physicians in practice for over 10 years had greater increases in their confidence scores than those in practice for less than 10 years. This greater increase in confidence correlates to lower initial confidence in physicians that had been practicing for greater than 10 years. The reported confidence after the teaching sessions then equalizes between these two groups, suggesting that these teaching sessions can bring participants of a variety of prior experience to a similar level of comfort.

With ever-increasing demands of patient care, trainee education, and productivity, efficient learning strategies need to be implemented that balance these competing interests [19]. In this study, the refresher course consisted of short, hour-long sessions spread throughout the year. Given that each session was a stand-alone application, participants were able to benefit from attending any number of sessions. Prior studies that have investigated US education involve single, longer sessions and are often meant for trainees with less experience [11-12]. Given that most practicing EM physicians have already had exposure to POCUS, a short refresher course was thought to be more appropriate and practical. This study demonstrates that short, focused sessions for US with a longitudinal component are effective and this model can be implemented at other institutions for emergency medicine faculty.

Limitations of this study include that it was performed at a single academic center with a small sample size. Participation was optional and the differences between faculty who chose to participate and those that did not, were not analyzed. There was no evaluation of skills or technical improvement with this intervention and there was no long-term follow up into whether the increase in confidence reported translated into measurable clinical outcomes including the increasing number of scans performed or supervised or the quality of future scans. Potential future investigations would be to correlate to US scan numbers, resident or mid-level supervision, and accuracy of scans before and after the intervention.

Conclusions

A longitudinal US refresher course for EM physicians at a single institution improved self-reported confidence in both performing and supervising trainees in all applications reviewed. Those in practice ≥10 years showed the largest increases in confidence.


References

  1. Whitson MR, Mayo PH: Ultrasonography in the emergency department. Crit Care. 2016, 20:227. 10.1186/s13054-016-1399-x
  2. Moore CL, Copel JA: Point-of-care ultrasonography. New Eng J Med. 2011, 364:749-757. 10.1056/NEJMra0909487
  3. Beeson MS, Carter WA, Christopher TA, et al.: Emergency medicine milestones. J Grad Med Educ. 2013, 5:5-13. 10.4300/JGME-05-01s1-02
  4. Ultrasound guidelines: emergency, point-of-care and clinical ultrasound guidelines in medicine. Ann Emerg Med. 2017, 69:e27-e54. 10.1016/j.annemergmed.2016.08.457
  5. Das D, Kapoor M, Brown C, Ndubuisi A, Gupta S: Current status of emergency department attending physician ultrasound credentialing and quality assurance in the United States. Crit Ultrasound J. 2016, 8:6. 10.1186/s13089-016-0042-z
  6. Jagsi R, Kitch BT, Weinstein DF, Campbell EG, Hutter M, Weissman JS: Residents report on adverse events and their causes. Arch Intern Med. 2005, 165:2607-2613. 10.1001/archinte.165.22.2607
  7. Kilroy DA: Clinical supervision in the emergency department: a critical incident study. Emerg Med J. 2006, 23:105-108. 10.1136/emj.2004.022913
  8. Rodriguez-Paz JM, Kennedy M, Salas E, Wu AW, Sexton JB, Hunt EA, Pronovost PJ: Beyond "see one, do one, teach one": toward a different training paradigm. Postgrad Med J. 2009, 85:244-249. 10.1136/qshc.2007.023903
  9. Lewiss RE, Hoffmann B, Beaulieu Y, Phelan MB: Point-of-care ultrasound education. J Ultrasound Med. 2014, 33:27-32. 10.7863/ultra.33.1.27
  10. Stolz LA, Amini R, Situ-Lacasse E, et al.: Multimodular ultrasound orientation: residents’ confidence and skill in performing point-of-care ultrasound. Cureus. 2018, 10:3597. 10.7759/cureus.3597
  11. Yamada T, Minami T, Soni NJ, Hiraoka E, Takahashi H, Okubo T, Sato J: Skills acquisition for novice learners after a point-of-care ultrasound course: does clinical rank matter?. BMC Med Educ. 2018, 18:202. 10.1186/s12909-018-1310-3
  12. Maw A, Jalali C, Jannat-Khah D, et al.: Faculty development in point of care ultrasound for internists. Med Educ Online. 2016, 21:33287. 10.3402/meo.v21.33287
  13. Ahern M, Mallin M, Weitzel S, Madson T, Hunt P: Variability in ultrasound education among emergency medicine residencies. West J Emerg Med. 2010, 11:314-318.
  14. Kim D, Theoret J, Liao M, Kendall J: Experience with emergency ultrasound training by Canadian emergency medicine residents. West J Emerg Med. 2014, 15:306-311. 10.5811/westjem.2013.9.18025
  15. Stolz LA, Stolz U, Fields JM, et al.: Emergency medicine resident assessment of the emergency ultrasound milestones and current training recommendations.. Acad Emerg Med. 2017, 24:353-361. 10.1111/acem.13113
  16. Jr. WA, Jr. WB, Stanush PL, Mcnelly TL: Factors that influence skill decay and retention: a quantitative review and analysis. Hum Perform. 1998, 11:57-101. 10.1207/s15327043hup1101_3
  17. Kovacs G, Bullock G, Ackroyd-Stolarz S, Cain E, Petrie D: A randomized controlled trial on the effect of educational interventions in promoting airway management skill maintenance. Ann Emerg Med. 2000, 36:301-309. 10.1067/mem.2000.109339
  18. Hall MK, Hall J, Gross CP, et al.: Use of point-of-care ultrasound in the emergency department. J Ultrasound Med. 2016, 35:2467-2474. 10.7863/ultra.16.01041
  19. Farley H, Casaletto J, Ankel F, Young KD, Hockberger R: An assessment of the faculty development needs of junior clinical faculty in emergency medicine. Acad Emerg Med. 2008, 15:664-668. 10.1111/j.1553-2712.2008.00152.x
Original article
peer-reviewed

Use of a Refresher Course Increases Confidence in Point-of-Care Ultrasound Skills in Emergency Medicine Faculty


Author Information

Madeline Schwid Corresponding Author

Emergency Medicine, Brigham & Women’s Hospital, Harvard Medical School, Boston, USA

Owen Harris

Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, USA

Adaira Landry

Emergency Medicine, Brigham & Women’s Hospital, Harvard Medical School, Boston, USA

Andrew Eyre

Emergency Medicine, Brigham & Women’s Hospital, Harvard Medical School, Boston, USA

Patricia Henwood

Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, USA

Heidi Kimberly

Emergency Medicine, Brigham & Women’s Hospital, Harvard Medical School, Boston, USA


Ethics Statement and Conflict of Interest Disclosures

Human subjects: Consent was obtained by all participants in this study. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.


Original article
peer-reviewed

Use of a Refresher Course Increases Confidence in Point-of-Care Ultrasound Skills in Emergency Medicine Faculty


Figures etc.

SIQ
-
RATED BY 0 READERS
CONTRIBUTE RATING

Scholary Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.