"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead
Original article
peer-reviewed

In Pursuit of the Most Effective Method of Teaching Feedback Skills to Emergency Medicine Residents in Qatar: A Mixed Design



Abstract

Purpose

The study aimed to find an effective method of teaching feedback skills to residents and to gauge their preference.

Method

This was a mixed design study conducted at the emergency department of a large tertiary care hospital. The residents were randomized to groups A, B, and C. Group A (control) received a traditional lecture, Group B read a specifically written brief document, and Group C received 1:1 tutoring from one faculty. Each resident individually watched a four-minute video on an emergency procedure and provided feedback in simulated settings, which was audio-recorded and rated by two blinded raters. An assessment form was created and validated. The residents’ preference was attained through a semi-structured interview.

Results

The baseline characteristics of the three groups were similar. Compared to Group A, Groups B and C scored significantly higher on the overall assessment and were statistically similar to each other. There was no sign of association between both gender and postgraduate score (PGY) year on the total score. Residents’ equally preferred self-reading and 1:1 tutoring.

Conclusion

The acquisition of feedback skills by emergency medicine (EM) residents was comparable between self-learning from an appropriately written document and 1:1 teaching by adequately trained faculty.

Introduction

Constructive feedback provided by appropriately trained faculty is a potent instrument in medical education. It helps the learners to comprehend strengths, understand limitations, and provide a roadmap for future progress in their clinical career. Poor feedback may result in errors not being corrected and good performance not being recognized as potentially impacting clinical performance [1]. Veloski and colleagues conducted a systematic review and found a positive correlation between feedback and clinical performance [2].

The residency curriculum requires future residents to be effective teachers who have competent feedback skills [3]. The residents are more likely to respond to feedback by experienced and appropriately trained faculty as compared to other instructions [4]. A study among medical students confirmed that there was a lack of effective feedback in almost half of the interactions due to the poor feedback skills of the clinical faculty and the absence of a clear feedback system [5]. In spite of the widespread recommendation of constructive feedback in the education literature, the feedback provided is of low quality [6]. One of the concerning aspects is that feedback in medical education is provided by faculty with little or no formal training in this teaching role [6]. Poor or inappropriate feedback is not only present in clinical medicine but is also rife in business administration, psychology, and educational literature [7]. Specific training may be required to provide effective feedback. In one study, a postgraduate qualification in medical education did not endorse the faculty’s ability to provide appropriate and constructive feedback [8].

There are several reasons why effective feedback does not happen. Some tutors feel that negative feedback may not help students to progress and damage their educational relationships [9]. Some teachers are reluctant to convey corrective feedback, particularly in front of patients and colleagues. At the same time, trainees report reluctance in asking for feedback, which if critical might make them defensive and may make the matter worse [10]. Lack of knowledge and training of the clinical faculty has also been cited as an important barrier to effective feedback [11].

While the literature supports the training of medical educators on how best to give effective feedback, there is no rigorous study recommending the best methods to improve the quality of such feedback [12-13]. Previous studies lack randomization and consist of small, heterogeneous study populations. In order to enhance the quality of feedback, such training should be undertaken during residency [14]. In the host institution, the teaching of feedback skills to the emergency medicine residents (EMR) is provided by emergency physicians (EP) through traditional lectures (TL) and small group teachings during normal didactic teaching days. TL has been criticized in the past due to difficulty in maintaining attention for more than 10-15 minutes [15]. The study aimed to compare the effectiveness of self-learning and 1:1 tutoring as compared to the standard lecturing in acquiring feedback skills among emergency medicine (EM) residents in Qatar. The secondary aim was to gauge their preference for an educational method.

Materials & Methods

Institutional ethics approval was obtained from the Medical Research Center (MRC) of Hamad Medical Corporation (MRC-01-18-136), and the study was classified as exempt category 1, research conducted in the established commonly accepted educational settings involving normal educational practices. MRC also funded the project.

Setting

The study was conducted in the largest teaching hospital in Doha, the capital of the state of Qatar. The host hospital emergency department sees approximately half a million patients every year. The study center runs a four-year emergency medicine residency (EMR) training program, which has been accredited by the international branch of the USA-based Accreditation Council for Graduate Medical Education. All residents attend their didactic educational training on Tuesday every week, and the study was conducted during these days in the months of March and April 2019.

Software package

The information from the study was recorded into a dataset in the statistical package Stata (version 15MP, Stata Corp, College Station, Texas). Stata was used for all graphing and analytics for this report.

Design

This was a mixed design study. The research assistant, statistician, and the faculty responsible for rating were blinded to the type of educational intervention. Two experienced faculty members prepared a standardized clinical skill teaching video of approximately four minutes duration. The video demonstrated an often-performed clinical skill relevant to EPs. The procedure is endotracheal intubation on a manikin performed by a clinical fellow in EM and supervised by EM faculty. These roles are played by two senior faculty members involved in the study. The video was reviewed by a senior faculty member with experience in medical education and in giving feedback. Following this review, it was concluded that the video is a suitable demonstration of a skill on which feedback could be provided during the study.

Randomization and blinding

Each resident was randomized into one of three groups (A, B, or C) through the utilization of computer-generated block randomizations. The educational material presented to the residents was similar in all three groups. The randomization was prepared using computer-generated random numbers. The block sizes were 3, 6, and 9. The EM residents were assigned with equal probability to the three groups. The randomization sequence was concealed in envelopes that were sealed and opaque. Like most other educational trials, it was difficult to blind the residents and faculty; however, the outcome raters and statisticians were kept blinded to the study groups.

Control and intervention groups

Forty-five EMRs partook in the study and were in different years of training. Fifteen EMRs were assigned to group A (TL), group B (self-reading), and group C (1-1 teaching). Group A (as well as the two other groups) received one-hour teaching during their residency training within the last 12 months. This teaching consisted of a lecture followed by discussion in small groups. The intervention group (B) read one A4-page instruction, specially prepared based on the attributes of good feedback (Appendix A). The document was reviewed by two experienced physicians, who found it to be appropriate for residents’ training. Finally, the document was successfully piloted on two recently board-certified EMRs. During the trial, most residents read the document twice, and some made the notes; however, all the written notes were removed while watching the video and audio-recording their feedback. The intervention group (C) received one 1:1 teaching on the characteristics of good feedback by one experienced faculty. Each resident in group B and C spent 15-20 minutes on the study.

Validation of the study questionnaire

Based on recent literature, 10 characteristics were identified that should be present in a good feedback discussion [16]. Based on these characteristics, a 10-item questionnaire with a yes/no answer was prepared. A four-minute educational video was prepared in which one faculty member is providing feedback to the resident about his lecture on shock. The video aimed to facilitate the validation of the questionnaire. The video was shown as a group to 19 senior fellows and consultants, and they were asked to rate the 10-part questionnaire individually. If all the raters agreed that the person addressed the issue in the video then the overall agreement would be close to 100% percent, if everyone agreed that the person did not address the issue on the video, the overall proportion would be close to 0%. If the percentage was close to 100% or close to 0%, there was a good agreement. In eight out of 10 questions, there was virtually 100% agreement, one-sided 97.5% CI, for the agreement point estimate of 100% (19/19) was 82.4%-100%. The 10-item number questionnaire was used in the study (Appendix B).

Outcome measures

Each feedback was audio-recorded and was given a code and anonymized. Using digital media, such as video and audio recording, in medical education research has been recommended in previous studies [17]. Two blinded raters who are experienced faculty were trained and assessed for rating the audio recordings. In 43/45 recordings, there was 100% agreement between the two raters. In the remaining two audio recordings, there was only a 50% agreement; hence, these were rated by a third blinded faculty. During the semi-structured interview, eight EMRs were asked about their preference for one educational intervention.

Analysis of data

Categorical variables were reported as counts and proportions, while continuous variables were summarized as means with their corresponding standard deviations (SD). Fischer’s exact test was used to compare categorical variables, while the mean scores across various categories were compared using the one-way analysis of variance (ANOVA). The degree of association between significant variables on one-way ANOVA was further explored using simple linear regression analysis. The information was recorded into a dataset in the statistical package Stata (version 15MP, Stata Corp, College Station, Texas USA). Stata was used for all graphing and analytics for this report.

Ethical approval

The study was approved by the Medical Research Center (MRC) of the host hospital (RP-01-18-136).

Participants consent

The study was classified as exempt under the Supreme Council of Health Qatar guidelines.

Author’s contribution

KB and AE conceived the study, obtained ethical approval and wrote the first manuscript, MS and SA conducted the study, collected the data, and critically appraised the literature, SH and ST did the detailed statistical analysis, wrote the report, and performed overall supervision.

Conflict of interest

There was no conflict of interest.

Results

A total of 45 EMR participated in the study; there was a difference in the number of males and females (Male = 68.9% vs Female = 31.1%; p=0.174) and postgraduate year (PGY) (PGY 1 = 28.9%, PGY 2 = 24.4%, PGY 3 = 26.7%, PGY 4 = 20.0%; p=0.530) across the different trial groups at baseline; however, it was not significantly different from the ratio of male and female EMRs in the institute where the study was conducted. The difference was also not relevant to the objectives and the results of the study with no significant difference in gender (Tables 1-2).

Gender Group Total
A B C
Female 2 (13.33%) 5 (33.33%) 7 (46.67%) 14 (31.11%)
Male 13 (86.67%) 10 (66.67%) 8 (53.33%) 31 (68.89%)
Total 15 (100.00%) 15 (100.00%) 15 (100.00%) 45 (100.00%)
Fisher’s exact = 0.174; No difference in gender across groups
PGY Group Total
A B C
1 4 (26.67%) 4 (26.67%) 5 (33.33 %) 13 (28.89%)
2 5 (33.33 %) 2 (13.33%) 4 (26.67%) 11 (24.44%)
3 2 (13.33%) 7 (46.67%) 3 (20.00%) 12 (26.67%)
4 4 (26.67%) 2 (13.33%) 3 (20.00%) 9 (20.00%)
Total 15 (100.00%) 15 (100.00%) 15 (100.00%) 45 (100.00%)
Fisher’s exact = 0.530 No difference in PGY spread across groups

There was a statistically significant difference in mean scores across the intervention groups on one-way analysis of variance (ANOVA) (Group A mean score [SD] = 3.5 [0.6]; Group B mean score [SD] = 4.8 [1.2]; and Group C mean score [SD] = 6.3 [1.2]; p=0.001) (Table 3).

  Number Mean Standard Deviation (SD)
Group A 15 3.533333 0.6399405
Group B 15 4.8 1.897367
Group C 15 6.266667 1.222799

However, no statistically significant difference in mean scores was noted across the PGY (p=0.594).

On simple linear regression, participants in groups B (b = 1.7, 95% confidence interval [CI] = 0.3 - 2.3, p= 0.014) and C (b = 2.7, 95% CI = 1.7 - 3.7, p= <0.001) had significantly higher mean scores as compared to those in Group A (no intervention). The regression model explained up to 42.1% of the total variability in mean scores across groups. The EMRs equally favored self-reading and 1:1 tutoring.

Discussion

The aim of the RCT was to compare commonly employed educational methods of teaching feedback skills to EM residents. The study demonstrated that there was a statistically significant better performance in the overall assessment of both educational interventions’ groups (B & C) as compared to the control group (A). There was no sign of association between either sex or PGY year on total score.

There was a statistically significant improvement in scores in both intervention groups. On simple linear regression, participants in groups B (b = 1.7, 95% confidence interval [CI] = 0.3 - 2.3, p= 0.014) and C (b = 2.7, 95% CI = 1.7 - 3.7, p= 0.001) had significantly higher mean scores as compared to those in Group A (no intervention). Self-reading is a common method employed for acquiring knowledge and skills among the residents. Self-reading and making notes of the important and relevant points helps to retain the information. This technique has been recommended for medical students learning [18]. 1:1 teaching encourages active learning, which is an outstanding method of acquiring knowledge and skills but involves a substantial amount of faculty time [19]. During 1:1 teaching, the faculty can easily focus on the needs of the learner, and this close interaction raises the potential for an effective outcome [20]. Both these methods were preferred by residents, and a recommendation has been forwarded to the residency program for future consideration [16].

We have identified several limitations in our study. First, the study was conducted in a single academic center and the sample may not be representative of other sites; hence, the results may not be generalizable. The EMRs completed their four years of training within the host hospital. For this practical reason, only local recruitment was done. Second, only two educational methods were considered in this trial and compared with the traditional didactic lecture. There are other popular educational methods, such as blended learning, which may have provided a better outcome. Due to the limited time available during the residency educational days, these two methods were most appropriate for the study. Third, we only assessed the short-term effect of educational interventions. The long-term effects, possibly six or 12 months after the educational intervention, would have been more useful to look at retention of feedback skills. EM residents spend 18 months outside the department rotating to other specialties; hence, their availability for a follow-up study is not guaranteed. The fourth limitation was the limited use of semi-structured interviews for assessing the preference for educational methods. The use of the Focus group would have provided more detailed information. Due to the limited time available during the residents’ education days, we were not able to conduct detailed interviews.

Conclusions

The study showed that either a carefully prepared article or 1:1 teaching resulted in an improvement of feedback-giving skills to EM residents in the short term. The study may have implications on the utilization of faculty time for teaching core skills to EMR. Future research should investigate the long-term retention of educational interventions.


References

  1. Ende J: Feedback in clinical medical education. JAMA. 1983, 250:777-781. 10.1001/jama.1983.03340060055026
  2. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DW: Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach. 2006, 728:117-128. 10.1080/01421590600622665
  3. Sexton KW, Kensinger CD, Vella MA, McMaster WG, Terhune KP, Snyder RA: Residents as educators: a modern model. J Surg Educ. 2015, 72:949-956. 10.1016/j.jsurg.2015.04.004
  4. Hardavella G, Aamli-Gaagnat A, Saad N, Rousalova I, Sreter KB: How to give and receive feedback effectively. Breathe. 2017, 13:327-333. 10.1183/20734735.009917
  5. Al-Haqwi AI, Al-Wahbi AM, Abdulghani HM, van der Molen HT: Barriers to feedback in undergraduate medical education. Saudi Med J. 2012, 33:557-561.
  6. Reddy ST, Zegarek MH, Fromme HB, Ryan MS, Schumann S-A, Harris IB: Barriers and facilitators to effective feedback: a qualitative analysis of data from multispecialty resident focus groups. J Gr Med Educ. 2015, 7:214-219. 10.4300/JGME-D-14-00461.1
  7. Hattie J, Timperley H: The power of feedback. Rev Educ Res. 2007, 77:81-112. 10.3102/003465430298487
  8. Al Wahbi A: The need for faculty training programs in effective feedback provision. Adv Med Educ Pract. 2014, 2014:263-268. 10.2147/AMEP.S62889
  9. Dudek NL, Marks MB, Regehr G: Failure to fail: the perspectives of clinical supervisors. Acad Med. 2005, 80:84-87. 10.1097/00001888-200510001-00023
  10. Henderson P, Ferguson-Smith AC, Johnson MH: Developing essential professional skills: a framework for teaching and learning about feedback. BMC Med Educ. 2005, 5:11. 10.1186/1472-6920-5-11
  11. Cantillon P, Sargeant J: Giving feedback in clinical settings. BMJ. 2008, 337:a1961. 10.1136/bmj.a1961
  12. Junod Perron N, Nendaz M, Louis-Simonet M, et al.: Effectiveness of a training program in supervisors’ ability to provide feedback on residents’ communication skills. Adv Heal Sci Educ. 2013, 18:901-915. 10.1007/s10459-012-9429-1
  13. Minehart RD, Rudolph J, Pian-Smith MC, Raemer DB: Improving faculty feedback to resident trainees during a simulated case: a randomized, controlled trial of an educational intervention. Anesthesiology. 2014, 120:160-171. 10.1097/ALN.0000000000000058
  14. Branch WT Jr, Paranjape A: Feedback and reflection. Teaching methods for clinical settings. Acad Med. 2002, 77:1185-1188. 10.1097/00001888-200212000-00005
  15. Stuart J, Rutherford RJD: Medical student concentration during lectures. Lancet. 1978, 2:514-516. 10.1016/S0140-6736(78)92233-X
  16. Ramani S, Krackov SK: Twelve tips for giving feedback effectively in the clinical environment. Med Teach. 2012, 34:787-791. 10.3109/0142159X.2012.684916
  17. Parry R, Pino M, Faull C, Feathers L: Acceptability and design of video-based research on healthcare communication: evidence and recommendations. Patient Educ Couns. 2016, 99:1271-1284. 10.1016/j.pec.2016.03.013
  18. Augustin M: How to learn effectively in medical school: test yourself, learn actively, and repeat in intervals. Yale J Biol Med. 2014, 87:207-212.
  19. Gordon J: ABC of learning and teaching in medicine: one to one teaching and feedback. BMJ. 2003, 326:543-5. 10.1136/bmj.326.7388.543
  20. Caldwell G: Real‐time assessment and feedback of junior doctors improves clinical performance. Clin Teach. 2006, 3:185-188. 10.1111/j.1743-498X.2006.00100.x

Appendices

Appendix A

Self-reading instructions

Please read the following instructions before watching the video and recording your feedback.

The following steps have been recommended in the literature for effective feedback:

1. Establish a respectful learning environment

2. Communicate goals and objectives for feedback

3. Make feedback timely and a regular occurrence

4. Provide feedback on direct observation only? Feedback on behaviors based on direct observation by the teacher has been reported to be more acceptable and instructive to trainees than feedback based on second-hand reporting.

5. Should the session begin with the learner’s self-assessment? A key goal of clinical training is to promote a reflective practitioner. Opening the feedback session by inviting the learner to self-assess can help achieve this goal. Use open-ended questions to start the meeting as a conversation and promote the learner’s reflection on his/her practices.

6. Reinforce positive behaviors?

7. Correct negative behaviors?

8. Use specific and neutral language to focus on performance?

9. Confirm the learner understands and facilitates acceptance? Concluded with an action plan?

10. Encouraged reflection on the skills of the faculty? Reflection by the teacher should follow every feedback session After the session ends, the teacher should reflect on what went well, what to change the next time and what new strategies he/she will adapt for future sessions.

Appendix B

MRC-01-18-136

Enrollment ID:

Gender:

PGY:

Rating Scale

Questions Yes No
1. Feedback provided on direct observation?    
2. The session began with the learner’s self-assessment?    
3. Reinforced positive behaviors?    
4. Corrected the negative behaviors?    
5. Used specific and neutral language to focus on performance?    
6. Confirmed the learner understands and facilitates acceptance?    
7. Concluded with an action plan?    
8. Encouraged reflection on the skills of the faculty?    
Original article
peer-reviewed

In Pursuit of the Most Effective Method of Teaching Feedback Skills to Emergency Medicine Residents in Qatar: A Mixed Design


Author Information

Khalid Bashir

Emergency Medicine, Hamad Medical Corporation, Doha, QAT

Medicine, Qatar University, Doha, QAT

Amr Elmoheen Corresponding Author

Emergency Medicine, Hamad Medical Corporation, Doha, QAT

Mohammed Seif

Emergency Medicine, Hamad Medical Corporation, Doha, QAT

Shahzad Anjum

Emergency Medicine, Hamad Medical Corporation, Doha, QAT

Saleem Farook

Emergency Medicine, Hamad Medical Corporation, Doha, QAT

Stephen Thomas

Emergency Medicine, Hamad Medical Corporation, Doha, QAT


Ethics Statement and Conflict of Interest Disclosures

Human subjects: Consent was obtained by all participants in this study. Medical Research Center (MRC) of Hamad Medical Corporation issued approval RP-01-18-136. The Medical Research Center has granted permission for this research to be published on condition that no patient-identifiable data (including patient name and photograph) are included. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Acknowledgements

The authors would like to thank Dr. Fareeha Khan for her help in the recruitment of the participants and data collection.


Original article
peer-reviewed

In Pursuit of the Most Effective Method of Teaching Feedback Skills to Emergency Medicine Residents in Qatar: A Mixed Design


Figures etc.

SIQ
6.2
RATED BY 2 READERS
CONTRIBUTE RATING

Scholary Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.