Abstracts
Abstract
Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen’s University Internal Medicine residency program introduced a facilitated, team-based assessment initiative (“Feedback Fridays”) in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training.
Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program’s assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically.
Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding.
Conclusions: The introduction of facilitated team-based assessment in the Queen’s Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.
Résumé
Objectif : La formation médicale fondée sur les compétences s'appuie sur la rétroaction faite lors de l'évaluation des apprentissages par observation directe dans le milieu de travail. Malheureusement, les évaluations dans le milieu de travail omettent souvent de fournir une rétroaction narrative exhaustive et privilégient les aspects des soins relevant de l'expertise médicale. En se basant sur la recherche ayant étudié les approches d'évaluation interactive, le programme de résidence en médecine interne de l'Université Queen's a introduit en juillet 2017 une initiative d'évaluation facilitée et en équipe (« Les vendredis rétroaction »), visant à améliorer l'évaluation holistique du rendement des résidents dans les unités d'enseignement clinique en médecine interne. Dans cette étude, nous visons à explorer comment ces « vendredis rétroaction » ont contribué à l'évaluation formative des résidents en médecine interne dans le cadre de notre modèle actuel de formation axée sur les compétences.
Méthode : Au total, 53 résidents ont participé à des séances d'évaluation de groupe facilitées et bi-hebdomadaires au cours de l'année universitaire 2017-2018. Chaque séance consistait en une discussion d'évaluation facilitée de 30 minutes menée avec une équipe de l’unité de soins, qui comprenait des étudiants en médecine, des résidents et le médecin superviseur. Les commentaires issus de la discussion ont été recueillis, résumés et documentés sous forme narrative dans des formulaires électroniques d’observation directe dans le milieu de travail par le responsable de l'évaluation du programme de résidence. À des fins de recherche, les transcriptions verbatim des séances de rétroaction ont été analysées de façon thématique.
Résultats : Les chercheurs ont identifié quatre thèmes principaux pour les commentaires : la communication, la conscience intra- et interpersonnelle, le leadership et le travail d'équipe, et les occasions d'apprentissage. Bien que la rétroaction concerne un large éventail d'activités, elle met fortement l'accent sur les compétences liées aux rôles intrinsèques de CanMEDS. De plus, le fait que la rétroaction avait un rôle clairement formatif est une autre constatation importante.
Conclusions : L'introduction de l'évaluation en équipe facilitée dans le programme de médecine interne à Queen's a comblé une lacune importante dans l'apprentissage par observation directe dans le milieu de travail en fournissant aux apprenants une rétroaction détaillée sur tous les rôles CanMEDS et en formulant des recommandations constructives sur les domaines à améliorer.
Appendices
Bibliography
- Schuwirth L, Ash J. Assessing tomorrow's learners: in competency-based education only a radically different holistic method of assessment will work. Six things we could forget. Med Teach. 2013;35(7):555-9. https://doi.org/10.3109/0142159X.2013.787140
- Schuwirth L, Valentine N, Dilena P. An application of programmatic assessment for learning (PAL) system for general practice training. GMS J Med Educ. 2017;34(5):Doc56.
- van der Vleuten CP, Schuwirth LW, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205-14. https://doi.org/10.3109/0142159X.2012.652239
- Branfield Day L, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7. https://doi.org/10.1097/ACM.0000000000003315
- Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. https://doi.org/10.1111/medu.13645
- Gaunt A, Patel A, Rusius V, Royle TJ, Markham DH, Pawlikowska T. 'Playing the game': How do surgical trainees seek feedback using workplace-based assessment? Med Educ. 2017;51(9):953-62. https://doi.org/10.1111/medu.13380
- Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34. https://doi.org/10.1097/ACM.0000000000001523
- Gaunt A, Markham DH, Pawlikowska TRB. Exploring the role of self-motives in postgraduate trainees' feedback-seeking behavior in the clinical workplace: a multicenter study of workplace-based assessments from the United Kingdom. Acad Med. 2018;93(10):1576-83. https://doi.org/10.1097/ACM.0000000000002348
- Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34. https://doi.org/10.1097/ACM.0000000000001523
- Teunissen PW, Stapel DA, van der Vleuten C, Scherpbier A, Boor K, Scheele F. Who wants feedback? An investigation of the variables influencing residents' feedback-seeking behavior in relation to night shifts. Acad Med. 2009;84(7):910-7. https://doi.org/10.1097/ACM.0b013e3181a858ad
- Dudek NL, Marks MB, Regehr G. Failure to fail: the perspectives of clinical supervisors. Acad Med. 2005;80(10 Suppl):S84-7. https://doi.org/10.1097/00001888-200510001-00023
- Schwind CJ, Williams RG, Boehler ML, Dunnington GL. Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies? Acad Med. 2004;79(5):453-7. https://doi.org/10.1097/00001888-200405000-00016
- Bandiera G, Lendrum D. Daily encounter cards facilitate competency-based feedback while leniency bias persists. Cjem. 2008;10(1):44-50. https://doi.org/10.1017/S1481803500010009
- Ginsburg S, Gold W, Cavalcanti RB, Kurabi B, McDonald-Blumer H. Competencies "plus": the nature of written comments on internal medicine residents' evaluation forms. Acad Med. 2011;86(10):S30-S4. https://doi.org/10.1097/ACM.0b013e31822a6d92
- Chou S, Cole G, McLaughlin K, Lockyer J. CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction. Med Educ. 2008;42:879-86. https://doi.org/10.1111/j.1365-2923.2008.03111.x
- Rida T-Z, Dubois D, Hui Y, Ghatalia J, McConnell M, LaDonna K, editors. Assessment of CanMEDS competencies in work-based assessment: challenges and lessons learned. 2020 CAS Annual Meeting.
- McConnell M, Sherbino J, Chan TM. Mind the gap: the prospects of missing data. J Grad Med Ed. 2016;8(5):708-12. https://doi.org/10.4300/JGME-D-16-00142.1
- Hemmer PA, Hawkins R, Jackson JL, Pangaro LN. Assessing how well three evaluation methods detect deficiencies in medical students' professionalism in two settings of an internal medicine clerkship. Acad Med. 2000;75(2):167-73. https://doi.org/10.1097/00001888-200002000-00016
- Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015;90(12):1698-706. https://doi.org/10.1097/ACM.0000000000000809
- Yin R. Case study research and applications: design and methods. Thousand Oaks, CA: SAGE Publications, Inc; 2017.
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-57. https://doi.org/10.1093/intqhc/mzm042
- McEwen LA EA, Chamberlain S, Taylor D. Queen's Periodic Performance assessments: capturing performance information beyond single observations in CBME Contexts. 2nd World Summit on CBME; Basel, Switzerland2018.
- Taylor DR. Feedback Friday Intro [Video]. 2017. Available from: https://www.youtube.com/watch?v=OsRB3NfL00g&t=7s [Accessed 2024/02/05]
- Royal College of Physicians and Surgeons of Canada. Competence by Design: Canada's model for competency-based medical education 2022. Available from: https://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e.
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psych. 2006;3(2):77-101. https://doi.org/10.1191/1478088706qp063oa
- Cofie N, Braund H, Dalgarno N. Eight ways to get a grip on intercoder reliability using qualitative-based measures. Can Med Ed J. 2022. https://doi.org/10.36834/cmej.72504
- Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res. 1999;9(1):26-44. https://doi.org/10.1177/104973299129121677
- Palaganas EC, Sanchez MC, Molintas VP, Caricativo RD. Reflexivity in qualitative research: a journey of learning. Qual Rep. 2017;22(2). https://doi.org/10.46743/2160-3715/2017.2552
- Day LB, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7. https://doi.org/10.1097/ACM.0000000000003315
- Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. https://doi.org/10.1111/medu.13645
- Ashenafi MM. Peer-assessment in higher education-twenty-first century practices, challenges and the way forward. Assess Eval High Educ. 2017;42(2):226-51. https://doi.org/10.1080/02602938.2015.1100711
- Wanner T, Palmer E. Formative self-and peer assessment for improved student learning: the crucial factors of design, teacher participation and feedback. Assess Eval High Educ. 2018;43(7):1032-47. https://doi.org/10.1080/02602938.2018.1427698
- Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency‐based medical education: are we addressing the concerns and challenges? Med Educ. 2015;49(11):1086-102. https://doi.org/10.1111/medu.12831
- Sirianni G, Glover Takahashi S, Myers J. Taking stock of what is known about faculty development in competency-based medical education: a scoping review paper. Med Teach. 2020;42(8):909-15. https://doi.org/10.1080/0142159X.2020.1763285
- Richardson D, Kinnear B, Hauer KE, et al. Growth mindset in competency-based medical education. Med Teach. 2021;43(7):751-7. https://doi.org/10.1080/0142159X.2021.1928036
- Young JQ, Sugarman R, Schwartz J, O'Sullivan PS. Faculty and resident engagement with a workplace-based assessment tool: use of implementation science to explore enablers and barriers. Acad Med. 2020;95(12):1937-44. https://doi.org/10.1097/ACM.0000000000003543
- Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Advan Health Sci Ed. 2016;21(2):455-73. https://doi.org/10.1007/s10459-015-9614-0
- Braund H, Dalgarno N, McEwen L, Egan R, Reid M-A, Baxter S. Involving ophthalmology departmental stakeholders in developing workplace-based assessment tools. Can J Ophthalmol. 2019;54(5):590-600. https://doi.org/10.1016/j.jcjo.2019.01.013
- Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159X.2010.501190
- Govaerts MJ, van der Vleuten CP, Holmboe ES. Managing tensions in assessment: moving beyond either-or thinking. Med Educ. 2019;53(1):64-75. https://doi.org/10.1111/medu.13656
- Cheung K, Rogoza C, Chung AD, Kwan BYM. Analyzing the administrative burden of competency based medical education. Can Assoc Radiol J 2021:08465371211038963. https://doi.org/10.1177/08465371211038963