Résumés
Abstract
Introduction: Point-of-care ultrasound (POCUS) is a valuable clinical skill that improves clinical care but requires substantial training. Validated assessment tools provide empirical evidence regarding trainee performance while also informing program-level evaluation. We developed two POCUS-specific stations for objective structured clinical examinations (OSCEs) to assess skill acquisition and inform best practices in undergraduate medical education.
Methods: A multidisciplinary group of POCUS educators identified two POCUS applications (pleural effusion and abdominal free fluid) well suited for the undergraduate level. A modified Delphi approach was used to develop POCUS-application-specific skill checklists and global rating scale. Two medical programs piloted the stations to inform reliability.
Results: Across two sites, 46 and 41 students participated in the pleural effusion and abdominal free fluid stations respectively. Checklists showed high internal reliability, with Cronbach’s alpha of 0.85 (95% CI 0.71-0.93) for the pleural effusion station and 0.87 (95% CI 0.74-0.95) for the abdominal free fluid station. Krippendorff’s alpha, a measure of inter-rater reliability, was also equally strong at 0.85 (95% CI 0.43-0.94) and 0.83 (95% CI 0.50-0.94) respectively.
Conclusion: Both POCUS OSCE stations demonstrated good internal and inter-rater reliability. Deployment of these OSCE stations at programs with integrated POCUS curricula may help refine programming and training expectations.
Résumé
Introduction : L'échographie au point d'intervention (POCUS) est une compétence clinique précieuse qui améliore les soins cliniques, mais qui nécessite une formation approfondie. Des outils d'évaluation validés fournissent des preuves empiriques concernant les performances des médecins résidents tout en éclairant l'évaluation au niveau du programme. Nous avons développé deux stations propres à POCUS pour les examens cliniques objectifs structurés (ECOS) afin d'évaluer l'acquisition des compétences et d'éclairer les meilleures pratiques dans l'enseignement médical de premier cycle.
Méthodes : Un groupe multidisciplinaire d'éducateurs POCUS a identifié deux applications POCUS (épanchement pleural et liquide libre abdominal) bien adaptées au niveau de premier cycle. Une approche Delphi modifiée a été utilisée pour développer des listes de contrôle des compétences spécifiques à l'application POCUS et une échelle d'évaluation globale. Deux programmes médicaux ont testé les stations afin d'en vérifier la fiabilité.
Résultats : Sur les deux sites, 46 et 41 étudiants ont respectivement participé aux stations consacrées à l'épanchement pleural et au liquide libre abdominal. Les listes de contrôle ont montré une fiabilité interne élevée, avec un coefficient alpha de Cronbach de 0,85 (IC à 95 % : 0,71-0,93) pour la station consacrée à l'épanchement pleural et de 0,87 (IC à 95 % : 0,74-0,95) pour la station consacrée au liquide libre abdominal. Le coefficient alpha de Krippendorff, qui mesure la fiabilité inter-évaluateurs, était également élevé, avec respectivement 0,85 (IC à 95 % : 0,43-0,94) et 0,83 (IC à 95 % : 0,50-0,94).
Conclusion : Les deux stations POCUS ECOS ont démontré une bonne fiabilité interne et inter-évaluateurs. Le déploiement de ces stations ECOS dans des programmes intégrant des cursus POCUS pourrait contribuer à affiner les attentes en matière de programmation et de formation.
Parties annexes
Bibliography
- Diaz-Gomez JL, Mayo PH, Koenig SJ. Point-of-care ultrasonography. N Engl J Med. 2021;385:1593-1602. https://doi.org/10.1056/nejmra1916062
- Sena A, Alerhand S, Lamba S. Milestone approach to designing a point-of-care ultrasound curriculum for transition-to-residency programs in the United States. Teach Learn Med. 2020;33(3):270-81. https://doi.org/10.1080/10401334.2020.1814296
- Tarique U, Tang B, Singh M, Kulasegaram KM, Ailon J. Ultrasound curricula in undergraduate medical education: a scoping review. J Ultrasound Med. 2018;37(1):69-82. https://doi.org/10.1002/jum.14333
- Davis JJ, Wessner CE, Potts J, Au AK, Pohl CA, Fields JM. Ultrasonography in undergraduate medical education: a systematic review. J Ultrasound Med. 2018;37(11):2667-79. https://doi.org/10.1002/jum.14628
- Steinmetz P, Dobrescu O, Oleskevich S, Lewis J. Bedside ultrasound education in Canadian medical schools: a national survey. Can Med Educ J. 2016;7(1):e78-e86. https://doi.org/10.36834/cmej.36646
- Russell FM, Zakeri B, Herbert A, et al. The state of point-of-care ultrasound training in undergraduate medical education: findings from a national survey. Acad Med. 2022;97(5):723-7. https://doi.org/10.1097/ACM.0000000000004512
- Hoppmann RA, Mladenovic J, Melniker L, et al. International consensus conference recommendations on ultrasound education for undergraduate medical students. Ultrasound J. 2022;14:31. https://doi.org/10.1186/s13089-022-00279-1
- Ma IWY, Steinmetz P, Weerdenburg K, et al. The Canadian medical student ultrasound curriculum: a statement from the Canadian Ultrasound Consensus for Undergraduate Medical Education Group. J Ultrasound Med. 2020;39(7):1279-87. https://doi.org/10.1002/jum.15218
- Dinh VA, Lakoff D, Hess J, et al. Medical student core clinical ultrasound milestones: a consensus among directors in the United States. J Ultrasound Med. 2016;35(2):421-34. https://doi.org/10.7863/ultra.15.07080
- DeBiasio C, Pageau P, Shefrin A, Woo MY, Cheung WJ. Point-of-care-ultrasound in undergraduate medical education: a scoping review of assessment methods. Ultrasound J. 2023;15(1):30. https://doi.org/10.1186/s13089-023-00325-6
- Khan KZ, Ramachandran S, Gaunt K, Pushkar P. The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: an historical and theoretical perspective. Med Teach. 2013;35(9):e1437-e46. https://doi.org/10.3109/0142159x.2013.818634
- Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9):S63-S7. https://doi.org/10.1097/00001888-199009000-00045
- Patrício MF, Julião M, Fareleira F, Carneiro AV. Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med Teach. 2013;35(6):503-14. https://doi.org/10.3109/0142159x.2013.774330
- Sheppard G, Williams KL, Metcalfe B, et al. Using Kane’s framework to build an assessment tool for undergraduate medical student’s clinical competency with point of care ultrasound. BMC Med Educ. 2023;23:43. https://doi.org/10.1186/s12909-023-04030-9
- Damewood SC, Leo M, Bailitz J, et al. Tools for measuring clinical ultrasound competency: recommendations from the Ultrasound Competency Work Group. AEM Educ Train. 2019;4(Suppl 1):S106-S12. https://doi.org/10.1002/aet2.10368
- Royal College of Physicians and Surgeons of Canada. Objectives of training in the specialty of emergency medicine. Available at: https://www.royalcollege.ca/rcsite/documents/ibd/emergency_otr_e.pdf. [Accessed Dec 10, 2024].
- Royal College of Physicians and Surgeons of Canada. Entrustable Professional Activities for anesthesiology Version 3.0. Available from: https://www.royalcollege.ca/rcsite/documents/cbd/epa-guide-anesthesiology-v3-e.pdf. [Accessed Dec 10, 2024].
- Emergency ultrasound guidelines. Ann Emerg Med. 2009;53:550-70. https://doi.org/10.1016/j.annemergmed.2008.12.013
- Blehar DJ, Barton B, Gaspari RJ. Learning curves in emergency ultrasound education. Acad Emerg Med. 2015;22(5):574-82. https://doi.org/10.1111/acem.12653
- Association of American Medical Colleges. Core EPAs guiding principles. Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide. Available at: https://store.aamc.org/downloadable/download/sample/sample_id/63/%20. [Accessed Dec 10, 2024].
- Association of Faculties of Medicine of Canada. AFMC Entrustable Professional Activities for the Transition from Medical School to Residency. Available at: https://www.afmc.ca/wp-content/uploads/2022/10/AFMC_Entrustable-Professional-Activities_EN_Final.pdf. [Accessed Nov 30, 2024].
- Steinmetz P, Oleskevich S, Dyachenko A, McCusker J, Lewis J. Accuracy of medical students in detecting pleural effusion using lung ultrasound as an adjunct to the physical examination. J Ultrasound Med. 2018;37:2545-52. https://doi.org/10.1002/jum.14612
- Olszynski P, Russell M, Neufeld A, Malin G. The clinical ultrasonography elective in clerkship (CUSEC): a pilot elective for senior clerkship students at the University of Saskatchewan. Can Med Educ J. 2020;11(1):e144-6. https://doi.org/10.36834/cmej.61810
- Kern DE, Thomas PA, Hughes MT. Curriculum development for medical education: a six-step approach. Baltimore: Johns Hopkins University Press; 2009.
- Bahner DP, Hughes D, Royall NA. I-AIM: a novel model for teaching and performing focused sonography. J Ultrasound Med. 2012;31(2):295-300. https://doi.org/10.7863/jum.2012.31.2.295
- Hejri SM, Jalili M, Muijtjens AM, Van Der Vleuten CP. Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination. J Res Med Sci. 2013;18(10):887-91.