Abstracts
Abstract
Student-written evaluation (N = 600) of professors was examined to determine the emotional tone of the words used to evaluate faculty. Using the revised Dictionary of Affect (DOA; Whissell, 2009), evaluation words (N = 26,764) uploaded to the Rate My Professors website between 2018 and October of 2023 were measured for their pleasantness, activation, and imagery. Overall, the emotional tone of the students’ written evaluation was very close to the DOA’s definition of everyday English (M = 50) for all three categories: pleasantness (M = 51.1, SD = 6.3), activation (M = 52.2, SD = 4.8), and imagery (M = 50.2, SD = 7.4). The results indicated that the written evaluations were uniform in expression and emotional tone: neither very pleasant/unpleasant, active/passive, or imagery/abstract. While significant relationships were found with professor quality and difficulty ratings, the number of words in the evaluation, and the instructor’s gender, all associations had small correlational strengths and weak effect sizes, indicating that the variables might not be strong predictors of the emotional tone of student evaluations. If student written evaluations are not emotionally charged, then there is an opportunity to reduce any negative feelings faculty members have attached to the process.
Keywords:
- academia,
- emotional tone,
- student evaluation of teaching,
- student evaluations
Résumé
L’évaluation écrite des professeures et professeurs réalisée par les personnes étudiantes (N = 600) a été examinée pour déterminer le ton émotionnel des mots utilisés pour les évaluer. À l’aide du dictionnaire Dictionary of Affect (DOA ; Whissell, 2009), les mots d’évaluation (N = 26,764) téléversés sur le site web Rate My Professors entre 2018 et octobre 2023 ont été mesurés pour leur caractère agréable, leur activation et leur imagerie. Dans l’ensemble, le ton émotionnel de l’évaluation écrite réalisée par les personnes étudiantes était très proche de la définition de l’anglais courant du DOA (M = 50) pour les trois catégories : caractère agréable (M = 51,1, SD = 6,3), activation (M = 52,2, SD = 4,8) et imagerie (M = 50,2, SD = 7,4). Les résultats indiquent que les évaluations écrites étaient uniformes en termes d’expression et du ton émotionnel : ni très agréables/désagréables, ni actives/passives, ni imagées/abstraites.Bien que des relations significatives aient été trouvées avec la qualité de la personne enseignante et les notes de difficulté, le nombre de mots dans l'évaluation et le genre de la personne enseignante, toutes les associations avaient des forces de corrélation faibles et des tailles d’effet faibles, ce qui indique que les variables pourraient ne pas être des prédicteurs forts du ton émotionnel des évaluations réalisées par les personnes étudiantes. Si les évaluations écrites réalisées par les personnes étudiantes ne sont pas chargées d’émotion, il est possible de réduire les sentiments négatifs que les personnes enseignantes attachent au processus.
Mots-clés :
- académie,
- ton émotionnel,
- évaluation de l'enseignement par les personnes étudiantes,
- évaluations par les personnes étudiantes
Appendices
Bibliography
- Abd-Elrahman, A., Andreu, M., & Abbott, T. (2010). Using text data mining techniques for understanding free-style question answers in course evaluation forms. Research in Higher Education Journal, 9, 1.
- Afonso, N. M., Cardozo, L. J., Mascarenhas, O. A. J., Aranha, A. N. F., & Shah, C. (2005). Are anonymous evaluations a better assessment of faculty teaching performance? A comparative analysis of open and anonymous evaluation processes. Family Medicine, 37(1), 43–47.
- Algozzine, B., Gretes, J., Flowers, C., Howley, L., Beattie, J., Spooner, F., Mohanty, G., & Bray, M. (2004). Student evaluation of college teaching: A practice in search of principles. College Teaching, 52(4), 134–141. https://doi.org/10.3200/CTCH.52.4.134-141
- Arthur, L. (2009). From performativity to professionalism: Lecturers’ responses to student feedback. Teaching in Higher Education, 14(4), 441–454. https://doi.org/10.1080/13562510903050228
- Balam, E. M., & Shannon, D. M. (2010). Student ratings of college teaching: A comparison of faculty and their students. Assessment & Evaluation in Higher Education, 35(2), 209–221. https://doi.org/10.1080/02602930902795901
- Barnes, L. L. B., & Barnes, M. W. (1993). Academic discipline and generalizability of student evaluations of instruction. Research in Higher Education, 34, 135–149. https://doi.org/10.1007/BF00992160
- Blackburn, R. T., & Clark, M. J. (1975). An assessment of faculty performance: Some correlates between administrator, colleague, student and self-ratings. Sociology of Education, 48(2), 242–256. https://doi.org/10.2307/2112478
- Brown, M. J., Baillie, M., & Fraser, S. (2009). Rating RateMyProfessors.com: A comparison of online and official student evaluations of teaching. College Teaching, 57(2), 89–92. https://doi.org/10.3200/CTCH.57.2.89-92
- Burdsal, C. A., & Bardo, J. W. (1986). Measuring student’s perceptions of teaching: Dimensions of evaluation. Educational and Psychological Measurement, 46(1), 63–79. https://doi.org/10.1177/0013164486461006
- Carmack, H. J., & LeFebvre, L. E. (2019). “Walking on eggshells”: Traversing the emotional and meaning making processes surrounding hurtful course evaluations. Communication Education, 68(3), 350–370. https://doi.org/10.1080/03634523.2019.1608366
- Clayson, D. E. (2018). Student evaluation of teaching and matters of reliability. Assessment & Evaluation in Higher Education, 43(4), 666–681. https://doi.org/10.1080/02602938.2017.1393495
- Colardarci, T., & Kornfield, I. (2007). RateMyProfessors com versus formal in-class student evaluations of teaching. Practical Assessment, Research and Evaluation, 44(12), 1–15. https://doi.org/10.7275/26ke-yz55
- Cook, S., Watson, D., & Webb, R. (2024). Performance evaluation in teaching: Dissecting student evaluations in higher education. Studies in Educational Evaluation, 81, 101342. https://doi.org/10.1016/j.stueduc.2024.101342
- Cooper, A. K., & Coetzee, S. (2020). On the ethics of using publicly-available data. In M. Hattingh, M. Matthee, H. Smuts, I. Pappas, Y. K. Dwivedi, & M. Mäntymäki (Eds)., Responsible design, implementation and use of information and communication technology: I3E 2020. Lecture notes in computer science, 12067 (pp. 159–171). Springer International Publishing. https://doi.org/10.1007/978-3-030-45002-1_14
- Dahal, K., & Rafiq, R. I. (2023, May). What makes a good course and professor: Through the lens of RateMyProfessor website. In Proceedings of the 2023 7th International Conference on Information System and Data Mining (ICISDM) (pp. 1–9). https://doi.org/10.1145/3603765.3603767
- Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online, and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. https://doi.org/10.1016/j.chb.2014.04.005
- Ellett, C. D., Loup, K. S., Culross, R. R., McMullen, J. H., & Rugutt, J. K. (1997). Assessing enhancement of learning, personal learning environment, and student efficacy: Alternatives to traditional faculty evaluation in higher education. Journal of Personnel Evaluation in Education, 11, 167–192. https://doi.org/10.1023/A:1007989320210
- Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education, 30, 137–194. https://doi.org/10.1007/BF00992716
- Felton, J., Mitchell, J., & Stinson, M. (2004). Web-based student evaluations of professors: The relations between perceived quality, easiness and sexiness. Assessment & Evaluation in Higher Education, 29(1), 91–108. https://doi.org/10.1080/0260293032000158180
- Foster, A. L. (2003). Picking apart Pick-A-Prof. Chronicle of Higher Education, 49(26), A33–A34.
- Howell, A. J., & Symbaluk, D. G. (2001). Published student ratings of instruction: Revealing and reconciling the views of students and faculty. Journal of Educational Psychology, 93(4), 790–796. https://psycnet.apa.org/doi/10.1037/0022-0663.93.4.790
- Janss, R., Rispens, S., Segers, M., & Jehn, K. A. (2012). What is happening under the surface? Power, conflict and the performance of medical teams. Medical Education, 46(9), 838–849. https://doi.org/10.1111/j.1365-2923.2012.04322.x
- Kember, D., Leung, D. Y. P., & Kwan, K. P. (2002). Does the use of student feedback questionnaires improve the overall quality of teaching? Assessment & Evaluation in Higher Education, 27(5), 411–425. https://doi.org/10.1080/0260293022000009294
- Kulik, J. A. (2001). Student ratings: Validity, utility, and controversy. New Directions for Institutional Research, 2001(109), 9–25. https://doi.org/10.1002/ir.1
- Legg, A. M., & Wilson, J. H. (2012). RateMyProfessors.com offers biased evaluations. Assessment & Evaluation in Higher Education, 37(1), 89–97. https://doi.org/10.1080/02602938.2010.507299
- Lutovac, S., Kaasila, R., Komulainen, J., & Maikkola, M. (2017). University lecturers’ emotional responses to and coping with student feedback: A Finnish case study. European Journal of Psychology of Education, 32, 235–250. https://doi.org/10.1007/s10212-016-0301-1
- MacNell, L., Driscoll, A., & Hunt, A. N. (2015). What’s in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40, 291–303. https://doi.org/10.1007/s10755-014-9313-4
- Marsh, H. W., & Roche, L. (1993). The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness. American Educational Research Journal, 30(1), 217–251. https://doi.org/10.3102/00028312030001217
- Murray, K. B., & Zdravkovic, S. (2016). Does MTV really do a good job of evaluating professors? An empirical test of the internet site RateMyProfessors.com. Journal of Education for Business, 91(3), 138–147. https://doi.org/10.1080/08832323.2016.1140115
- Olvet, D. M., Willey, J. M., Bird, J. B., Rabin, J. M., Pearlman, R. E., & Brenner, J. (2021). Third year medical students impersonalize and hedge when providing negative upward feedback to clinical faculty. Medical Teacher, 43(6), 700–708. https://doi.org/10.1080/0142159X.2021.1892619
- Otto, J., Sanford Jr, D. A., & Ross, D. N. (2008). Does ratemyprofessor.com really rate my professor? Assessment & Evaluation in Higher Education, 33(4), 355–368. https://doi.org/10.1080/02602930701293405
- Overall, J. U., & Marsh, H. W. (1980). Students’ evaluations of instruction: A longitudinal study of their stability. Journal of Educational Psychology, 72(3), 321–325. https://psycnet.apa.org/doi/10.1037/0022-0663.72.3.321
- Penny, A. R. (2003). Changing the agenda for research into students’ views about university teaching: Four shortcomings of SRT research. Teaching in Higher Education, 8(3), 399–411. https://doi.org/10.1080/13562510309396
- Quansah, F., Cobbinah, A., Asamoah-Gyimah, K., & Hagan Jr., J. E. (2024, February). Validity of student evaluation of teaching in higher education: A systematic review. Frontiers in Education, 9, 1–12. https://doi.org/10.3389/feduc.2024.1329734
- Reid, L. D. (2010). The role of perceived race and gender in the evaluation of college teaching on RateMyProfessors.com. Journal of Diversity in Higher Education, 3(3), 137–152. https://psycnet.apa.org/doi/10.1037/a0019865
- Robins, L., Smith, S., Kost, A., Combs, H., Kritek, P. A., & Klein, E. J. (2020). Faculty perceptions of formative feedback from medical students. Teaching and Learning in Medicine, 32(2), 168–175. https://doi.org/10.1080/10401334.2019.1657869
- Shah, M., & Pabel, A. (2020). Making the student voice count: Using qualitative student feedback to enhance the student experience. Journal of Applied Research in Higher Education, 12(2), 194–209. https://doi.org/10.1108/JARHE-02-2019-0030
- Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment & Evaluation in Higher Education, 25(4), 397–405. https://doi.org/10.1080/713611436
- Sidwell, D., Lee, D., Zimmerman, P.-A., Bentley, S., & Barton, M. (2025). Teaching faculty experiences with student evaluation of instruction: A mixed-methods study. Teaching and Learning in Nursing, 20(1), e276-e284. https://doi.org/10.1016/j.teln.2024.11.009
- Silva, K. M., Silva, F. J., Quinn, M. A., Draper, J. N., Cover, K. R., & Munoff, A. A. (2008). Rate my professor: Online evaluations of psychology instructors. Teaching of Psychology, 35(2), 71–80. https://doi.org/10.1080/00986280801978434
- Sonntag, M. E., Bassett, J. R., & Snyder, T. (2009). An empirical test of the validity of student evaluations of teaching made on RateMyProfessors.com. Assessment & Evaluation in Higher Education, 34(5), 499–504. https://doi.org/10.1080/02602930802079463
- Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870
- Stroebe, W. (2020). Student evaluations of teaching encourages poor teaching and contributes to grade inflation: A theoretical and empirical analysis. Basic and Applied Social Psychology, 42(4), 276–294. https://doi.org/10.1080/01973533.2020.1756817
- Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good? A cross-case analysis of the connection between teacher effectiveness and student achievement. Journal of Teacher Education, 62(4), 339–355.
- Stupans, I., McGuren, T., & Babey, A. M. (2016). Student evaluation of teaching: A study exploring student rating instrument free-form text comments. Innovative Higher Education, 41, 33–42. https://doi.org/10.1007/s10755-015-9328-5
- Timmerman, T. (2008). On the validity of RateMyProfessors.com. Journal of Education for Business, 84(1), 55–61. https://doi.org/10.3200/JOEB.84.1.55-61
- Uttl, B. (2021). Lessons learned from research on student evaluation of teaching in higher education. Student Feedback on Teaching in Schools: Using Student Perceptions for the Development of Teaching and Teachers, 13, 237–256. https://doi.org/10.1007/978-3-030-75150-0_15
- Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007
- Wagenaar, T. C. (1995). Student evaluation of teaching: Some cautions and suggestions. Teaching Sociology, 23(1), 64–68. https://doi.org/10.2307/1319382
- Wang, H., Burić, I., Chang, M., & Gross, J. J. (2023). Teachers’ emotion regulation and related environmental, personal, instructional, and well-being factors: A meta-analysis. Social Psychology of Education, 26, 1651–1696. https://doi.org/10.1007/s11218-023-09810-1
- Whissell, C. (1996). Traditional and emotional stylometric analysis of the songs of Beatles Paul McCartney and John Lennon. Computers and the Humanities, 30, 257–265. https://doi.org/10.1007/BF00055109
- Whissell, C. (2009). Using the Revised Dictionary of Affect in Language to quantify the emotional undertones of samples of natural language. Psychological Reports, 105(2), 509–521. https://doi.org/10.2466/PR0.105.2.509-521
- Whissell, C. (2012a, January). Emotional consistency as evidence of dynamic equivalence among English translations of the Bible. Comprehensive Psychology, 1. https://doi.org/10.2466/28.49.CP.1.15
- Whissell, C. (2012b). The emotionality and complexity of public political language in Canada’s Question Period. English Language and Literature Studies, 2(4), 68–76. http://dx.doi.org/10.5539/ells.v2n4p68
- White, E., Whissell, C., & Dewson, M. (1989). An objective quantification of the affective tone of language in children’s television programing. Journal of Social Behavior & Personality, 4(1), 127–131.
- Wright, S. L., & Jenkins-Guarnieri, M. A. (2012). Student evaluations of teaching: Combining the meta-analyses and demonstrating further evidence for effective use. Assessment & Evaluation in Higher Education, 37(6), 683–699. https://doi.org/10.1080/02602938.2011.563279
- Yan, Z., & Carless, D. (2022). Self-assessment is about more than self: The enabling role of feedback literacy. Assessment & Evaluation in Higher Education, 47(7), 1116–1128. https://doi.org/10.1080/02602938.2021.2001431
- Zhao, J., & Gallant, D. J. (2012). Student evaluation of instruction in higher education: Exploring issues of validity and reliability. Assessment & Evaluation in Higher Education, 37(2), 227–235. https://doi.org/10.1080/02602938.2010.523819