Abstracts
Abstract
Massive open online courses (MOOCs) offer rich opportunities to comprehend learners’ learning experiences by examining their self-generated course evaluation content. This study investigated the effectiveness of fine-tuned BERT models for the automated classification of topics in online course reviews and explored the variations of these topics across different disciplines and course rating groups. Based on 364,660 course review sentences across 13 disciplines from Class Central, 10 topic categories were identified automatically by a BERT-BiLSTM-Attention model, highlighting the potential of fine-tuned BERTs in analysing large-scale MOOC reviews. Topic distribution analyses across disciplines showed that learners in technical fields were engaged with assessment-related issues. Significant differences in topic frequencies between high- and low-star rating courses indicated the critical role of course quality and instructor support in shaping learner satisfaction. This study also provided implications for improving learner satisfaction through interventions in course design and implementation to monitor learners’ evolving needs effectively.
Keywords:
- learner-generated content,
- automatic classification,
- fine-tuned,
- BERTs,
- course evaluation
Download the article in PDF to read it.
Download
Appendices
Biographical notes
Xieling Chen is an Associate Professor at Guangzhou University, China. Her research interests include artificial intelligence in education and text mining. She has over 60 publications. Stanford University has listed her as one of the World's Top 2% Scientists in 2022, 2023, and 2024.
Di Zou is an Associate Professor at The Hong Kong Polytechnic University. Her research interests include AI in language education and TELL. She has over 150 publications. Stanford University has listed her as one of the World's Top 2% Scientists in 2021, 2022, 2023, and 2024. She is an Editor of Computers & Education.
Haoran Xie is a Professor at Lingnan University, Hong Kong. His research interests include artificial intelligence in education and big data. He has over 320 publications. He is the Editor-in-Chief/Associate Editor of several SCI/SSCI journals. Stanford University has listed him as one of the World's Top 2% Scientists in 2021, 2022, 2023, and 2024.
Gary Cheng is an Associate Professor at The Education University of Hong Kong, Hong Kong. He has been actively involved in organizing events and activities with colleagues to promote STEM among children and adolescents (e.g., STEM Competition in Smart Home Design and STEAM Education: 3-D Chinese Cultural Architectural Design Competition). His interests include data mining, deep learning, and computer programming education.
Zongxi Li is an Assistant Professor at Lingnan University, Hong Kong. He has authored over 27 papers including top-tier journal articles, such as Pattern Recognition, Knowledge-based Systems, Information Processing & Management, and IEEE Transactions on Affective Computing, and high-impact conference proceedings, such as AAAI and ACL. He was awarded the Best Paper Award from WI-IAT and the Best Paper Runner-up Award from BESC.
Fu Lee Wang is the Dean and Professor at Hong Kong Metropolitan University, Hong Kong. His research interests include e-learning and information retrieval. Professor Wang has over 300 publications and 40 grants with more than 80 million Hong Kong dollars. He was also the Chair of ACM Hong Kong Chapter and IEEE Hong Kong Section Computer Society.
Bibliography
- Alario-Hoyos, C., Estévez-Ayres, I., Pérez-Sanagustín, M., Delgado Kloos, C., & Fernández-Panadero, C. (2017). Understanding learners’ motivation and learning strategies in MOOCs. International Review of Research in Open and Distributed Learning, 18(3), 119–137. https://doi.org/10.19173/irrodl.v18i3.2996
- Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., & Gašević, D. (2020, March). How good is my feedback? A content analysis of written feedback. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge (pp. 428–437). https://doi.org/10.1145/3375462.3375477
- Cavalcanti, A. P., Mello, R. F., Gašević, D., & Freitas, F. (2023). Towards explainable prediction feedback messages using BERT. International Journal of Artificial Intelligence in Education, 34, 1046–1071. https://doi.org/10.1007/s40593-023-00375-w
- Chen, X., Zou, D., Cheng, G., & Xie, H. (2024). Deep neural networks for the automatic understanding of the semantic content of online course reviews. Education and Information Technologies, 29(4), 3953–3991. https://doi.org/10.1007/s10639-023-11980-6
- Conrad, D., & Openo, J. (2018). Assessment strategies for online learning: Engagement and authenticity. Athabasca University Press. https://doi.org/10.15215/aupress/9781771992329.01
- El-Rashidy, M. A., Farouk, A., El-Fishawy, N. A., Aslan, H. K., & Khodeir, N. A. (2023). New weighted BERT features and multi-CNN models to enhance the performance of MOOC posts classification. Neural Computing and Applications, 35(24), 18019–18033. https://doi.org/10.1007/s00521-023-08673-z
- Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320–341. https://doi.org/10.1111/bjet.12235
- Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers & Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724
- Li, L., Johnson, J., Aarhus, W., & Shah, D. (2022). Key factors in MOOC pedagogy based on NLP sentiment analysis of learner reviews: What makes a hit. Computers & Education, 176, 104354. https://doi.org/10.1016/j.compedu.2021.104354
- Liu, Z., Kong, X., Chen, H., Liu, S., & Yang, Z. (2023). MOOC-BERT: Automatically identifying learner cognitive presence from MOOC discussion data. IEEE Transactions on Learning Technologies, 16(4), 528–542. https://doi.org/10.1109/TLT.2023.3240715
- Moore, M. G. (2013). The theory of transactional distance. In Moore, M. G. (Eds.) Handbook of distance education (pp. 66–85). Routledge. https://doi.org/10.4324/9780203803738
- Qaddumi, H., Bartram, B., & Qashmar, A. L. (2021). Evaluating the impact of ICT on teaching and learning: A study of Palestinian students’ and teachers’ perceptions. Education and Information Technologies, 26(2), 1865–1876. https://doi.org/10.1007/s10639-020-10339-5
- Sebbaq, H., & El Faddouli, N. (2022). Fine-tuned BERT model for large scale and cognitive classification of MOOCs. International Review of Research in Open and Distributed Learning, 23(2), 170–190. https://doi.org/10.19173/irrodl.v23i2.6023
- Wulff, P., Mientus, L., Nowak, A., & Borowski, A. (2023). Utilizing a pretrained language model (BERT) to classify preservice physics teachers’ written reflections. International Journal of Artificial Intelligence in Education, 33(3), 439–466. https://doi.org/10.1007/s40593-022-00290-6
- Yousef, A. M. F., & Sumner, T. (2021). Reflections on the last decade of MOOC research. Computer Applications in Engineering Education, 29(4), 648–665. https://doi.org/10.1002/cae.22334