Keywords
Medical education
Indicator system
Delphi method
Analytic hierarchy process
Submitted : 2026-06-22
Accepted : 2026-07-07
Published : 2026-07-22
Abstract
Objective: This study addresses the extensive utilization of Massive Open Online Courses (MOOCs) in medical education and acknowledges the multitude of factors affecting their quality. Given the absence of a scientific and comprehensive assessment system, we aimed to formulate an evaluation indicator system for assessing the quality of medical MOOC construction in China and to determine the weight of each indicator. Methods: Using the Donabedian Structure-Process-Outcome model as theoretical guidance and based on relevant literature retrieval and analysis, we first constructed an evaluation indicator system for medical MOOC construction quality. Then, after two rounds of expert consultation, a final indicator content was built, and the weight of the indicators at each level was determined using the Analytic Hierarchy Process (AHP). Results: The effective recovery rate of the questionnaires in the two rounds of expert consultation was 68.75% (11/16) and 93.75% (15/16), respectively, and the expert authority coefficient (Cr) was 0.955 and 0.921, respectively. Kendall’s W coefficients for indicators at the second and third levels were 0.174 and 0.238, respectively (p < 0.01), indicating substantial expert consensus, and the weighted values of the indicators were determined with a consistency ratio (C.R.) < 0.1. Consequently, the final evaluation index system comprises three first-level indicators, six secondary indicators, and 71 tertiary indicators. Conclusion: This study successfully established a comprehensive evaluation indicator system for assessing the quality of medical MOOCs, which is characterized by scientific rigor and reliability, and could serve as a valuable reference for evaluating the quality of Medical MOOCs in China.
References
Williams R, 2024, An Overview of MOOCs and Blended Learning: Integrating MOOC Technologies Into Traditional Classes. IETE Journal of Education, 65(2): 84–91.
Ahmad I, Sharma S, Singh R, et al., 2022, MOOC 5.0: A Roadmap to the Future of Learning. Sustainability, 14(18): 11199.
Pichayakul P, 2024, Academic Outreach During COVID-19 Lockdown at a University Level: Using MOOC as a Strategy. Journal of Education and Innovation, 26(3): 86–99.
Guo Y, 2014, MOOCs and the Future of China’s Higher Education. Journal of Higher Education Management, 5: 29–33.
People’s Daily, 2023, Technology-Enabled Education Shares University Resources, visited on 2026/6/11,http://www.moe.gov.cn/jyb_xwfb/xw_zt/moe_357/2023/2023_zt01/mtbd/202302/t20230212_1043924.html.
Zhu H, Xu J, Wang P, et al., 2023, The Irreplaceable Role of Medical Massive Open Online Courses in China During the COVID-19 Pandemic. BMC Medical Education, 23(1): 323.
Huang H, Jew L, Qi D, 2023, Take a MOOC and Then Drop: A Systematic Review of MOOC Engagement Pattern and Dropout Factor. Heliyon, 9(4): e15220.
Celik B, Cagiltay K, 2024, Uncovering MOOC Completion: A Comparative Study of Completion Rates From Different Perspectives. Open Praxis, 16(3): 445–456.
Zhou L, Tang M, Liu J, 2025, Analysis of Factors Influencing MOOC Quality Based on I-DEMATEL-ISM Method. Systems and Soft Computing, 7: 200220.
Yang L, Zou J, Gao J, et al., 2023, Assessing the Effectiveness of Massive Open Online Courses on Improving Clinical Skills in Medical Education in China: A Meta-Analysis. Heliyon, 9(8): e19263.
Cheng J, Yuen K, Chiu W, 2023, Systematic Review of MOOC Research in Mainland China. Library Hi Tech, 41(5): 1476–1497.
Jain R, 2023, Premchand’s The Chess Players: A Comparative Analysis of YouTube and MOOC Platforms. New Literaria, 4(1): 45–52.
Ponnaiah M, Bhatnagar T, Ganeshkumar P, et al., 2022, Design and Implementation Challenges of Massive Open Online Course on Research Methods for Indian Medical Postgraduates and Teachers – Descriptive Analysis of Inaugural Cycle. BMC Medical Education, 22(1): 369.
Shang Z, 2023, Use of Delphi in Health Sciences Research: A Narrative Review. Medicine, 102(7): e32829.
Robby A, Nurjazuli N, Izzah L, et al., 2025, From Structure to Outcome: The Role of the Donabedian Model in Global Health Service Management Research (1987–2024). E3S Web of Conferences, 650: 02025.
Hsu C, Sandford A, 2007, The Delphi Technique: Making Sense of Consensus. Practical Assessment, Research and Evaluation, 12(1): 10.
Saaty L, 1990, How to Make a Decision: The Analytic Hierarchy Process. European Journal of Operational Research, 48(1): 9–26.
Kelly L, Whitaker A, Burchinal M, et al., 2026, Testing the Structure–Process–Outcome Model Across Different Early Care and Education Policy Contexts. Developmental Psychology, 62(2): 442–461.
Chen H, 2023, Influencing Factors of the Quality of MOOCs Based on the KANO Model. International Journal of Emerging Technologies in Learning, 18(17): 20–32.
Mehrabi M, Safarpour R, Keshtkar A, 2022, Massive Open Online Courses (MOOCs) Dropout Rate in the World: A Protocol for Systematic Review and Meta-Analysis. Interdisciplinary Journal of Virtual Learning in Medical Sciences, 13(2): 85–92.
Hendriks A, de Jong M, Admiraal F, et al., 2020, Instructional Design Quality in Medical Massive Open Online Courses for Integration Into Campus Education. Medical Teacher, 42(2): 156–163.
King I, Lee I, 2023, MOOC Policies Across the Globe. In: King I, Lee W I (eds.) A Decade of MOOCs and Beyond. Springer International Publishing: 61–88.
Liu X, Feng J, Liu C, et al., 2023, Medical Education Systems in China: Development, Status, and Evaluation. Academic Medicine, 98(1): 43–49.
Moore R, Blackmon S, 2022, From the Learner’s Perspective: A Systematic Review of MOOC Learner Experiences (2008–2021). Computers & Education, 190: 104596.
Oo S, Schofield S, Aung T, et al., 2022, Integrating a Mobile-Learning Platform for Enhancing Clinical Teaching: The Learners’ Perspective. International Journal of Learning, Teaching and Educational Research, 21(11): 87–111.
Hui Z, Zewu Z, Jiao H, et al., 2025, Application of ChatGPT-Assisted Problem-Based Learning Teaching Method in Clinical Medical Education. BMC Medical Education, 25(1): 50.
Sun H, Chen H, Dong C, et al., 2021, Building a First-Class Nursing Major With National Standards for Nursing Teaching Quality. Chinese Journal of Nursing Education, 18(5): 389–394.
Meng Q, Wang F, Chen K, et al., 2026, Development and Evaluation of an Evidence-Based Management Indicator System for Transitional Care in Preterm Infants Using a Delphi and AHP Approach. Journal of Nursing Management, 2026(1): 2638144.
Paré G, Cameron F, Poba-Nzaou P, et al., 2013, A Systematic Assessment of Rigor in Information Systems Ranking-Type Delphi Studies. Information & Management, 50(5): 207–217.
Hayes B, Fitzgerald D, Doherty S, et al., 2015, Quality Care, Public Perception and Quick-Fix Service Management: A Delphi Study on Stressors of Hospital Doctors in Ireland. BMJ Open, 5(12): e009564.
Donohoe H, Stellefson M, Tennant B, 2012, Advantages and Limitations of the e-Delphi Technique: Implications for Health Education Researchers. American Journal of Health Education, 43(1): 38–46.