Applying Machine Learning Methods to Analyse Cognitive Presence in MOOC Discussions

This study investigates the application of machine learning methods for analysing cognitive presence in discussion transcripts of Massive Online Open Courses (MOOCs). Efficient real-time support is essential for teaching and learning in large-scale online courses. However, numerous MOOC learners lack tailored and prompt feedback from a limited number of educators. Educators strive to monitor cognitive presence among a vast number of learners, adapting pedagogical designs and teaching strategies for ongoing courses. The applicability of the existing cognitive presence coding rubric, initially developed for small-scale for-credit online courses, to MOOC discussions remains uncertain. This study aims to 1) provide researchers, educators, and learners with accurate automatic analysis of cognitive presence within the Community of Inquiry (CoI) framework in MOOC discussions and 2) validate and refine the existing cognitive presence rubric and the CoI framework, particularly for MOOCs. This study employs a mixed-methods research design. Expert coders manually classified cognitive presence phases in discussion messages, adapting and cross-validating the existing coding rubric. Both single-label and multi-label classifiers were trained on these reliably classified discussion data to analyse cognitive presence phases automatically. Also, a qualitative analysis was conducted on messages where expert coders and automatic classifiers encountered difficulties in identifying a unique cognitive presence phase. This study yielded four significant outcomes: 1) Adapting and validating a widely-used coding rubric to analyse learners’ cognitive presence in MOOCs; 2) Identifying a new set of indicators for cognitive presence in MOOC discussions using single-label machine learning classifiers; 3) Applying multi-label deep learning classifiers to uncover potential subjectivity in manual categorisation of MOOC discussions; 4) Recognising overlaps between adjacent cognitive phases in MOOC discussions and proposing three sub-phases of cognitive presence (i.e., diagnosis, negotiation, and application). The findings contribute to refining cognitive presence in MOOC discussions and exhibit the advantages and limitations of contemporary machine learning techniques for analysing cognitive presence in text-based conversations. Future research can build on these findings to investigate cognitive presence for 1) aiding educators in monitoring learning progress and identifying urgent messages from learners and 2) empowering learners to receive tailored and timely feedback from educators in large-scale online learning environments.

Name thesis:degree_name

PhD

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

software engineering

Grantor dc:publisher

ResearchSpace@Auckland

Year dc:date.issued

2024

Author dc:creator

• Hu, Yuanyuan

Advisors dc:contributor.advisor

• Donald, Claire
• Giacaman, Nasser

Statement dc:rights

• Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Licence dc:rights.uri

• https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm

Handle dc:identifier.uri

https://hdl.handle.net/2292/68791

OAI identifier oai:identifier

oai:researchspace.auckland.ac.nz:2292/68791