Ranking features used in modeling student collaboration using multimodal learning analytics

This dissertation tackles a persistent challenge in computer-supported collaborative learning (CSCL): how to identify the behavioral signals that matter most for detecting productive peer interaction in noisy, face-to-face classrooms. Drawing on the fourth design cycle of the NSF-funded CSTEPS project, the study asks (a) how individual data streams—video, audio, log traces, and speech transcripts—characterize specific collaborative behaviors, (b) whether fusing those streams improves machine classification, and (c) which fine-grained features contribute most to accurate prediction. By ranking features across modalities, the work aims to guide researchers and learning-technology designers in selecting sensors and analytics that yield the greatest instructional value. A quasi-experimental field study captured 31 undergraduate engineering groups over seven weeks (53 students wore head-mounted microphones, generating 34 multimodal sessions) during tablet-mediated problem-solving discussions. After synchronizing video, individual audio, shared-tablet logs, and automatic transcripts, hundreds of prosodic, linguistic, visual, and interaction features were extracted. Feature space was reduced through correlation filtering, PCA, and recursive feature elimination, then modeled with six classical classifiers (decision tree, random forest, SVM, AdaBoost, XGBoost, Naïve Bayes). A five-fold group-level cross-validation protocol, augmented by SMOTE to counter class imbalance, ensured generalization to unseen groups. Unimodal baselines were followed by early- and late-fusion multimodal models. Unimodal models confirmed complementary strengths: log interaction ratios best signaled On Task engagement (balanced accuracy = 0.84) while audio prosody captured OffTopicTalk and ToolTalk. Early fusion raised balanced accuracy for most codes (e.g., 0.84 for On Task), but late fusion delivered the most consistent gains, peaking at 0.90 balanced accuracy and 0.93 F1 for On Task, with moderate improvements for cognitively demanding behaviors such as MetaCognition and BuildingIdeas. Feature-importance analysis revealed that (i) loudness and pitch slopes were dominant audio cues for unstructured off-task dialogue, (ii) sentiment-laden textual n-grams differentiated idea-building talk, (iii) gaze minima and cheek-raiser action units flagged explanations, and (iv) scroll-balance ratios in the whiteboard log robustly indexed group engagement. Methodologically, the study offers a tested pipeline—group-wise cross-validation, targeted SMOTE, and decision-level fusion—for researchers modeling small-group collaboration at scale. Theoretically, it demonstrates that multimodal signals are synergistic rather than redundant: preserving modality-specific classifiers before aggregation yields the most balanced view of complex behaviors. Practically, the ranked feature set pinpoints a lean, interpretable subset (e.g., loudness variance, gaze convergence, scrolling equality) that can power real-time teacher dashboards without excessive sensor overhead. Future work should extend adaptive fusion strategies and leverage emerging 360° video and unobtrusive audio wearables to boost data fidelity and ecological validity.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Curriculum and Instruction

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Rajarathinam, Robin Jephthah

Contributors dc:contributor

• Mercier, Emma
• Kang, Jina
• D'Angelo, Cynthia
• Paquette, Luc

• Collaborative Learning
• Multimodal Learning Analytics
• Collaborative Learning Analytics
• Feature Ranking
• Machine Learning
• Group Behavior Modeling
• Engineering Education

Statement dc:rights

• Copyright 2025 Robin Jephthah Rajarathinam

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/130113

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F130113