Incorporating LLM-based Interactive Learning Environments in CS Education: Learning Data Structures and Algorithms using the Gurukul platform

Large Language Models (LLMs) have emerged as a revolutionary force in Computer Science Education, offering unprecedented opportunities to facilitate learning and comprehension. Their application in the classroom, however, is not without challenges. LLMs are prone to hallucination and contextual inaccuracies. Furthermore, they risk exposing learning processes to cheating illicit practices and providing explicit solutions that impede the development of critical thinking skills in students. To address these pitfalls and investigate how specialized LLMs can enhance engagement among learners particularly using LLMs, we present Gurukul, a unique coding platform incorporating dual features - Retrieval Augmented Generation and Guardrails. Gurukul's practice feature provides a hands-on code editor to solve DSA problems with the help of a dynamically Guardrailed LLM to prevent explicit code solutions. On the other hand, Gurukul's Study feature incorporates a Retrieval Augmented Generation mechanism that uses OpenDSA as its source of truth, allowing the LLM to fetch and present information accurately and relevantly, thereby trying to overcome the issue of inaccuracies. We present these features to evaluate the user perceptions of LLM-assisted educational tools. To evaluate the effectiveness and utility of Gurukul in a real-world educational setting, we conducted a User Study and a User Expert Review with students (n=40) and faculty (n=2), respectively, from a public state university in the US specializing in DSA courses. We examine student's usage patterns and perceptions of the tool and report reflections from instructors and a series of recommendations for classroom use. Our findings suggest that Gurukul had a positive impact on student learning and engagement in learning DSA. This feedback analyzed through qualitative and quantitative methods indicates the promise of the utility of specialized LLMs in enhancing student engagement in DSA learning.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Computer Science & Applications

Department dc:contributor.department

Computer Science and#38; Applications

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2024

Author dc:creator

• Rachha, Ashwin Kedari

Chair dc:contributor.committeechair

• Seyam, Mohammed Saad Mohamed Elmahdy

Committee members dc:contributor.committeemember

• Brown, Dwayne Christian
• Shaffer, Clifford A.

• Large Language Models
• Retrieval Augmented Generation
• Guardrails
• Computer Science Education
• Adaptive Learning Technologies

Statement dc:rights

• Creative Commons Attribution 4.0 International

Licence dc:rights.uri

• http://creativecommons.org/licenses/by/4.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:41051

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/121208