A Large Language Model and Retrieval-Augmented Generation System to Assist with Undergraduate Academic Advising

<p>The Fowler School of Engineering currently lacks the manpower to fulfill the advising needs of its students. Advisors spend a significant percentage of advising time handling common, basic questions which could be easily answered from existing documentation. In order to tackle this problem, we propose a locally hosted large language model leveraging a modern retrieval augmented generation pipeline in order to guarantee the language model generates the correct advising information for each student. Our RAG pipeline uses Qwen3.6:27b as the main LLM to generate responses, and a fine-tuned variant of Qwen3.5:4b as an intermediate LLM for miscellaneous tasks such as summarizing documents, query routing to collections, and message compression to preserve a manageable context window size. Two main databases are used, with a 4096 dimension Qdrant vector database used for retrieval and a PostgreSQL database used for memory. In addition, within the RAG pipeline, we have incorporated metadata filtering with the student’s catalog year, major, and minor in order to guarantee deterministic retrieval, hybrid search via both semantic dense search and sparse BM25 retrieval, and re-ranking via the BGE-Reranker-V2-M3 cross-encoder to improve retrieval results.</p>

Name thesis:degree_name

Master of Science (MS)

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical Engineering and Computer Science

Year dc:date.available

2026

Author dc:creator

• Au, Spencer

Contributors dc:contributor

• Jonathan D. Humphreys
• Erik J. Linstead
• Chelsea M. Parlett-Pelleriti
• Elia E. Lledo

• LLM
• RAG
• advising
• undergraduate
• human augmented ai
• large language model
• retrieval augmented generation
• Artificial Intelligence and Robotics
• Data Science
• Systems Architecture

Repository record dc:identifier

https://digitalcommons.chapman.edu/eecs_theses/11

OAI identifier oai:identifier

oai:digitalcommons.chapman.edu:eecs_theses-1011