Adaptive self-prompting in agentic LLM frameworks for embedded code fault detection

This thesis investigates adaptive self-prompting in agentic large language model (LLM) frameworks for code fault detection in embedded systems. Traditional static analysis and existing LLM-based approaches often rely on fixed prompts and exhibit overconfidence, leading to misclassifications. To address these limitations, we propose two complementary frameworks: Agentic Retrieval-Augmented Generation (A-RAG) and Agentic Supervised Fine-Tuning (A-SFT), along with a hybrid architecture integrating both. A-RAG employs confidence-triggered adaptive retrieval on a curated vector knowledge base of coding standards, while A-SFT leverages self-evaluation and instruction adaptation to refine the model’s reasoning during training. The frameworks are evaluated on a unified dataset combining the Toyota ITC benchmark and a subset of Big-Vul. Experimental results demonstrate that adaptive self-prompting improves overall F1-score by 6.7-18% over static baselines and reduces high-confidence misclassifications by 6.1-17.6%. Pairwise improvements are statistically significant under Bonferroni-corrected McNemar tests, indicating enhanced safety for embedded C code analysis.

Name thesis:degree_name

Master of Applied Science (MASc)

Discipline thesis:degree_discipline

Software Engineering

Grantor

University of Ontario Institute of Technology

Year dc:date.issued

2026

Author dc:creator

• Muhtadi, Maher

Advisors dc:contributor.advisor

• Mahmoud, Qusay H.
• Azim, Akramul

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/10155/2091

OAI identifier oai:identifier

oai:ontariotechu.scholaris.ca:10155/2091