Enhancing Retrieval Augmented Generation Through Robust Information Retrieval

Retrieval-Augmented Generation (RAG) is a popular technique for grounding the responses of Large Language Models (LLMs). RAG works by extending Information Retrieval (IR) to incorporate LLMs that generate responses based on retrieved information and a query. RAG is commonly used within the field of Natural Language Processing (NLP), specifically Natural Language Generation (NLG). Existing literature tends to overemphasize elaborate retrieval strategies that involve multiple LLM calls, but fail to detail what algorithm and configurations, constituted the naïve approach that it was benchmarked against. We compare standard and elaborate retrieval methods and observe that both had similar performances, with standard methods being Generative AI (GenAI) independent. Despite LLMs displaying graduate level problem solving capabilities, there was much dissatisfaction regarding GenAI technologies, indicating a need for greater emphasis and focus on the errors, limitations and applicability of proposed solutions. We attempt to reverse this dissatisfaction by offering design guidance and clarity to anyone attempting to incorporate GenAI capabilities into their workflows by providing techniques that facilitate Exploratory Data Analysis (EDA) in a RAG context, which allowed us to notice that relevance does not necessarily correlate with higher similarity/lexical scores. We detail the internal workings of exhaustive and partial semantic similarity and lexical, rule-based, retrieval algorithms, and provide formal representation for deterministic evaluation metrics and error analysis techniques. Our experiments show that lexical retrieval covers more of the limitations of semantic similarity retrieval in open domain environments, indicating that lexical based algorithms are still relevant and the NFL (No Free Lunch) Theorem still applies in IR, as we notice that semantic similarity and lexical based approaches, excel in closed and open domain environments, respectively. It stands to reason that existing RAG techniques and methodologies will improve with better Retrieval insight, which is the focus of this thesis.

Grantor dc:publisher

Stellenbosch : Stellenbosch University

Year dc:date.issued

2026

Author dc:creator

• Singh, Nikhiel Rahul

Advisor dc:contributor.advisor

• Gwetu, Mandla

Language dc:language.iso

en

Repository record dc:identifier.uri

https://scholar.sun.ac.za/handle/10019.1/135844

OAI identifier oai:identifier

oai:scholar.sun.ac.za:10019.1/135844