Comparative evaluation of conventional search architectures and large language model-based approaches

Search technologies are fundamental to how individuals and organisations access, interpret, and synthesise information. Conventional search engines such as Google and Bing rely on keyword-based indexing and symbolic ranking algorithms that provide transparency, verifiability, and computational efficiency. However, these systems often struggle to capture semantics, intent, and contextual nuance in more complex queries. In contrast, large language model (LLM)-based systems, including Gemini and chat generative pre-trained transformer (ChatGPT) with browsing, employ neural embeddings and generative reasoning to deliver contextualised and conversational outputs. While these advances enhance informational synthesis, they also introduce challenges such as hallucinations, opaque reasoning processes, and reduced efficiency. Hybrid models, exemplified by Google artificial intelligence overviews, attempt to bridge these paradigms by combining symbolic precision with generative flexibility. The purpose of this thesis is to conduct a quantitative comparative evaluation of conventional, hybrid, and LLM-based search systems. A dataset of user-centred queries was used to assess retrieval effectiveness, latency, and stability across factual, ambiguous, and informational categories. The findings highlight distinct trade-offs across systems. LLM-based search demonstrated the strongest performance in contextual reasoning and informational synthesis, but was limited by higher latency and variability. Conventional systems, while less accurate in complex queries, proved to be efficient and stable. Hybrid approaches offered the most balanced performance, integrating the efficiency of conventional search with the contextual depth of LLMs. This study concludes that hybrid systems, particularly Google AI Overviews, represent a promising pathway for advancing search technologies. By combining the reliability of conventional methods with the contextual intelligence of LLMs, they provide a more comprehensive foundation for addressing the diverse spectrum of modern search needs.

Grantor dc:publisher

Stellenbosch : Stellenbosch University

Year dc:date.issued

2026

Author dc:creator

• Molefe, Kgothatso Ishmael

Advisor dc:contributor.advisor

• Engelbrecht, Andries P.

Language dc:language.iso

en

Repository record dc:identifier.uri

https://scholar.sun.ac.za/handle/10019.1/136172

OAI identifier oai:identifier

oai:scholar.sun.ac.za:10019.1/136172