Large Language Models have emerged to great fanfare in the Information Technology market. Business and Information Technology leaders are currently exploring ways to apply these models to assist their organizations in executing business processes and generating innovation. Software vendors, consultants, and academics promote various approaches to making Large Language Models work effectively for business. However, little academic literature is available today that quantifies the degree of improvement possible with these domain-specific approaches over the standard capabilities of generalized Large Language Models.
The study seeks to quantify the benefits of one approach, Retrieval Augmented Generation. The study uses a collection of questions across several topics with known reference answers. The context for these questions is used to assemble a study corpus. Responses are generated by both a standard Large Language Model system and a Retrieval Augmented Generation system. The study analyzes the quality of generated responses to determine the degree to which the Retrieval Augmented Generation responses differ from those generated by the standard Large Language Model.
","abstract_html":"<p>Large Language Models have emerged to great fanfare in the Information Technology market. Business and Information Technology leaders are currently exploring ways to apply these models to assist their organizations in executing business processes and generating innovation. Software vendors, consultants, and academics promote various approaches to making Large Language Models work effectively for business. However, little academic literature is available today that quantifies the degree of improvement possible with these domain-specific approaches over the standard capabilities of generalized Large Language Models.</p> <p>The study seeks to quantify the benefits of one approach, Retrieval Augmented Generation. The study uses a collection of questions across several topics with known reference answers. The context for these questions is used to assemble a study corpus. Responses are generated by both a standard Large Language Model system and a Retrieval Augmented Generation system. The study analyzes the quality of generated responses to determine the degree to which the Retrieval Augmented Generation responses differ from those generated by the standard Large Language Model.</p>","abstract_has_math":false,"creators":["Crippen, Phillip D."],"institution":null,"degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":null,"degree_department":null,"school":null,"contributors":["Ryan Elmore","Kellie Keeling","Benjamin Williams"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-06-15T07:00:00Z","date_published":"2024-06-15T07:00:00Z","updated_at":"2026-07-24T02:01:48Z","subjects":["Corpus","Large language model (LLM)","Non-factoid question taxonomy","Retrieval augmented generation (RAG)","Vector database","Business","Software Engineering","Technology and Innovation"],"languages":["English (eng)"],"rights":["Copyright is held by the author. User is responsible for all copyright compliance.
"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.du.edu/etd/2418","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Ryan Elmore","Kellie Keeling","Benjamin Williams"]},{"key":"dc:creator","label":"Author","values":["Crippen, Phillip D."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2026-08-12T07:00:00Z"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Corpus","Large language model (LLM)","Non-factoid question taxonomy","Retrieval augmented generation (RAG)","Vector database","Business","Software Engineering","Technology and Innovation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["English (eng)"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright is held by the author. User is responsible for all copyright compliance.
"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.du.edu/etd/2418"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Large Language Models have emerged to great fanfare in the Information Technology market. Business and Information Technology leaders are currently exploring ways to apply these models to assist their organizations in executing business processes and generating innovation. Software vendors, consultants, and academics promote various approaches to making Large Language Models work effectively for business. However, little academic literature is available today that quantifies the degree of improvement possible with these domain-specific approaches over the standard capabilities of generalized Large Language Models.
The study seeks to quantify the benefits of one approach, Retrieval Augmented Generation. The study uses a collection of questions across several topics with known reference answers. The context for these questions is used to assemble a study corpus. Responses are generated by both a standard Large Language Model system and a Retrieval Augmented Generation system. The study analyzes the quality of generated responses to determine the degree to which the Retrieval Augmented Generation responses differ from those generated by the standard Large Language Model.
"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Evaluating the Effect of Domain-Specific Large Language Models on Question and Response"]}]}],"canonical_facts":{"dc:contributor":["Ryan Elmore","Kellie Keeling","Benjamin Williams"],"dc:creator":["Crippen, Phillip D."],"dc:date.available":["2026-08-12T07:00:00Z"],"dc:description.abstract":["Large Language Models have emerged to great fanfare in the Information Technology market. Business and Information Technology leaders are currently exploring ways to apply these models to assist their organizations in executing business processes and generating innovation. Software vendors, consultants, and academics promote various approaches to making Large Language Models work effectively for business. However, little academic literature is available today that quantifies the degree of improvement possible with these domain-specific approaches over the standard capabilities of generalized Large Language Models.
The study seeks to quantify the benefits of one approach, Retrieval Augmented Generation. The study uses a collection of questions across several topics with known reference answers. The context for these questions is used to assemble a study corpus. Responses are generated by both a standard Large Language Model system and a Retrieval Augmented Generation system. The study analyzes the quality of generated responses to determine the degree to which the Retrieval Augmented Generation responses differ from those generated by the standard Large Language Model.
"],"dc:format":["application/pdf"],"dc:identifier":["https://digitalcommons.du.edu/etd/2418"],"dc:language":["English (eng)"],"dc:rights":["Copyright is held by the author. User is responsible for all copyright compliance.
"],"dc:subject":["Corpus","Large language model (LLM)","Non-factoid question taxonomy","Retrieval augmented generation (RAG)","Vector database","Business","Software Engineering","Technology and Innovation"],"dc:title":["Evaluating the Effect of Domain-Specific Large Language Models on Question and Response"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."]},"updated_at":"2026-07-24T02:01:48Z"}