{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129521"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129521","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Improving reasoning capabilities of large language models","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","abstract_has_math":false,"creators":["Dixit, Tanay"],"institution":"University of Illinois Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Han, Jiawei"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-04-11","date_published":"2025-04-11","updated_at":"2026-07-22T22:25:05Z","subjects":["Natural Language Processing","Large Language Models"],"languages":["en","eng"],"rights":["Copyright 2025 Tanay Dixit"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129521","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Han, Jiawei"]},{"key":"dc:creator","label":"Author","values":["Dixit, Tanay"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-04-11","2025-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Natural Language Processing","Large Language Models"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Tanay Dixit"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129521"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","The student, Tanay Dixit, accepted the attached license on 2025-04-10 at 15:51.","The student, Tanay Dixit, submitted this Thesis for approval on 2025-04-10 at 15:55.","This Thesis was approved for publication on 2025-04-11 at 15:07.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21739 on 2025-10-19 at 19:14:38","Large Language Models (LLMs) have succeeded at solving several wide range of tasks like mathematical problem solving, code generation, common-sense reasoning, etc. The recent success of these models is largely attributed to scaling in both model size and training data, which often includes massive amounts of web-scale and synthetic data — raising important questions about their true generalization capabilities. Several studies have highlighted critical failure cases in the reasoning abilities of LLMs, such as token biases in logical problem-solving and sensitivity to the order of premises, indicating a reliance on surface-level cues rather than true logical understanding. Additionally, these reasoning abilities ares hown to emerge only when models are trained on extremely large datasets. This technique of learning to reason deviates from how humans learn to reason and think. Humans learn to solve problems by first understanding and acquiring the fundamental principles involved in reasoning, and then learn to apply these principles to new tasks, rather than directly learning to solve hundreds of complex problems. Inspired by this, we aim to train LLMs to learn to reason with the help of axioms - fundamental principles of reasoning, in particular causal axioms. Causal axioms lay the crucks of causal inference which humans use in making decisions or inferences in several scenarios. The influence of causal axioms on the reasoning abilities of LLMs remains underexplored; in this work, we demonstrate that causal axiomatic training can enhance LLM performance across a broad range of reasoning tasks, even those not directly related to causality. Our extensive evaluation results across 16 benchmarks, shows that LLMs fine-tuned using our axiomatic data show stronger gains compared to baseline approaches on most tasks."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Improving reasoning capabilities of large language models"]}]}],"canonical_facts":{"dc:contributor":["Han, Jiawei"],"dc:creator":["Dixit, Tanay"],"dc:date":["2025-04-11","2025-05"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","The student, Tanay Dixit, accepted the attached license on 2025-04-10 at 15:51.","The student, Tanay Dixit, submitted this Thesis for approval on 2025-04-10 at 15:55.","This Thesis was approved for publication on 2025-04-11 at 15:07.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21739 on 2025-10-19 at 19:14:38","Large Language Models (LLMs) have succeeded at solving several wide range of tasks like mathematical problem solving, code generation, common-sense reasoning, etc. The recent success of these models is largely attributed to scaling in both model size and training data, which often includes massive amounts of web-scale and synthetic data — raising important questions about their true generalization capabilities. Several studies have highlighted critical failure cases in the reasoning abilities of LLMs, such as token biases in logical problem-solving and sensitivity to the order of premises, indicating a reliance on surface-level cues rather than true logical understanding. Additionally, these reasoning abilities ares hown to emerge only when models are trained on extremely large datasets. This technique of learning to reason deviates from how humans learn to reason and think. Humans learn to solve problems by first understanding and acquiring the fundamental principles involved in reasoning, and then learn to apply these principles to new tasks, rather than directly learning to solve hundreds of complex problems. Inspired by this, we aim to train LLMs to learn to reason with the help of axioms - fundamental principles of reasoning, in particular causal axioms. Causal axioms lay the crucks of causal inference which humans use in making decisions or inferences in several scenarios. The influence of causal axioms on the reasoning abilities of LLMs remains underexplored; in this work, we demonstrate that causal axiomatic training can enhance LLM performance across a broad range of reasoning tasks, even those not directly related to causality. Our extensive evaluation results across 16 benchmarks, shows that LLMs fine-tuned using our axiomatic data show stronger gains compared to baseline approaches on most tasks."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129521"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Tanay Dixit"],"dc:subject":["Natural Language Processing","Large Language Models"],"dc:title":["Improving reasoning capabilities of large language models"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:05Z"}