{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/132794"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/132794","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Improving problem-solving capabilities of language model: data, architecture and algorithms","abstract":"Artificial intelligence (AI), particularly large language models (LLMs), has exhibited formidable problem-solving abilities across a myriad of domains. These range from constrained arenas, such as sentiment analysis, to expansive fields including coding and mathematical reasoning. Furthermore, LLMs display potential in complex scientific disciplines, encompassing Medicine, Biology, and Physics. The pressing demand for AI to expedite advancements across these domains necessitates a concentrated effort to enhance the problem-solving capabilities of LLMs. In this thesis, I elucidate the current challenges associated with improving these capabilities in language models, focusing on three critical areas: data, architecture, and algorithms. Subsequently, I present four significant contributions that address these challenges from distinct perspectives. First, I introduce a novel data augmentation methodology that performs mix-up operations within the language embedding layer of varying inputs, followed by the application of projection techniques to generate new textual inputs. This augmentation significantly elevates the performance of knowledge distillation from teacher models, consequently enhancing the capabilities of student models in addressing closed-domain challenges, specifically exemplified by the General Language Understanding Evaluation (GLUE) benchmark. Next, I shift my focus to the Transformer architecture and identify the factors contributing to position bias, a detrimental effect that impedes the reasoning capabilities of models. By eliminating position bias through the implementation of bidirectional attention mechanisms and position re-assignment strategies, I demonstrate that models achieve superior performance on downstream tasks, including applications where LLMs operate as evaluators. In terms of algorithmic advancements, I developed a self-improvement reinforcement learning algorithm designed to incentivize models to produce enhanced responses by iteratively learning from their prior outputs. Central to this algorithm is the modeling of the reward gap between different responses, which facilitates the generation of superior responses in comparison to previous iterations. Finally, I propose an enhanced inference-time algorithm, which incorporates test-time training processes, aimed at bolstering robustness to varying hyperparameter selections. This work spans a comprehensive range of considerations, including optimizer choices, regularization techniques, and the tuning of parameter selections. In conclusion, this thesis posits that the future trajectory of LLM development hinges on advancing reasoning capabilities through reinforcement learning, with a pronounced emphasis on self-correction and self-improvement mechanisms.","abstract_html":"Artificial intelligence (AI), particularly large language models (LLMs), has exhibited formidable problem-solving abilities across a myriad of domains. These range from constrained arenas, such as sentiment analysis, to expansive fields including coding and mathematical reasoning. Furthermore, LLMs display potential in complex scientific disciplines, encompassing Medicine, Biology, and Physics. The pressing demand for AI to expedite advancements across these domains necessitates a concentrated effort to enhance the problem-solving capabilities of LLMs. In this thesis, I elucidate the current challenges associated with improving these capabilities in language models, focusing on three critical areas: data, architecture, and algorithms. Subsequently, I present four significant contributions that address these challenges from distinct perspectives. First, I introduce a novel data augmentation methodology that performs mix-up operations within the language embedding layer of varying inputs, followed by the application of projection techniques to generate new textual inputs. This augmentation significantly elevates the performance of knowledge distillation from teacher models, consequently enhancing the capabilities of student models in addressing closed-domain challenges, specifically exemplified by the General Language Understanding Evaluation (GLUE) benchmark. Next, I shift my focus to the Transformer architecture and identify the factors contributing to position bias, a detrimental effect that impedes the reasoning capabilities of models. By eliminating position bias through the implementation of bidirectional attention mechanisms and position re-assignment strategies, I demonstrate that models achieve superior performance on downstream tasks, including applications where LLMs operate as evaluators. In terms of algorithmic advancements, I developed a self-improvement reinforcement learning algorithm designed to incentivize models to produce enhanced responses by iteratively learning from their prior outputs. Central to this algorithm is the modeling of the reward gap between different responses, which facilitates the generation of superior responses in comparison to previous iterations. Finally, I propose an enhanced inference-time algorithm, which incorporates test-time training processes, aimed at bolstering robustness to varying hyperparameter selections. This work spans a comprehensive range of considerations, including optimizer choices, regularization techniques, and the tuning of parameter selections. In conclusion, this thesis posits that the future trajectory of LLM development hinges on advancing reasoning capabilities through reinforcement learning, with a pronounced emphasis on self-correction and self-improvement mechanisms.","abstract_has_math":false,"creators":["Wang, Ziqi"],"institution":"University of Illinois Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Ji, Heng","Zhang, Tong","Peng, Hao","Hou, Le"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-12","date_published":"2025-12","updated_at":"2026-07-22T22:25:07Z","subjects":["Artificial Intelligence","Language Model","Knowledge Distillation","Reinforcement Learning","Transformers","Test-Time Training"],"languages":["en"],"rights":["Copyright 2025 Ziqi Wang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/132794","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Ji, Heng","Zhang, Tong","Peng, Hao","Hou, Le"]},{"key":"dc:creator","label":"Author","values":["Wang, Ziqi"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-12","2025-12-04"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Artificial Intelligence","Language Model","Knowledge Distillation","Reinforcement Learning","Transformers","Test-Time Training"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Ziqi Wang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/132794"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Artificial intelligence (AI), particularly large language models (LLMs), has exhibited formidable problem-solving abilities across a myriad of domains. These range from constrained arenas, such as sentiment analysis, to expansive fields including coding and mathematical reasoning. Furthermore, LLMs display potential in complex scientific disciplines, encompassing Medicine, Biology, and Physics. The pressing demand for AI to expedite advancements across these domains necessitates a concentrated effort to enhance the problem-solving capabilities of LLMs. In this thesis, I elucidate the current challenges associated with improving these capabilities in language models, focusing on three critical areas: data, architecture, and algorithms. Subsequently, I present four significant contributions that address these challenges from distinct perspectives. First, I introduce a novel data augmentation methodology that performs mix-up operations within the language embedding layer of varying inputs, followed by the application of projection techniques to generate new textual inputs. This augmentation significantly elevates the performance of knowledge distillation from teacher models, consequently enhancing the capabilities of student models in addressing closed-domain challenges, specifically exemplified by the General Language Understanding Evaluation (GLUE) benchmark. Next, I shift my focus to the Transformer architecture and identify the factors contributing to position bias, a detrimental effect that impedes the reasoning capabilities of models. By eliminating position bias through the implementation of bidirectional attention mechanisms and position re-assignment strategies, I demonstrate that models achieve superior performance on downstream tasks, including applications where LLMs operate as evaluators. In terms of algorithmic advancements, I developed a self-improvement reinforcement learning algorithm designed to incentivize models to produce enhanced responses by iteratively learning from their prior outputs. Central to this algorithm is the modeling of the reward gap between different responses, which facilitates the generation of superior responses in comparison to previous iterations. Finally, I propose an enhanced inference-time algorithm, which incorporates test-time training processes, aimed at bolstering robustness to varying hyperparameter selections. This work spans a comprehensive range of considerations, including optimizer choices, regularization techniques, and the tuning of parameter selections. In conclusion, this thesis posits that the future trajectory of LLM development hinges on advancing reasoning capabilities through reinforcement learning, with a pronounced emphasis on self-correction and self-improvement mechanisms.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-12-01","The student, Ziqi Wang, accepted the attached license on 2025-12-03 at 03:22.","The student, Ziqi Wang, submitted this Dissertation for approval on 2025-12-03 at 03:32.","This Dissertation was approved for publication on 2025-12-04 at 07:22.","DSpace SAF Submission Ingestion Package generated from Vireo submission #23036 on 2026-02-19 at 20:09:55"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Improving problem-solving capabilities of language model: data, architecture and algorithms"]}]}],"canonical_facts":{"dc:contributor":["Ji, Heng","Zhang, Tong","Peng, Hao","Hou, Le"],"dc:creator":["Wang, Ziqi"],"dc:date":["2025-12","2025-12-04"],"dc:description":["Artificial intelligence (AI), particularly large language models (LLMs), has exhibited formidable problem-solving abilities across a myriad of domains. These range from constrained arenas, such as sentiment analysis, to expansive fields including coding and mathematical reasoning. Furthermore, LLMs display potential in complex scientific disciplines, encompassing Medicine, Biology, and Physics. The pressing demand for AI to expedite advancements across these domains necessitates a concentrated effort to enhance the problem-solving capabilities of LLMs. In this thesis, I elucidate the current challenges associated with improving these capabilities in language models, focusing on three critical areas: data, architecture, and algorithms. Subsequently, I present four significant contributions that address these challenges from distinct perspectives. First, I introduce a novel data augmentation methodology that performs mix-up operations within the language embedding layer of varying inputs, followed by the application of projection techniques to generate new textual inputs. This augmentation significantly elevates the performance of knowledge distillation from teacher models, consequently enhancing the capabilities of student models in addressing closed-domain challenges, specifically exemplified by the General Language Understanding Evaluation (GLUE) benchmark. Next, I shift my focus to the Transformer architecture and identify the factors contributing to position bias, a detrimental effect that impedes the reasoning capabilities of models. By eliminating position bias through the implementation of bidirectional attention mechanisms and position re-assignment strategies, I demonstrate that models achieve superior performance on downstream tasks, including applications where LLMs operate as evaluators. In terms of algorithmic advancements, I developed a self-improvement reinforcement learning algorithm designed to incentivize models to produce enhanced responses by iteratively learning from their prior outputs. Central to this algorithm is the modeling of the reward gap between different responses, which facilitates the generation of superior responses in comparison to previous iterations. Finally, I propose an enhanced inference-time algorithm, which incorporates test-time training processes, aimed at bolstering robustness to varying hyperparameter selections. This work spans a comprehensive range of considerations, including optimizer choices, regularization techniques, and the tuning of parameter selections. In conclusion, this thesis posits that the future trajectory of LLM development hinges on advancing reasoning capabilities through reinforcement learning, with a pronounced emphasis on self-correction and self-improvement mechanisms.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-12-01","The student, Ziqi Wang, accepted the attached license on 2025-12-03 at 03:22.","The student, Ziqi Wang, submitted this Dissertation for approval on 2025-12-03 at 03:32.","This Dissertation was approved for publication on 2025-12-04 at 07:22.","DSpace SAF Submission Ingestion Package generated from Vireo submission #23036 on 2026-02-19 at 20:09:55"],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/132794"],"dc:language":["en"],"dc:rights":["Copyright 2025 Ziqi Wang"],"dc:subject":["Artificial Intelligence","Language Model","Knowledge Distillation","Reinforcement Learning","Transformers","Test-Time Training"],"dc:title":["Improving problem-solving capabilities of language model: data, architecture and algorithms"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:07Z"}