Large Language Models for Reasoning: From Indirect Supervision to Self-supervision

In recent years, large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, challenges remain in guiding LLMs toward reasoning tasks that require complex decision-making and nuanced judgment across diverse domains. This dissertation explores novel techniques to enhance the reasoning capabilities of LLMs by shifting from traditional indirect supervision methods to more autonomous self-supervision paradigms. The first part of this dissertation presents two published works that illustrate how task-specific training with indirect supervision improves LLMs’ reasoning ability on selection-based NLP tasks. These approaches demonstrate the power of leveraging indirect supervision to strengthen decision-making in multi-option tasks such as intent detection and question answering. The second part introduces a study on developing expertise-level benchmarks to assess and improve LLMs’ reasoning in scientific (meta-)review processes, paving the way for more accurate and robust evaluation of model performance in complex, expert-driven reasoning tasks. The third part presents a plug-and-play framework that adapts off-the-shelf pre-trained encoder–decoder models — leveraging their inherent pre-trained knowledge — to process long sequences and enhance long-context reasoning ability. Together, these contributions provide architectures, benchmarks, and methodologies that strengthen the reasoning capabilities of NLP systems, highlighting the transformative potential of generative AI in both scientific research and real-world applications. All experiments are conducted on public datasets (UFET, BANKING77, HWU64, CLINC150, MCTest, SummScreen, GovReport, BookSum) and do not need any Institutional Review Board (IRB).

Author dc:creator

• Jiangshu Du (23291464)

• Computer Science

Statement dc:rights

• In Copyright

DOI dc:identifier

https://doi.org/10.25417/uic.31451227.v1

OAI identifier oai:identifier

oai:figshare.com:article/31451227