Harnessing large language models for software engineering

Automatic code generation, often referred to as program synthesis, has seen remarkable progress with the emergence of large language models (LLMs) trained on extensive datasets of text and programming-related data. This thesis examines the role that LLMs play in contemporary software engineering, with a particular focus on solving programming exercises, infilling code within existing projects, and automated program repair (APR). We begin by showing how effectively LLMs can address self-contained programming challenges. Using methods like step-by-step code synthesis and feedback-driven refinement, we show that the quality of solutions improves significantly when iterative processes are applied. Additionally, we present a novel framework, PairCoder, which leverages a collaborative approach to code generation. In this setup, two LLMs work together, adopting distinct roles: one acts as the driver to generate the core solutions, while the other serves as the navigator, providing guidance, critique, and suggestions. This iterative interplay between the two models results in more accurate and robust code by bringing multiple perspectives to the problem-solving process. When it comes to code infilling, we critically analyze existing methods for generating synthetic data and highlight their limitations when applied to more complex, real-world scenarios. To address these shortcomings, we propose adaptations to these techniques that allow LLMs to better understand and operate within intricate codebases. Lastly, we evaluate the capabilities of LLMs in the field of automated program repair. This includes a comparative analysis of models specialized for code infilling and those designed for sequence-to-sequence generation. We also perform data contamination analyses on popular benchmark datasets to understand the impact of training and testing data overlap on model performance. Through these investigations, this thesis highlights the promising potential of LLMs in addressing a wide range of software engineering challenges and offers insights into how these models can be further refined and optimized for real-world applications.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Wei, Yuxiang

Contributors dc:contributor

• Zhang, Lingming

• Large Language Models
• Software Engineering
• Program Synthesis
• Automated Program Repair

Statement dc:rights

• Copyright 2024 Yuxiang Wei

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/127392

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F127392