Analyzing the portability of configuration testing for open-source Java projects

The advancement of Java-based open-source projects underscores the critical role of configuration testing in ensuring software reliability, particularly since misconfiguration remains a dominant factor contributing to production failures and widespread outages. Ctest, proposed by Sun et al., offers an essential approach to improving software reliability through its capability to expose dormant software bugs and diverse misconfigurations by leveraging existing test code under production configurations [1]. Despite these advantages, Ctest has not been adopted by any existing project, and systematic studies on applying Ctest to diverse Java-based open-source projects and evaluating its portability are still lacking. Without a comprehensive understanding of the porting process and portability, the application scope of Ctest remains limited, restricting its potential benefits in enhancing software robustness. Therefore, conducting systematic research on Ctest portability is both urgent and meaningful. This thesis contributes in three main aspects: First, from a technical perspective, we developed various tools to automate and facilitate the porting process, significantly minimizing the manual effort required to integrate Ctest into Java projects. Specifically, we automated critical porting steps such as dependency inclusion, package imports, and Ctest annotations, ensuring that portability assessment primarily reflects intrinsic project attributes rather than the complexity of porting itself. We also developed static analysis tools to extract project attributes relevant to portability systematically. Second, from an analytical perspective, we comprehensively analyzed the portability of Ctest by studying 18 prominent Java-based open-source projects. We identified key attributes significantly influencing Ctest portability and developed metrics to quantify them, such as the number of configuration interfaces, the types of configuration frameworks, and the structure of configuration systems. For projects unsuitable for porting, we identified underlying causes and proposed targeted solutions. Lastly, from an application perspective, we successfully expanded Ctest’s applicability by porting nine additional projects and validated the effectiveness of Ctest through comparative analysis. Results demonstrated consistent effectiveness with minimal to no runtime degradation, highlighting the practicality and reliability of Ctest in diverse Java environments. In summary, this thesis systematically investigates the technical, analytical, and practical aspects of configuration testing portability, achieving positive outcomes and laying a solid foundation for future improvements in both theoretical understanding and practical tooling.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Engineering

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Li, Qingyu

Contributors dc:contributor

• Xu, Tianyin

• Configuration
• Configuration Testing
• Portability
• Configuration Api
• Ctest
• Ctest4j.

Statement dc:rights

• Copyright 2025 Qingyu Li

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129313

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129313