Cost-saving in Continuous Integration: Development, Improvement, and Evaluation of Build Selection Approaches

Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice — Google and Mozilla estimate their CI systems in millions of dollars. In this dissertation, I propose a collection of novel build selection approaches that are able to save the cost of CI. I also propose the first exhaustive comparison of techniques to improve CI including build and test granularity approaches. I firstly design a build selection approach (SMARTBUILDSKIP) for CI cost reduction in a balanceable way. The evaluation of SMARTBUILDSKIP shows that it can save a median of 30% of builds by only incurring a median delay of 1 build in a median of 15% of failing builds under its most conservative configuration. To minimize the delayed failure observation, I then propose the second build selection approach (PRECISEBUILDSKIP) that can save cost without delaying failure observation. We find that PRECISEBUILDSKIP can save a median of 5.5% of builds while capturing the majority of failing builds (100% in median) from the evaluation. After that, I evaluate the strengths and weaknesses of 10 techniques that can improve CI including SMARTBUILDSKIP. The findings of the comparison motivate my next work to design a hybrid technique (HYBRIDBUILDSKIP) that combines these techniques to produce more cost-saving while keeping a low proportion of failing builds that are delayed in observation. Finally, I design an experiment to understand how different weights of test duration among the whole build duration can influence the cost-saving of build and test selection techniques.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Computer Science and Applications

Department dc:contributor.department

Computer Science

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2022

Author dc:creator

• Jin, Xianhao

Chair dc:contributor.committeechair

• Servant Cortes, Francisco Javier

Committee members dc:contributor.committeemember

• Tilevich, Eli
• Elbaum, Sebastian
• Meng, Na
• Prakash, Bodicherla Aditya

• Continuous Integration
• Build Strategies
• Maintenance Cost

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:34039

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/110323