Test suite parallelization in open-source projects: a study on its usage and impact

Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests.

Grantor dc:publisher

Universidade Federal de Pernambuco

Year dc:date.issued

2018

Author dc:creator

• CÂNDIDO, Jeanderson Barros

Advisor dc:contributor.advisor

• D'AMORIM, Marcelo Bezerra

• Engenharia de software
• Teste de software
• Paralelismo de testes

Statement dc:rights

• openAccess
• Attribution-NonCommercial-NoDerivs 3.0 Brazil

Licence dc:rights.uri

• http://creativecommons.org/licenses/by-nc-nd/3.0/br/

Language dc:language.iso

eng

Repository record dc:identifier.uri

https://repositorio.ufpe.br/handle/123456789/30520

OAI identifier oai:identifier

oai:repositorio.ufpe.br:123456789/30520