Automatic continuous testing to speed software development

Continuous testing is a new feature for software development environments that uses excess cycles on a developer's workstation to continuously run regression tests in the background, providing rapid feedback about test failures as source code is edited. It is intended to reduce the time and energy required to keep code well-tested, and to prevent regression errors from persisting uncaught for long periods of time. The longer that regression errors are allowed to linger during development, the more time is wasted debugging and fixing them once they are discovered. By monitoring and measuring software projects, we estimate that the wasted time, consisting of this preventable extra fixing cost added to the time spent running tests and waiting for them to complete, accounts for 10-15% of total development time. We present a model of developer behavior that uses data from past projects to infer developer beliefs and predict behavior in new environments -in particular, when changing testing methodologies or tools to reduce wasted time. This model predicts that continuous testing would reduce wasted time by 92-98%, a substantial improvement over other approaches we evaluated, such as automatic test prioritization and changing manual test frequencies. A controlled human experiment indicates that student developers using continuous testing were three times more likely to complete a task before the deadline than those without, with no significant effect on time worked.

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2004

Author dc:creator

• Saff, David, 1976-

Advisor dc:contributor.advisor

• Michael D. Ernst.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/30096

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/30096