Test2Vec: An Execution Trace Embedding for Behaviour Coverage Analysis of Test Cases

In software testing, test adequacy metrics are mostly based on the source code and ignore the dynamic behavior of the program. In this thesis, we demonstrate that code coverage metrics are not able to distinguish between a Failed and Passed behavior in most cases. Neural program embeddings have shown much promise recently for a variety of program analysis tasks. We propose a new embedding approach called Test2Vec with an attention layer for test representation in order to capture behavioral patterns of test cases. We used test execution traces including the input values as the training dataset and evaluated our approach on 93 real faulty versions of 4 open source Java projects. Our results show that Test2Vec can rank the failing test in the top 5, 10, and 20 tests, on average, 20.43%, 32.26%, and 44.09% of the times, while branch coverage and statement coverage can only distinguish the failing tests in 13% and 16% of cases, respectively.

Name thesis:degree_name

Master of Science (MSc)

Discipline thesis:degree_discipline

Engineering – Electrical & Computer

Grantor dc:publisher.institution

Schulich School of Engineering

Year dc:date.issued

2021

Author dc:creator

• Zangeneh, Soheila

Advisor dc:contributor.advisor

• Hemmati, Hadi

Committee members dc:contributor.committeemember

• Far, Behrouz Homayoun
• Uddin, Gias

• Software Engineering
• Software Testing
• Machine Learning
• Behaviour Analysis
• Test Adequacy
• Software Analysis

Statement dc:rights

• University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission.

Language dc:language.iso

eng

DOI

http://dx.doi.org/10.11575/PRISM/38591

OAI identifier oai:identifier

oai:ucalgary.scholaris.ca:1880/113023