Evaluate and Benchmark Aris

In this paper, we present evaluation and benchmark of Aris (Analogical Reasoning for reuse of Implementation & Specification). Aris aims to increase the number of verified programs by promotes the advantages of code reuse and the possibility of transferring specifications between similar implementations. Source code retrieval in Aris acts as an enabling technology for the reuse of formal specifications. Although the result of the early version of Aris is encouraging and show potential reuse of formal specifications, it still has many rooms for improvements and wide possibility for feature enhancement. By using experimental methodology, we identify the issues and limitation of Aris 1.0. We develop Aris 2.0 that improve the construction of conceptual graph, reduce the occurrences of false variable and loop mapping, enable adjustment for transferring specifications between different iteration process of for loop, and support assert and assume specifications transfer. We also introduce new metric (specification score) to ensure that the top ranked retrieved documents possess good quality specifications for transferring specifications. Finally, we compare the performance of Aris 1.0 and Aris 2.0 as retrieval and mapping system and also its ability to create the verified specification. In order to evaluate and benchmark Aris, we use 2 million methods of unverified implementations real world example amongst verified implementation. Our overall result shows the improvement in Aris 2.0 able to produce more successful mapping between similar source code files, increase ranking precision in retrieval phase, and generate more verified specifications.

Level dc:type.qualificationlevel

masters

Grantor dc:publisher.institution

National University of Ireland Maynooth

Year dc:date.issued

2014

Author dc:creator

• Halim, Felicia

• Computer Science

Language dc:language

en