Understanding and improving the fidelity of local developer testing for cloud-based software

Modern software projects have been progressing towards a cloud-based, serverless programming model, where software applications use cloud services as important components. Such cloud-based programming practice greatly simplifies software development by harvesting cloud benefits (e.g., high availability and elasticity). However, it imposes significant challenges for software testing and analysis, due to the opaqueness of cloud backends and the monetary cost of invoking cloud services for continuous integration and deployment. As a result, software developers commonly use cloud emulators for offline development and testing, before online testing and deployment. This thesis presents a systematic analysis of cloud emulators from the perspective of cloud-based software testing. Our goal is to (1) understand the discrepancies introduced by cloud emulation with regard to software quality assurance and deployment safety and (2) address inevitable gaps between emulated and real cloud services. The analysis results are concerning. Among 255 APIs of five cloud services from Azure and Amazon Web Services (AWS), we detected discrepant behavior between the emulated and real services in 94 (37%) of the APIs. These discrepancies lead to inconsistent testing results, threatening deployment safety, introducing false alarms, and creating debuggability issues. The root causes are diverse, including accidental implementation defects and essential emulation challenges. We discuss potential solutions and develop a practical mitigation technique to address discrepancies of cloud emulators for software testing.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Mazhar, Anna

Contributors dc:contributor

• Xu, Tianyin

• Cloud-based Software Testing
• Emulators
• Root Cause Analysis
• Azure And Aws Cloud Services
• Api Discrepancies
• Local Testing
• Deployment Safety
• Software Quality Assurance.

Statement dc:rights

• Copyright 2024 Anna Mazhar

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/124353

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F124353