Identifying End-User Challenges and Mitigation Strategies in Software Ecosystems: A Large-Scale Empirical Study on User Feedback

Objectives: The aim of this thesis is to analyze and articulate end-user challenges and pain points in Software Ecosystems (SECOs) based on user feedback. The problems faced by end-users in SECOs and the methods to address them have not been studied in existing research. The objectives include understanding developer responses to feedback, examining the growth of SECO reviews over time, and providing strategies to mitigate user challenges in SECOs. Methods: Over 2.4 million user reviews were scraped from 283 ecosystem platforms in app stores and review websites. Among these, over 40,000 relevant reviews from 139 platforms were classified as "SECO reviews" through manual pair coding and automated techniques. Subsequently, a training dataset of 5,000 SECO reviews was labeled as problem categories identified. An XGBoost Machine Learning classifier was trained using this dataset to categorize SECO reviews into six distinct problem categories with an accuracy of 93%. Negative reviews were identified using sentiment analysis, and TF-IDF and Chi-Squared analysis were employed to extract features associated with each problem category. Finally, a thematic analysis was conducted on interviews with platform owners using a semi-structured interview approach to gather mitigation strategies. Results & Recommendations: Six categories of challenges in SECOs were identified: Integration, Customer Support, Design & Complexity, Privacy & Security, Cost & Pricing, and Performance & Compatibility. Each category presented its own set of pain points, encompassing a wide range of issues experienced by users. Examples include dysfunctional API errors, user preference for live customer support instead of chat-based interactions, user switching to competitors due to interface complexity, concerns about scams and fake reviews in marketplaces, unexpected charges to bank accounts, and device-specific and operating system-specific compatibility issues. The findings show that developers respond to SECO reviews less frequently compared to general reviews and SECO-related reviews have exhibited significant growth in the past five years, with the integration demands heightened by the COVID-19 pandemic. To effectively address the challenges faced by end-users and developers, platforms should adopt an API-first mentality, foster a supportive community, implement stringent vetting processes in marketplaces, employ live feedback tracking tools, consider developing cross-platform software, establish comprehensive documentation and guidelines, and maintain transparent policies in user data management.

Author dc:creator

• Ghimire, Bachan

Advisor dc:contributor.supervisor

• Damian, Daniela

• Software Ecosystem
• Data Mining
• User Feedback
• Machine Learning
• Ecosystem Strategy

Statement dc:rights

• Available to the World Wide Web

Language dc:language.iso

en, English

Handle dc:identifier.uri

http://hdl.handle.net/1828/15333

OAI identifier oai:identifier

oai:dspace.library.uvic.ca:1828/15333