A Performance Survey of Text-Based Sentiment Analysis Methods for Automating Usability Evaluations

<p>Usability testing, or user experience (UX) testing, is increasingly recognized as an important part of the user interface design process. However, evaluating usability tests can be expensive in terms of time and resources and can lack consistency between human evaluators. This makes automation an appealing expansion or alternative to conventional usability techniques.</p> <p>Early usability automation focused on evaluating human behavior through quantitative metrics but the explosion of opinion mining and sentiment analysis applications in recent decades has led to exciting new possibilities for usability evaluation methods.</p> <p>This paper presents a survey of modern, open-source sentiment analyzers’ usefulness in extracting and correctly identifying moments of semantic significance in the context of recorded mock usability evaluations. Though our results did not find a text-based sentiment analyzer that could correctly parse moments as well as human evaluators, one analyzer was found to be able to parse positive moments found through audio-only cues as well as human evaluators. Further research into adjusting settings on current sentiment analyzers for usability evaluations and using multimodal tools instead of text-based analyzers could produce valuable tools for usability evaluations when used in conjunction with human evaluators.</p>

Name thesis:degree_name

MS in Computer Science

Discipline thesis:degree_discipline

Computer Science

Year dc:date.available

2021

Author dc:creator

• Van Damme, Kelsi

Contributors dc:contributor

• Franz Kurfess
• Computer Science
• College of Engineering

• Usability
• User Experience
• Sentiment Analysis
• Agreement
• Automation
• Affectual
• Data Science
• Graphics and Human Computer Interfaces
• Software Engineering

Identifier

10.15368/theses.2021.107

OAI identifier oai:identifier

oai:digitalcommons.calpoly.edu:theses-3905