Comparative Study of Synchronous Remote and Traditional In-Lab Usability Evaluation Methods

Traditional in lab usability evaluation has been used as the 'standard' evaluation method for evaluating and improving usability of software user interfaces (Andre, Williges, & Hartson, 2000). However, traditional in lab evaluation has its drawbacks such as availability of representative end users, high cost of testing and lack of true representation of a user's actual work environment. To counteract these issues various alternative and less expensive usability evaluation methods (UEMs) have been developed over the past decade. One such UEM is the Remote Usability Evaluation method. Remote evaluation is a relatively new area and lacks empirical data to support the approach. The need for empirical support was addressed in this study. The overall purpose of this study was to determine the differences in the effectiveness of the two evaluation types, the remote evaluation approach (SREM) and the traditional evaluation approach, in collecting usability data. This study also compared the effectiveness between the two methods based on user type, usability novice users and usability experienced users. Finally, the hypothesis that users, in general, will prefer the remote evaluation approach of reporting to the traditional in-lab evaluation approach was also tested. Results indicated that, in general, the synchronous remote approach is at least as effective as the traditional in lab usability evaluation approach in collecting usability data across all user types. However, when user type was taken into consideration, it was found that there was a significant difference in the high severity negative critical incident data collected between the two approaches for the novice user group. The traditional approach collected significantly more high severity negative critical incident data than the remote approach. Additionally, results indicate that users tend to be more willing to participate in the same approach as the one they participated previously. Recommendations for usability evaluators for conducting the SREM approach and areas for future research are identified in the study.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Industrial and Systems Engineering

Department dc:contributor.department

Industrial and Systems Engineering

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2004

Author dc:creator

• Selvaraj, Prakaash V.

Chair dc:contributor.committeechair

• Smith-Jackson, Tonya L.

Committee members dc:contributor.committeemember

• Kleiner, Brian M.
• Choong, Yee-yin

• remote usability evaluation
• critical incidents
• usability method
• synchronous remote usability testing

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Dc Identifier Other

etd-05192004-122952

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/9939