Comparing text and visual annotation tools for design feedback

Designers rely on critiques to develop their skills and iterate toward effective designs. Designers are increasingly turning to online tools and communities to collect affordable, scalable feedback. There are many tools available but little empirical evidence to guide the decision to select one over another. We conducted a study on Amazon Mechanical Turk (N=360) to contrast two popular classes of feedback collection interfaces: text and spatial. The text interface has one text box for providers to submit their feedback, whereas the spatial interface allows the providers to annotate the design. We also manipulated the presence and content of history of feedback visible to providers. The three history conditions were aesthetic, goal-related, and ‘no-history’ if they did not have to option to view feedback. We found that the presence of sample feedback induces a fixation effect in design reviews, causing feedback to be more similar to the feedback that the providers reviewed. We found that both the Interface and History conditions have statistically significant effects on the content of feedback provided. However, neither resulted in greater perceived quality of feedback. Our study found that feedback in the text condition was 27% longer than in the spatial condition. We also found that providers who reviewed goal-oriented sample feedback rated the task as being easier than providers who did not receive sample feedback. These results indicate that the more important decision for designers is not which class of interface to use but whether to include history or sample feedback.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2017

Author dc:creator

• DeJong, Antoine Jean

Contributors dc:contributor

• Bailey, Brian P.

• Crowdsourcing
• Design
• Feedback
• Fixation

Statement dc:rights

• Copyright 2017 Antoine DeJong

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/98181

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/98181