Designing with crowds

Feedback plays an essential role in the evolution of a design from initial concept to final representation. There are several approaches to receive feedback on designs such as organizing critiques, informally asking peers, and participating in online feedback communities. However, these social approaches highly depend on a user’s social capital and peer networks, and thus the access to desired feedback is limited. This dissertation explores the use of non-expert crowds to generate helpful, affordable, and timely feedback for visual design. The key innovation is the concept of generating structured feedback from non-experts. Based on a formative study, I identified several types of perception-oriented feedback desired from non-experts. To generate this feedback, I designed and implemented a new crowd-based system called Voyant. Voyant enables any user who creates visual designs to receive perception-oriented feedback on the designs from a selected audience. The generated feedback includes the elements seen in a design, the order in which elements are noticed, impressions formed when the design is first viewed, and interpretation of the design relative to guidelines in the domain and the user’s stated goals. I evaluated Voyant with users and their own designs. My evaluations reveal that crowd feedback helps users improve their designs in practice. The feedback helped users identify design problems, gain insight into their designs, and consider their designs from diverse perspectives. The monetary cost of generating the feedback was considered a fair tradeoff for not having to organize critiques or interrupt peers. The content analysis reveals that Voyant generated more interpretative, diverse, and critical feedback than free-form prompts. This dissertation provides the empirical evidence that crowd feedback benefits the end user and contributes new lessons for designing crowd feedback systems that support design activities. Crowd feedback systems provide users more opportunities to make design decisions based on audience input rather than solely on intuitions. These systems have the potential to tighten the design cycle and help users iterate toward solutions that better connect with the intended audience.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2014

Author dc:creator

• Xu, Anbang

Contributors dc:contributor

• Bailey, Brian P.
• Fu, Wai-Tat
• Karahalios, Karrie G.
• Dow, Steven P.

• Crowd feedback
• Crowdsourcing
• Design Iteration
• Visual Design
• Creativity
• Critique

Statement dc:rights

• Copyright 2014 Anbang Xu

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/50719

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/50719