How interface design affects the composition, interpretation, and utilization of feedback

Interfaces for composing and reviewing feedback vary widely in terms of the information they include and how they present it. These interfaces can have a strong impact on how content creators and their feedback providers engage with the feedback, ranging from task performance and perceptions of fairness and helpfulness to strategies used for composing and navigating the feedback. In my thesis, I report the results from three experiments exploring how user interfaces present throughout the feedback exchange process influence feedback composition, interpretation, and utilization. In the first experiment, we examine how various combinations of constructive and summative feedback influence a creator’s perceptions of the feedback and the revisions they subsequently perform. Participants (N=441) wrote and revised short stories when presented with quality scores, pre-authored constructive comments, both scores and comments, or no feedback. We found that while showing scores can have marginal benefits compared to showing no feedback, constructive comments without scores led to the most favorable feedback perceptions and revisions. In the second experiment, we investigate how feedback’s level of detail influences perceptions of the feedback by both the provider and recipient, and how these perceptions translate to revisions and creative outcomes. Writers (N=285) received feedback from expert providers (N=4) in the form of a writing rubric, open comments, a rubric with open comments, or a rubric with per-criterion comments. We found that writers’ revision quality and feedback perceptions increased with the feedback’s level of detail, but providers felt the interface that required the most detail diminished their ability to effectively articulate their thoughts. In the third experiment, we explore how students integrate an interactive visualization of feedback’s topic and opinion structure into their proceses for navigating and interpreting feedback. Teams (N=18) of 3-5 students each used the tool to review feedback on three different project deliverables and revise these deliverables for a grade in an authentic UI design course. We found that students used the visualization to assess their work quality, prioritize revisions, and justify design decisions to their teammates. Some students additionally developed emergent techniques for reviewing and annotating feedback that they adapted to their other projects outside the course. My dissertation represents a large step towards a future where the interfaces with which feedback is composed and presented are given as much consideration as the feedback’s content when designing tools for supporting feedback exchange between content creators and feedback providers.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Crain, Patrick

Contributors dc:contributor

• Bailey, Brian
• Huang, Yun
• Kirlik, Alex
• Kim, Joy

• human computer interaction
• interface design
• iterative design
• online feedback exchange
• mixed-methods

Statement dc:rights

• Copyright 2022 Patrick Crain

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/117710

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F117710