{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/98181"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/98181","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Comparing text and visual annotation tools for design feedback","abstract":"Designers rely on critiques to develop their skills and iterate toward effective designs. Designers are increasingly turning to online tools and communities to collect affordable, scalable feedback. There are many tools available but little empirical evidence to guide the decision to select one over another. We conducted a study on Amazon Mechanical Turk (N=360) to contrast two popular classes of feedback collection interfaces: text and spatial. The text interface has one text box for providers to submit their feedback, whereas the spatial interface allows the providers to annotate the design. We also manipulated the presence and content of history of feedback visible to providers. The three history conditions were aesthetic, goal-related, and ‘no-history’ if they did not have to option to view feedback. We found that the presence of sample feedback induces a fixation effect in design reviews, causing feedback to be more similar to the feedback that the providers reviewed. We found that both the Interface and History conditions have statistically significant effects on the content of feedback provided. However, neither resulted in greater perceived quality of feedback. Our study found that feedback in the text condition was 27% longer than in the spatial condition. We also found that providers who reviewed goal-oriented sample feedback rated the task as being easier than providers who did not receive sample feedback. These results indicate that the more important decision for designers is not which class of interface to use but whether to include history or sample feedback.","abstract_html":"Designers rely on critiques to develop their skills and iterate toward effective designs. Designers are increasingly turning to online tools and communities to collect affordable, scalable feedback. There are many tools available but little empirical evidence to guide the decision to select one over another. We conducted a study on Amazon Mechanical Turk (N=360) to contrast two popular classes of feedback collection interfaces: text and spatial. The text interface has one text box for providers to submit their feedback, whereas the spatial interface allows the providers to annotate the design. We also manipulated the presence and content of history of feedback visible to providers. The three history conditions were aesthetic, goal-related, and ‘no-history’ if they did not have to option to view feedback. We found that the presence of sample feedback induces a fixation effect in design reviews, causing feedback to be more similar to the feedback that the providers reviewed. We found that both the Interface and History conditions have statistically significant effects on the content of feedback provided. However, neither resulted in greater perceived quality of feedback. Our study found that feedback in the text condition was 27% longer than in the spatial condition. We also found that providers who reviewed goal-oriented sample feedback rated the task as being easier than providers who did not receive sample feedback. These results indicate that the more important decision for designers is not which class of interface to use but whether to include history or sample feedback.","abstract_has_math":false,"creators":["DeJong, Antoine Jean"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Bailey, Brian P."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2017,"date_issued":"2017-09-29T17:45:33Z","date_published":"2017-09-29T17:45:33Z","updated_at":"2026-07-22T22:24:35Z","subjects":["Crowdsourcing","Design","Feedback","Fixation"],"languages":["en"],"rights":["Copyright 2017 Antoine DeJong"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/98181","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Bailey, Brian P."]},{"key":"dc:creator","label":"Author","values":["DeJong, Antoine Jean"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2017-09-29T17:45:33Z","2020-03-03T10:15:11Z","2017-06-26","2017-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Crowdsourcing","Design","Feedback","Fixation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2017 Antoine DeJong"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/98181"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Designers rely on critiques to develop their skills and iterate toward effective designs. Designers are increasingly turning to online tools and communities to collect affordable, scalable feedback. There are many tools available but little empirical evidence to guide the decision to select one over another. We conducted a study on Amazon Mechanical Turk (N=360) to contrast two popular classes of feedback collection interfaces: text and spatial. The text interface has one text box for providers to submit their feedback, whereas the spatial interface allows the providers to annotate the design. We also manipulated the presence and content of history of feedback visible to providers. The three history conditions were aesthetic, goal-related, and ‘no-history’ if they did not have to option to view feedback. We found that the presence of sample feedback induces a fixation effect in design reviews, causing feedback to be more similar to the feedback that the providers reviewed. We found that both the Interface and History conditions have statistically significant effects on the content of feedback provided. However, neither resulted in greater perceived quality of feedback. Our study found that feedback in the text condition was 27% longer than in the spatial condition. We also found that providers who reviewed goal-oriented sample feedback rated the task as being easier than providers who did not receive sample feedback. These results indicate that the more important decision for designers is not which class of interface to use but whether to include history or sample feedback.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2019-08-01","The student, Antoine DeJong, accepted the attached license on 2017-06-22 at 21:00.","The student, Antoine DeJong, submitted this Thesis for approval on 2017-06-22 at 21:12.","This Thesis was approved for publication on 2017-06-26 at 14:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #11256 on 2017-09-29 at 10:46:04","Made available in DSpace on 2017-09-29T17:45:33Z (GMT). No. of bitstreams: 2 DEJONG-THESIS-2017.pdf: 1834128 bytes, checksum: 240dd48a04c146a39e5ec0a6fabef0de (MD5) LICENSE.txt: 4211 bytes, checksum: 3d2ca8f0664971a7ab06cd80e674aa3c (MD5) Previous issue date: 2017-06-26","Embargo set by: Colleen Fallaw for item 103478 Lift date: 2019-09-29T17:48:06Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103478 Lift date: 2020-03-02T19:56:41Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103478 Lift date: 2020-03-02T19:59:52Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103478 Lift date: 2020-03-02T20:02:46Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 103478 on 2020-03-03T10:15:11Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Comparing text and visual annotation tools for design feedback"]}]}],"canonical_facts":{"dc:contributor":["Bailey, Brian P."],"dc:creator":["DeJong, Antoine Jean"],"dc:date":["2017-09-29T17:45:33Z","2020-03-03T10:15:11Z","2017-06-26","2017-08"],"dc:description":["Designers rely on critiques to develop their skills and iterate toward effective designs. Designers are increasingly turning to online tools and communities to collect affordable, scalable feedback. There are many tools available but little empirical evidence to guide the decision to select one over another. We conducted a study on Amazon Mechanical Turk (N=360) to contrast two popular classes of feedback collection interfaces: text and spatial. The text interface has one text box for providers to submit their feedback, whereas the spatial interface allows the providers to annotate the design. We also manipulated the presence and content of history of feedback visible to providers. The three history conditions were aesthetic, goal-related, and ‘no-history’ if they did not have to option to view feedback. We found that the presence of sample feedback induces a fixation effect in design reviews, causing feedback to be more similar to the feedback that the providers reviewed. We found that both the Interface and History conditions have statistically significant effects on the content of feedback provided. However, neither resulted in greater perceived quality of feedback. Our study found that feedback in the text condition was 27% longer than in the spatial condition. We also found that providers who reviewed goal-oriented sample feedback rated the task as being easier than providers who did not receive sample feedback. These results indicate that the more important decision for designers is not which class of interface to use but whether to include history or sample feedback.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2019-08-01","The student, Antoine DeJong, accepted the attached license on 2017-06-22 at 21:00.","The student, Antoine DeJong, submitted this Thesis for approval on 2017-06-22 at 21:12.","This Thesis was approved for publication on 2017-06-26 at 14:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #11256 on 2017-09-29 at 10:46:04","Made available in DSpace on 2017-09-29T17:45:33Z (GMT). No. of bitstreams: 2 DEJONG-THESIS-2017.pdf: 1834128 bytes, checksum: 240dd48a04c146a39e5ec0a6fabef0de (MD5) LICENSE.txt: 4211 bytes, checksum: 3d2ca8f0664971a7ab06cd80e674aa3c (MD5) Previous issue date: 2017-06-26","Embargo set by: Colleen Fallaw for item 103478 Lift date: 2019-09-29T17:48:06Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103478 Lift date: 2020-03-02T19:56:41Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103478 Lift date: 2020-03-02T19:59:52Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103478 Lift date: 2020-03-02T20:02:46Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 103478 on 2020-03-03T10:15:11Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/98181"],"dc:language":["en"],"dc:rights":["Copyright 2017 Antoine DeJong"],"dc:subject":["Crowdsourcing","Design","Feedback","Fixation"],"dc:title":["Comparing text and visual annotation tools for design feedback"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:35Z"}