{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/115961"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/115961","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Contrasting with adversarial examples improves self-supervised representation learning","abstract":"DSpace SAF Submission Ingestion Package generated from Vireo submission #18414 on 2022-11-16 at 10:56:42","abstract_html":"DSpace SAF Submission Ingestion Package generated from Vireo submission #18414 on 2022-11-16 at 10:56:42","abstract_has_math":false,"creators":["Yuan, Meilu"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Koyejo, Oluwasanmi"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-08","date_published":"2022-08","updated_at":"2026-07-22T22:24:55Z","subjects":["Self-Supervised Learning","Representation Learning"],"languages":["en","eng"],"rights":["Copyright 2022 Meilu Yuan"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/115961","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Koyejo, Oluwasanmi"]},{"key":"dc:creator","label":"Author","values":["Yuan, Meilu"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-08","2022-07-21"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Self-Supervised Learning","Representation Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Meilu Yuan"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/115961"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["DSpace SAF Submission Ingestion Package generated from Vireo submission #18414 on 2022-11-16 at 10:56:42","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","The student, Meilu Yuan, accepted the attached license on 2022-07-21 at 12:43.","The student, Meilu Yuan, submitted this Thesis for approval on 2022-07-21 at 12:45.","This Thesis was approved for publication on 2022-07-21 at 13:21.","Lately, self-supervised contrastive learning has enjoyed enormous attention because of its good performance. This set of methods target at learning a good representation for downstream tasks by contrasting objects in different views. Such that, the distance between similar data’s representations can be minimized while representations of dissimilar data will become apart. Existing works suggest combining different meaningful transformations randomly to form positive examples, like, cropping, rotation, color distortion, blurring etc. As a special view of the data, specifically designed adversarial examples can maximize the loss values. Thus, they misguide the final prediction towards the wrongest direction. In this work, we explore the impacts of combining adversarially distorted examples as positive examples in self-supervised representation learning. By having more challenging positive examples in the contrasting stage, in the forward pass, we generate adversarially distorted images, extract representations of each anchor-adversarial data pair and calculate contrastive loss in the latent space; in the backward pass, we force the representation vectors of these two positive examples fully correlated. As adversarial attacks improve the feature invariance of learned representation, experimental results show adversarial attacks can further improve downstream task performance, robustness as well as the generalization ability of the trained encoder."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Contrasting with adversarial examples improves self-supervised representation learning"]}]}],"canonical_facts":{"dc:contributor":["Koyejo, Oluwasanmi"],"dc:creator":["Yuan, Meilu"],"dc:date":["2022-08","2022-07-21"],"dc:description":["DSpace SAF Submission Ingestion Package generated from Vireo submission #18414 on 2022-11-16 at 10:56:42","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","The student, Meilu Yuan, accepted the attached license on 2022-07-21 at 12:43.","The student, Meilu Yuan, submitted this Thesis for approval on 2022-07-21 at 12:45.","This Thesis was approved for publication on 2022-07-21 at 13:21.","Lately, self-supervised contrastive learning has enjoyed enormous attention because of its good performance. This set of methods target at learning a good representation for downstream tasks by contrasting objects in different views. Such that, the distance between similar data’s representations can be minimized while representations of dissimilar data will become apart. Existing works suggest combining different meaningful transformations randomly to form positive examples, like, cropping, rotation, color distortion, blurring etc. As a special view of the data, specifically designed adversarial examples can maximize the loss values. Thus, they misguide the final prediction towards the wrongest direction. In this work, we explore the impacts of combining adversarially distorted examples as positive examples in self-supervised representation learning. By having more challenging positive examples in the contrasting stage, in the forward pass, we generate adversarially distorted images, extract representations of each anchor-adversarial data pair and calculate contrastive loss in the latent space; in the backward pass, we force the representation vectors of these two positive examples fully correlated. As adversarial attacks improve the feature invariance of learned representation, experimental results show adversarial attacks can further improve downstream task performance, robustness as well as the generalization ability of the trained encoder."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/115961"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Meilu Yuan"],"dc:subject":["Self-Supervised Learning","Representation Learning"],"dc:title":["Contrasting with adversarial examples improves self-supervised representation learning"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}