{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129555"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129555","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Inference-driven perceptual optimization","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","abstract_has_math":false,"creators":["Xu, Xin"],"institution":"University of Illinois Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Wang, Yuxiong"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-04-22","date_published":"2025-04-22","updated_at":"2026-07-22T22:25:05Z","subjects":["Perception","Inference","Diffusion Models","Sparsity"],"languages":["en","eng"],"rights":["Copyright 2025 Xin Xu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129555","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Wang, Yuxiong"]},{"key":"dc:creator","label":"Author","values":["Xu, Xin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-04-22","2025-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Perception","Inference","Diffusion Models","Sparsity"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Xin Xu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129555"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","The student, Xin Xu, accepted the attached license on 2025-04-21 at 23:00.","The student, Xin Xu, submitted this Thesis for approval on 2025-04-21 at 23:27.","This Thesis was approved for publication on 2025-04-22 at 09:08.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21876 on 2025-10-19 at 19:15:16","Perception is fundamental to modern computer vision. While most research has focused on improving datasets, model architectures, or training objectives, this thesis explores a complementary yet under-explored perspective: optimizing perception models through inference guidance. The core insight is that models often exhibit informative behaviors during inference, such as deviations from training assumptions or signals that reveal opportunities for further optimization. Although typically overlooked, these behaviors can be leveraged to improve model effectiveness and efficiency. To demonstrate the idea, this thesis presents two case studies, each targeting one of these dimensions. The first part investigates Inference Guidance for Diffusion Models, aiming to improve model effectiveness by aligning training objectives with inference behavior. Diffusion models, increasingly adopted for perception tasks, generate samples through an iterative denoising process. However, this process often misaligns with the objectives of discriminative tasks. Inference-time analysis reveals two key issues: (1) denoising timesteps contribute unevenly to perception quality, and (2) a distribution shift between training and inference leads to performance degradation. To address these challenges, we propose ADDP (Aligning Diffusion Denoising with Perception), a framework that reweights the training objective based on timestep importance and introduces data augmentations to simulate inference-time denoising errors. Experiments on depth estimation and referring image segmentation tasks demonstrate that ADDP improves both perceptual alignment and overall performance. The second part focuses on Inference Acceleration for Visual Encoders, aiming to enhance model efficiency by leveraging inference properties. Analysis of pre-trained vision transformers (ViTs) shows that sparsity naturally emerges in the attention patterns of deeper layers, enabling selective computation without sacrificing performance. Building on this observation, we introduce SVE (Sparse Vision Encoder), a framework that identifies target layers, restores performance through distillation, and reduces inference latency via sparsity prediction. Experiments across multiple vision encoders show that SVE achieves up to a 23% speedup while maintaining accuracy on classification and segmentation benchmarks. Together, these studies advocate for inference-aware optimization as a promising paradigm for advancing perception models. They demonstrate that careful analysis of inference-time behavior can uncover new opportunities for performance gains beyond conventional training-time interventions."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Inference-driven perceptual optimization"]}]}],"canonical_facts":{"dc:contributor":["Wang, Yuxiong"],"dc:creator":["Xu, Xin"],"dc:date":["2025-04-22","2025-05"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","The student, Xin Xu, accepted the attached license on 2025-04-21 at 23:00.","The student, Xin Xu, submitted this Thesis for approval on 2025-04-21 at 23:27.","This Thesis was approved for publication on 2025-04-22 at 09:08.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21876 on 2025-10-19 at 19:15:16","Perception is fundamental to modern computer vision. While most research has focused on improving datasets, model architectures, or training objectives, this thesis explores a complementary yet under-explored perspective: optimizing perception models through inference guidance. The core insight is that models often exhibit informative behaviors during inference, such as deviations from training assumptions or signals that reveal opportunities for further optimization. Although typically overlooked, these behaviors can be leveraged to improve model effectiveness and efficiency. To demonstrate the idea, this thesis presents two case studies, each targeting one of these dimensions. The first part investigates Inference Guidance for Diffusion Models, aiming to improve model effectiveness by aligning training objectives with inference behavior. Diffusion models, increasingly adopted for perception tasks, generate samples through an iterative denoising process. However, this process often misaligns with the objectives of discriminative tasks. Inference-time analysis reveals two key issues: (1) denoising timesteps contribute unevenly to perception quality, and (2) a distribution shift between training and inference leads to performance degradation. To address these challenges, we propose ADDP (Aligning Diffusion Denoising with Perception), a framework that reweights the training objective based on timestep importance and introduces data augmentations to simulate inference-time denoising errors. Experiments on depth estimation and referring image segmentation tasks demonstrate that ADDP improves both perceptual alignment and overall performance. The second part focuses on Inference Acceleration for Visual Encoders, aiming to enhance model efficiency by leveraging inference properties. Analysis of pre-trained vision transformers (ViTs) shows that sparsity naturally emerges in the attention patterns of deeper layers, enabling selective computation without sacrificing performance. Building on this observation, we introduce SVE (Sparse Vision Encoder), a framework that identifies target layers, restores performance through distillation, and reduces inference latency via sparsity prediction. Experiments across multiple vision encoders show that SVE achieves up to a 23% speedup while maintaining accuracy on classification and segmentation benchmarks. Together, these studies advocate for inference-aware optimization as a promising paradigm for advancing perception models. They demonstrate that careful analysis of inference-time behavior can uncover new opportunities for performance gains beyond conventional training-time interventions."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129555"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Xin Xu"],"dc:subject":["Perception","Inference","Diffusion Models","Sparsity"],"dc:title":["Inference-driven perceptual optimization"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:05Z"}