{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/115689"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/115689","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Adaptive model inference under computational resource constraints","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","abstract_has_math":false,"creators":["Yu, Haichao"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Shi, Humphrey","Liang, Zhi-Pei","Hasegawa-Johnson, Mark","Hua, Gang","Huang, Gao"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-05","date_published":"2022-05","updated_at":"2026-07-22T22:24:55Z","subjects":["Adaptive Inference","Dynamic Neural Networks","Efficient Inference","Neural Network Quantization","Gradient Boosting"],"languages":["en","eng"],"rights":["Copyright 2022 Haichao Yu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/115689","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Shi, Humphrey","Liang, Zhi-Pei","Hasegawa-Johnson, Mark","Hua, Gang","Huang, Gao"]},{"key":"dc:creator","label":"Author","values":["Yu, Haichao"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-05","2022-04-11"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Adaptive Inference","Dynamic Neural Networks","Efficient Inference","Neural Network Quantization","Gradient Boosting"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Haichao Yu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/115689"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","The student, Haichao Yu, accepted the attached license on 2022-04-11 at 10:39.","The student, Haichao Yu, submitted this Dissertation for approval on 2022-04-11 at 10:57.","This Dissertation was approved for publication on 2022-04-11 at 12:31.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17626 on 2022-11-11 at 12:19:02","Deep learning has achieved great success in various computer vision tasks. When deploying deep learning models on different devices or a single device at different times, the amount of available resources can be different. Therefore, it is an important research topic to design neural networks that can adapt to various scenarios with different computational resources and achieve a trade-off between accuracy and efficiency. To this end, one branch of existing methods is adaptive inference with sample-wise dynamic architecture. These methods can be grouped into three categories: dynamic routing, dynamic depth, and dynamic width. In this dissertation, we will focus on dynamic post-training quantization (PQ) in the dynamic routing category and early exiting in the dynamic depth category. Existing PQ methods have two problems: (a) require a possibly inaccessible training dataset for model calibration, and (b) are hard to be generalized to low bit-width quantization precision. For early-exiting methods, we find there exists a train-test mismatch problem in prior works that can cause a train-test domain gap. In this dissertation, we will explore how to solve these three problems to improve the generalizability and performance of the dynamic models. Given constrained computational resources, PQ uses a calibration dataset to quantize a trained floating-point neural network to an efficient quantized one. The calibration dataset is usually sampled from training data. However, training data may not be available because of its large storage requirement or confidential issues when the model is being quantized. In the first part of this dissertation, we propose a cross-domain calibration method that uses out-of-domain data for calibration when in-domain training data is unavailable so that PQ can be applied more flexibly. PQ works well in the case of relatively high quantization bit-widths. To make the model work in a wider range of bit-widths, we propose Any-Precision Deep Neural Network (APDNN), which is trained with a new method that allows the learned network to be flexible in numerical precision during inference. The same model in runtime can be flexibly set to different bit-widths to support the dynamic trade-off of speed and accuracy. At each bit-width, the model achieves comparable accuracy to dedicated models trained at the same precision. We demonstrate that this achievement is agnostic to model architectures and applicable to multiple vision tasks. Going further in the direction of dynamic inference, we propose an early-exiting dynamic neural network (EDNN) that can adjust its inference complexity depending on not only computational resources but also input difficulty. When resources are limited or inputs are easy to process, the model will exit at some shallow layer of the model. Otherwise, the model can execute more layers to get a more confident prediction. Different from existing EDNN methods, we formulate EDNN as an additive model inspired by gradient boosting and propose Boosted Dynamic Neural Network (BoostNet). Our method mitigates the train-test mismatch problem that exists in the existing EDNN methods. In summary, the proposed cross-domain calibration method enables more flexible PQ for dynamic quantization without in-domain training data, the APDNN method generalizes dynamic quantization to extremely low bit-widths, and BoostNet introduces extra flexibility by making the model architecture depend on inputs and mitigates the train-test mismatch problem. In addition, BoostNet can be combined with the APDNN model to allow even more inference flexibility."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Adaptive model inference under computational resource constraints"]}]}],"canonical_facts":{"dc:contributor":["Shi, Humphrey","Liang, Zhi-Pei","Hasegawa-Johnson, Mark","Hua, Gang","Huang, Gao"],"dc:creator":["Yu, Haichao"],"dc:date":["2022-05","2022-04-11"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","The student, Haichao Yu, accepted the attached license on 2022-04-11 at 10:39.","The student, Haichao Yu, submitted this Dissertation for approval on 2022-04-11 at 10:57.","This Dissertation was approved for publication on 2022-04-11 at 12:31.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17626 on 2022-11-11 at 12:19:02","Deep learning has achieved great success in various computer vision tasks. When deploying deep learning models on different devices or a single device at different times, the amount of available resources can be different. Therefore, it is an important research topic to design neural networks that can adapt to various scenarios with different computational resources and achieve a trade-off between accuracy and efficiency. To this end, one branch of existing methods is adaptive inference with sample-wise dynamic architecture. These methods can be grouped into three categories: dynamic routing, dynamic depth, and dynamic width. In this dissertation, we will focus on dynamic post-training quantization (PQ) in the dynamic routing category and early exiting in the dynamic depth category. Existing PQ methods have two problems: (a) require a possibly inaccessible training dataset for model calibration, and (b) are hard to be generalized to low bit-width quantization precision. For early-exiting methods, we find there exists a train-test mismatch problem in prior works that can cause a train-test domain gap. In this dissertation, we will explore how to solve these three problems to improve the generalizability and performance of the dynamic models. Given constrained computational resources, PQ uses a calibration dataset to quantize a trained floating-point neural network to an efficient quantized one. The calibration dataset is usually sampled from training data. However, training data may not be available because of its large storage requirement or confidential issues when the model is being quantized. In the first part of this dissertation, we propose a cross-domain calibration method that uses out-of-domain data for calibration when in-domain training data is unavailable so that PQ can be applied more flexibly. PQ works well in the case of relatively high quantization bit-widths. To make the model work in a wider range of bit-widths, we propose Any-Precision Deep Neural Network (APDNN), which is trained with a new method that allows the learned network to be flexible in numerical precision during inference. The same model in runtime can be flexibly set to different bit-widths to support the dynamic trade-off of speed and accuracy. At each bit-width, the model achieves comparable accuracy to dedicated models trained at the same precision. We demonstrate that this achievement is agnostic to model architectures and applicable to multiple vision tasks. Going further in the direction of dynamic inference, we propose an early-exiting dynamic neural network (EDNN) that can adjust its inference complexity depending on not only computational resources but also input difficulty. When resources are limited or inputs are easy to process, the model will exit at some shallow layer of the model. Otherwise, the model can execute more layers to get a more confident prediction. Different from existing EDNN methods, we formulate EDNN as an additive model inspired by gradient boosting and propose Boosted Dynamic Neural Network (BoostNet). Our method mitigates the train-test mismatch problem that exists in the existing EDNN methods. In summary, the proposed cross-domain calibration method enables more flexible PQ for dynamic quantization without in-domain training data, the APDNN method generalizes dynamic quantization to extremely low bit-widths, and BoostNet introduces extra flexibility by making the model architecture depend on inputs and mitigates the train-test mismatch problem. In addition, BoostNet can be combined with the APDNN model to allow even more inference flexibility."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/115689"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Haichao Yu"],"dc:subject":["Adaptive Inference","Dynamic Neural Networks","Efficient Inference","Neural Network Quantization","Gradient Boosting"],"dc:title":["Adaptive model inference under computational resource constraints"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}