{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/121494"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/121494","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Efficient and robust web scale language model based retrieval, generation, and understanding","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-12-04 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-12-04 without embargo terms","abstract_has_math":false,"creators":["Campos, Daniel Fernando"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Zhai, Cheng Xiang","Magnani, Alessandro","Han, Jiawei","Chang, Kevin"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-08","date_published":"2023-08","updated_at":"2026-07-22T22:24:57Z","subjects":["Efficient Inference","Language Model","Semantic Retrieval","Web-scale Inference"],"languages":["en","eng"],"rights":["Copyright 2023 Daniel Campos"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/121494","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhai, Cheng Xiang","Magnani, Alessandro","Han, Jiawei","Chang, Kevin"]},{"key":"dc:creator","label":"Author","values":["Campos, Daniel Fernando"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-08","2023-07-13"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Efficient Inference","Language Model","Semantic Retrieval","Web-scale Inference"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Daniel Campos"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/121494"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-12-04 without embargo terms","The student, Daniel Campos, accepted the attached license on 2023-07-10 at 15:42.","The student, Daniel Campos, submitted this Dissertation for approval on 2023-07-10 at 15:45.","This Dissertation was approved for publication on 2023-07-13 at 08:27.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19606 on 2023-12-04 at 17:01:40","Large language models effectively generate contextualized word representations across languages, domains, and tasks. Drive by these abilities, these models have become a build- ing staple for many researchers and engineers who use text as their medium of representation, much like concrete is a staple in the construction world. Via the broad study and imple- mentation, problems with large models have come to light: they can be expensive, brittle to noise, and produce unwanted outputs. Their large size and computational overhead make them difficult and costly to deploy and use for inference. Minor variations in text inputs, such as typos or misspellings, can cause significant losses in model accuracy. Seeking to improve how these models can be used for real world usage and deployments, this thesis fo- cuses on approaches for improving performance by compressing, hardening, and optimizing models’ performance based on deployment needs. To explore the challenges with large- scale deployments concerning robustness and inference efficiency, we explore four commonly used language workloads: textual understanding and classification, passage retrieval, and text generation. We chose these broad but connected tasks to ensure that our compres- sion approaches broadly apply to natural language processing. First, we propose a general framework for improving model inference on broad language understanding workloads by studying how unstructured pruning, structured pruning, and quantization can be leveraged to compress models and improve inference speeds. Second, we examine how models can be deployed for usage in web-scale generation and understanding workloads. Leveraging multi-task modeling, asymmetrical pruning, knowledge distillation, and quantization allows for cost-efficient scaling to web-scale workloads. Third, we explore methods of tuning and optimizing dense retrieval methods post-training to ensure they perform well on real-world data. Our experiments yield simple and effective ways of increasing model robustness and decreasing inference costs without any need for retraining or index re-generation. Finally, we discuss future work, focusing on sequential compression approaches to sequence LLMs to allow generative workloads to reach web-scale deployments."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Efficient and robust web scale language model based retrieval, generation, and understanding"]}]}],"canonical_facts":{"dc:contributor":["Zhai, Cheng Xiang","Magnani, Alessandro","Han, Jiawei","Chang, Kevin"],"dc:creator":["Campos, Daniel Fernando"],"dc:date":["2023-08","2023-07-13"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-12-04 without embargo terms","The student, Daniel Campos, accepted the attached license on 2023-07-10 at 15:42.","The student, Daniel Campos, submitted this Dissertation for approval on 2023-07-10 at 15:45.","This Dissertation was approved for publication on 2023-07-13 at 08:27.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19606 on 2023-12-04 at 17:01:40","Large language models effectively generate contextualized word representations across languages, domains, and tasks. Drive by these abilities, these models have become a build- ing staple for many researchers and engineers who use text as their medium of representation, much like concrete is a staple in the construction world. Via the broad study and imple- mentation, problems with large models have come to light: they can be expensive, brittle to noise, and produce unwanted outputs. Their large size and computational overhead make them difficult and costly to deploy and use for inference. Minor variations in text inputs, such as typos or misspellings, can cause significant losses in model accuracy. Seeking to improve how these models can be used for real world usage and deployments, this thesis fo- cuses on approaches for improving performance by compressing, hardening, and optimizing models’ performance based on deployment needs. To explore the challenges with large- scale deployments concerning robustness and inference efficiency, we explore four commonly used language workloads: textual understanding and classification, passage retrieval, and text generation. We chose these broad but connected tasks to ensure that our compres- sion approaches broadly apply to natural language processing. First, we propose a general framework for improving model inference on broad language understanding workloads by studying how unstructured pruning, structured pruning, and quantization can be leveraged to compress models and improve inference speeds. Second, we examine how models can be deployed for usage in web-scale generation and understanding workloads. Leveraging multi-task modeling, asymmetrical pruning, knowledge distillation, and quantization allows for cost-efficient scaling to web-scale workloads. Third, we explore methods of tuning and optimizing dense retrieval methods post-training to ensure they perform well on real-world data. Our experiments yield simple and effective ways of increasing model robustness and decreasing inference costs without any need for retraining or index re-generation. Finally, we discuss future work, focusing on sequential compression approaches to sequence LLMs to allow generative workloads to reach web-scale deployments."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/121494"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Daniel Campos"],"dc:subject":["Efficient Inference","Language Model","Semantic Retrieval","Web-scale Inference"],"dc:title":["Efficient and robust web scale language model based retrieval, generation, and understanding"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:57Z"}