{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/121956"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/121956","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Efficient and effective learning of text representations","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","abstract_has_math":false,"creators":["Meng, Yu"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Han, Jiawei","Abdelzaher, Tarek","Tong, Hanghang","Zettlemoyer, Luke"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-12","date_published":"2023-12","updated_at":"2026-07-22T22:25:00Z","subjects":["Representation Learning","Natural Language Processing","Text Mining"],"languages":["en","eng"],"rights":["Copyright 2023 Yu Meng"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/121956","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Han, Jiawei","Abdelzaher, Tarek","Tong, Hanghang","Zettlemoyer, Luke"]},{"key":"dc:creator","label":"Author","values":["Meng, Yu"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-12","2023-10-13"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Representation Learning","Natural Language Processing","Text Mining"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Yu Meng"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/121956"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","The student, Yu Meng, accepted the attached license on 2023-10-11 at 13:53.","The student, Yu Meng, submitted this Dissertation for approval on 2023-10-11 at 14:11.","This Dissertation was approved for publication on 2023-10-13 at 14:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19857 on 2024-03-01 at 13:13:48","Text representation learning has played a pivotal role in enabling a wide range of Natural Language Processing (NLP) tasks that involve the processing, analysis, and utilization of human-generated textual data. These representations are typically obtained by transforming raw texts into vectors via deep neural networks (e.g., Transformers). Recent advances in Large Language Models (LLMs) have demonstrated the great potential of learning generic text representations applicable across a wide spectrum of applications. This success is underpinned by two critical factors: (1) the utilization of extensive text data for training LLMs in both pretraining and fine-tuning, and (2) the scaling up of LLMs to encompass tens or even hundreds of billions of parameters. Consequently, training LLMs entails substantial costs, including the acquisition of massive labeled data and the infrastructure required to support these large models. Motivated by these challenges, my doctoral research seeks to develop efficient and effective methods for learning text representations, spanning several key subtopics: 1. Leveraging the spherical space for text representation learning (Chapter 2). While the conventional choice for representation space is Euclidean, the non-Euclidean spherical space exhibits superior abilities to capture semantic correlations through directional similarity. My work focuses on self-supervised techniques that harness the spherical representation space for text representation learning. 2. Discovering topical structures with spherical text representations (Chapter 3). Based on the text representations learned in the spherical space, I have developed methods that automatically discover topical structures from a given corpus by jointly modeling topical and textual semantics. 3. Generating training data with LLMs for label-efficient natural language understanding (NLU) (Chapter 4). Achieving robust performance on NLU tasks typically necessitates a substantial number of human-annotated training samples for fine-tuning pretrained text representations. To mitigate the demands of human labeling, I have developed a new paradigm that employs LLMs as training data generators to replace the human annotation process. These endeavors collectively contribute to the more efficient and effective learning of text representations, addressing the challenges posed by the resource-intensive nature of training and using LLMs."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Efficient and effective learning of text representations"]}]}],"canonical_facts":{"dc:contributor":["Han, Jiawei","Abdelzaher, Tarek","Tong, Hanghang","Zettlemoyer, Luke"],"dc:creator":["Meng, Yu"],"dc:date":["2023-12","2023-10-13"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","The student, Yu Meng, accepted the attached license on 2023-10-11 at 13:53.","The student, Yu Meng, submitted this Dissertation for approval on 2023-10-11 at 14:11.","This Dissertation was approved for publication on 2023-10-13 at 14:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19857 on 2024-03-01 at 13:13:48","Text representation learning has played a pivotal role in enabling a wide range of Natural Language Processing (NLP) tasks that involve the processing, analysis, and utilization of human-generated textual data. These representations are typically obtained by transforming raw texts into vectors via deep neural networks (e.g., Transformers). Recent advances in Large Language Models (LLMs) have demonstrated the great potential of learning generic text representations applicable across a wide spectrum of applications. This success is underpinned by two critical factors: (1) the utilization of extensive text data for training LLMs in both pretraining and fine-tuning, and (2) the scaling up of LLMs to encompass tens or even hundreds of billions of parameters. Consequently, training LLMs entails substantial costs, including the acquisition of massive labeled data and the infrastructure required to support these large models. Motivated by these challenges, my doctoral research seeks to develop efficient and effective methods for learning text representations, spanning several key subtopics: 1. Leveraging the spherical space for text representation learning (Chapter 2). While the conventional choice for representation space is Euclidean, the non-Euclidean spherical space exhibits superior abilities to capture semantic correlations through directional similarity. My work focuses on self-supervised techniques that harness the spherical representation space for text representation learning. 2. Discovering topical structures with spherical text representations (Chapter 3). Based on the text representations learned in the spherical space, I have developed methods that automatically discover topical structures from a given corpus by jointly modeling topical and textual semantics. 3. Generating training data with LLMs for label-efficient natural language understanding (NLU) (Chapter 4). Achieving robust performance on NLU tasks typically necessitates a substantial number of human-annotated training samples for fine-tuning pretrained text representations. To mitigate the demands of human labeling, I have developed a new paradigm that employs LLMs as training data generators to replace the human annotation process. These endeavors collectively contribute to the more efficient and effective learning of text representations, addressing the challenges posed by the resource-intensive nature of training and using LLMs."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/121956"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Yu Meng"],"dc:subject":["Representation Learning","Natural Language Processing","Text Mining"],"dc:title":["Efficient and effective learning of text representations"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:00Z"}