{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/124722"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/124722","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Learning structured representations with hyperbolic embeddings","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","abstract_has_math":false,"creators":["Sinha, Aditya"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Zhao, Han"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-05","date_published":"2024-05","updated_at":"2026-07-22T22:25:02Z","subjects":["Representation Learning","Hyperbolic Geometry","Hierarchical Labels","Ood Detection"],"languages":["en","eng"],"rights":["Copyright 2024 Aditya Sinha"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/124722","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhao, Han"]},{"key":"dc:creator","label":"Author","values":["Sinha, Aditya"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-05","2024-05-01"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Representation Learning","Hyperbolic Geometry","Hierarchical Labels","Ood Detection"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Aditya Sinha"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/124722"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","The student, Aditya Sinha, accepted the attached license on 2024-04-30 at 16:28.","The student, Aditya Sinha, submitted this Thesis for approval on 2024-04-30 at 16:44.","This Thesis was approved for publication on 2024-05-01 at 09:57.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20726 on 2024-09-16 at 00:51:12","Most real-world data consists of a natural hierarchy or an inherent label structure that is either already available or can be constructed/inferred cheaply. However, majority of the existing models for representation learning either completely ignore this hierarchy, treating the labels as permutation invariant, or attempt to utilize this information using less desirable distance metrics with bounded dimensionality. This leads to distortion of the semantic context in the label hierarchy, and also adversely affect it's performance on the in-distribution (ID) classification task. In addition, in the context of real-world machine learning systems, Out-of-distribution (OOD) detection is an even more challenging and critical task to ensure reliability of the deployed models. However, the current distance-based approaches do not consider any structured knowledge and rely on a distance measurement from the ID cluster-centroids, learnt in a label invariant fashion. To approach these challenges, in this thesis, we propose using hyperbolic geometry for incorporating this rich structured hierarchy about the label space into the representation learning. We demonstrate that accurately embedding the label information can lead to more fine-grained learning of structure-informed features, which are discriminative and helpful for a variety of tasks. For this purpose, we propose a novel method HypCPCC: Hyperbolic Cophenetic Correlation Coefficient to embed the label hierarchy into features using a powerful hyperbolic geometry based tree regularization objective. Our proposed objective can easily be combined and optimized with any classification loss for improving representation learning. We also empirically demonstrate that HypCPCC accurately embeds the hierarchical relationships between the labels in web-scale real-world vision datasets and leads to learning semantically rich features that result in simultaneous improvements in the performance of both in-distribution (ID) classification tasks (upto 1%) and AUROC on Out-of-Distribution (OOD) detection tasks (upto 4%). Motivated by the expressiveness of hyperbolic geometry in embedding the tree-based structures, we also propose two principled non-parametric hyperbolic-distance based OOD detection scores: HypKNN+ and HypDist-O. We demonstrate that using these scores can lead to improvements in OOD detection FPR95 upto 2%. Finally, we also empirically show that the learnt features from our proposed methodology are geometrically and semantically more interpretable using hyperbolic visualizations, paving the way for explainable feature learning using hierarchical label information."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Learning structured representations with hyperbolic embeddings"]}]}],"canonical_facts":{"dc:contributor":["Zhao, Han"],"dc:creator":["Sinha, Aditya"],"dc:date":["2024-05","2024-05-01"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-05-01","The student, Aditya Sinha, accepted the attached license on 2024-04-30 at 16:28.","The student, Aditya Sinha, submitted this Thesis for approval on 2024-04-30 at 16:44.","This Thesis was approved for publication on 2024-05-01 at 09:57.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20726 on 2024-09-16 at 00:51:12","Most real-world data consists of a natural hierarchy or an inherent label structure that is either already available or can be constructed/inferred cheaply. However, majority of the existing models for representation learning either completely ignore this hierarchy, treating the labels as permutation invariant, or attempt to utilize this information using less desirable distance metrics with bounded dimensionality. This leads to distortion of the semantic context in the label hierarchy, and also adversely affect it's performance on the in-distribution (ID) classification task. In addition, in the context of real-world machine learning systems, Out-of-distribution (OOD) detection is an even more challenging and critical task to ensure reliability of the deployed models. However, the current distance-based approaches do not consider any structured knowledge and rely on a distance measurement from the ID cluster-centroids, learnt in a label invariant fashion. To approach these challenges, in this thesis, we propose using hyperbolic geometry for incorporating this rich structured hierarchy about the label space into the representation learning. We demonstrate that accurately embedding the label information can lead to more fine-grained learning of structure-informed features, which are discriminative and helpful for a variety of tasks. For this purpose, we propose a novel method HypCPCC: Hyperbolic Cophenetic Correlation Coefficient to embed the label hierarchy into features using a powerful hyperbolic geometry based tree regularization objective. Our proposed objective can easily be combined and optimized with any classification loss for improving representation learning. We also empirically demonstrate that HypCPCC accurately embeds the hierarchical relationships between the labels in web-scale real-world vision datasets and leads to learning semantically rich features that result in simultaneous improvements in the performance of both in-distribution (ID) classification tasks (upto 1%) and AUROC on Out-of-Distribution (OOD) detection tasks (upto 4%). Motivated by the expressiveness of hyperbolic geometry in embedding the tree-based structures, we also propose two principled non-parametric hyperbolic-distance based OOD detection scores: HypKNN+ and HypDist-O. We demonstrate that using these scores can lead to improvements in OOD detection FPR95 upto 2%. Finally, we also empirically show that the learnt features from our proposed methodology are geometrically and semantically more interpretable using hyperbolic visualizations, paving the way for explainable feature learning using hierarchical label information."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/124722"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Aditya Sinha"],"dc:subject":["Representation Learning","Hyperbolic Geometry","Hierarchical Labels","Ood Detection"],"dc:title":["Learning structured representations with hyperbolic embeddings"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:02Z"}