{"id":{"repo_id":"uic","oai_identifier":"oai:figshare.com:article/32991827"},"canonical_url":"https://search.dev.ndltd.org/etd/uic/oai:figshare.com:article/32991827","repository":{"repo_id":"uic","name":"University of Illinois - Chicago","base_url":"https://api.figshare.com/v2/oai"},"display":{"title":"Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory","abstract":"The rapid growth of deep learning has brought both unprecedented capabilities and pressing challenges. On one hand, large neural networks deliver state-of-the-art performance across natural language processing, computer vision, and multimodal domains. On the other, their reliability is compromised by hallucinations and out-of-distribution errors, while their scale imposes severe efficiency and deployment barriers. This thesis develops a unifying spectral framework, grounded in Random Matrix Theory (RMT), to address both reliability and efficiency in modern AI systems. First, it introduces EigenTrack, a real-time detector of hallucination and distributional shift in large language and vision-language models. By extracting spectral statistics from sliding-window activation covariances, entropy, eigenvalue gaps, and divergence from the Marchenko–Pastur law, and modeling their temporal evolution with lightweight recurrent classifiers, EigenTrack detects anomalies before they manifest in model outputs. Results show state-of-the-art performance across multiple LLM and VLM families, offering interpretable signatures of failure dynamics. Second, the thesis presents RMT-KD, an iterative knowledge distillation method that applies random matrix principles to compress deep networks. By isolating outlier eigenvalues of hidden activations as carriers of causal structure, RMT-KD progressively projects models onto informative subspaces while preserving accuracy through self-distillation. Experiments on BERT, ResNet, and benchmark datasets demonstrate major parameter reduction with minimal accuracy loss, yielding faster inference and significant energy savings. These contributions establish spectral geometry as a principled lens for both diagnosing uncertainty and guiding compression. By linking eigenvalue dynamics to representation quality, the thesis advances interpretable, mathematically grounded methods that make AI systems simultaneously more trustworthy and more efficient.","abstract_html":"The rapid growth of deep learning has brought both unprecedented capabilities and pressing challenges. On one hand, large neural networks deliver state-of-the-art performance across natural language processing, computer vision, and multimodal domains. On the other, their reliability is compromised by hallucinations and out-of-distribution errors, while their scale imposes severe efficiency and deployment barriers. This thesis develops a unifying spectral framework, grounded in Random Matrix Theory (RMT), to address both reliability and efficiency in modern AI systems. First, it introduces EigenTrack, a real-time detector of hallucination and distributional shift in large language and vision-language models. By extracting spectral statistics from sliding-window activation covariances, entropy, eigenvalue gaps, and divergence from the Marchenko–Pastur law, and modeling their temporal evolution with lightweight recurrent classifiers, EigenTrack detects anomalies before they manifest in model outputs. Results show state-of-the-art performance across multiple LLM and VLM families, offering interpretable signatures of failure dynamics. Second, the thesis presents RMT-KD, an iterative knowledge distillation method that applies random matrix principles to compress deep networks. By isolating outlier eigenvalues of hidden activations as carriers of causal structure, RMT-KD progressively projects models onto informative subspaces while preserving accuracy through self-distillation. Experiments on BERT, ResNet, and benchmark datasets demonstrate major parameter reduction with minimal accuracy loss, yielding faster inference and significant energy savings. These contributions establish spectral geometry as a principled lens for both diagnosing uncertainty and guiding compression. By linking eigenvalue dynamics to representation quality, the thesis advances interpretable, mathematically grounded methods that make AI systems simultaneously more trustworthy and more efficient.","abstract_has_math":false,"creators":["Davide Ettori (24398936)"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2026,"date_issued":"2026-07-15T11:58:32Z","date_published":"2026-07-15T11:58:32Z","updated_at":"2026-07-27T21:33:07Z","subjects":["Deep Learning","Large Language Models","Mathematical Modeling"],"languages":[],"rights":["In Copyright"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.25417/uic.32991827.v1","outbound_label":"DOI","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Davide Ettori (24398936)"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2026-07-15T11:58:32Z"]},{"key":"dc:relation","label":"Dc Relation","values":["https://figshare.com/articles/thesis/Spectral_Geometry_for_Deep_Learning_Compression_and_Hallucination_Detection_via_Random_Matrix_Theory/32991827"]},{"key":"dc:type","label":"Dc Type","values":["Text","Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Deep Learning","Large Language Models","Mathematical Modeling"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["10.25417/uic.32991827.v1"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["The rapid growth of deep learning has brought both unprecedented capabilities and pressing challenges. On one hand, large neural networks deliver state-of-the-art performance across natural language processing, computer vision, and multimodal domains. On the other, their reliability is compromised by hallucinations and out-of-distribution errors, while their scale imposes severe efficiency and deployment barriers. This thesis develops a unifying spectral framework, grounded in Random Matrix Theory (RMT), to address both reliability and efficiency in modern AI systems. First, it introduces EigenTrack, a real-time detector of hallucination and distributional shift in large language and vision-language models. By extracting spectral statistics from sliding-window activation covariances, entropy, eigenvalue gaps, and divergence from the Marchenko–Pastur law, and modeling their temporal evolution with lightweight recurrent classifiers, EigenTrack detects anomalies before they manifest in model outputs. Results show state-of-the-art performance across multiple LLM and VLM families, offering interpretable signatures of failure dynamics. Second, the thesis presents RMT-KD, an iterative knowledge distillation method that applies random matrix principles to compress deep networks. By isolating outlier eigenvalues of hidden activations as carriers of causal structure, RMT-KD progressively projects models onto informative subspaces while preserving accuracy through self-distillation. Experiments on BERT, ResNet, and benchmark datasets demonstrate major parameter reduction with minimal accuracy loss, yielding faster inference and significant energy savings. These contributions establish spectral geometry as a principled lens for both diagnosing uncertainty and guiding compression. By linking eigenvalue dynamics to representation quality, the thesis advances interpretable, mathematically grounded methods that make AI systems simultaneously more trustworthy and more efficient."]},{"key":"dc:title","label":"Title","values":["Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory"]}]}],"canonical_facts":{"dc:creator":["Davide Ettori (24398936)"],"dc:date":["2026-07-15T11:58:32Z"],"dc:description":["The rapid growth of deep learning has brought both unprecedented capabilities and pressing challenges. On one hand, large neural networks deliver state-of-the-art performance across natural language processing, computer vision, and multimodal domains. On the other, their reliability is compromised by hallucinations and out-of-distribution errors, while their scale imposes severe efficiency and deployment barriers. This thesis develops a unifying spectral framework, grounded in Random Matrix Theory (RMT), to address both reliability and efficiency in modern AI systems. First, it introduces EigenTrack, a real-time detector of hallucination and distributional shift in large language and vision-language models. By extracting spectral statistics from sliding-window activation covariances, entropy, eigenvalue gaps, and divergence from the Marchenko–Pastur law, and modeling their temporal evolution with lightweight recurrent classifiers, EigenTrack detects anomalies before they manifest in model outputs. Results show state-of-the-art performance across multiple LLM and VLM families, offering interpretable signatures of failure dynamics. Second, the thesis presents RMT-KD, an iterative knowledge distillation method that applies random matrix principles to compress deep networks. By isolating outlier eigenvalues of hidden activations as carriers of causal structure, RMT-KD progressively projects models onto informative subspaces while preserving accuracy through self-distillation. Experiments on BERT, ResNet, and benchmark datasets demonstrate major parameter reduction with minimal accuracy loss, yielding faster inference and significant energy savings. These contributions establish spectral geometry as a principled lens for both diagnosing uncertainty and guiding compression. By linking eigenvalue dynamics to representation quality, the thesis advances interpretable, mathematically grounded methods that make AI systems simultaneously more trustworthy and more efficient."],"dc:identifier":["10.25417/uic.32991827.v1"],"dc:relation":["https://figshare.com/articles/thesis/Spectral_Geometry_for_Deep_Learning_Compression_and_Hallucination_Detection_via_Random_Matrix_Theory/32991827"],"dc:rights":["In Copyright"],"dc:subject":["Deep Learning","Large Language Models","Mathematical Modeling"],"dc:title":["Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory"],"dc:type":["Text","Thesis"]},"updated_at":"2026-07-27T21:33:07Z"}