Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 39 for “"Knowledge distillation"”.
-
Enhanced knowledge distillation by auxiliary classifiers
… proposed approaches to mitigate this limitation, knowledge distillation has gained much attention due to its generalizability and simplicity in implementation. This thesis introduces the enhanced knowledge distillation (EKD), a simple yet effective approach to outperform the canonical knowledge …
-
Efficient transformer-based panoptic segmentation via knowledge distillation
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms
-
Temperature-centric investigation of speculative decoding with knowledge distillation
… speculative decoding’s efficacy. Beginning with knowledge distillation (KD), we first highlight the challenge of decoding at higher temperatures, and demonstrate KD in a consistent temperature setting could be a remedy. We also investigate the effects of out-of-domain testing sets with …
-
Knowledge Distillation for Interpretable Clinical Time Series Outcome Prediction
… used in the real world. In this thesis, we use knowledge distillation, which is a technique for taking a model with high predictive power (known as the "teacher model"), and using it to train a model that has other desirable traits such as interpretability (known as the "student model"). For our …
-
Enhancing knowledge distillation in large language models via domain adaptation
… on specialized domains, yet its interaction with knowledge distillation (KD) remains poorly understood. In particular, intermediate DAPT checkpoints are rarely analyzed, and the evolution of teacher uncertainty across such checkpoints has not been systematically studied. This thesis develops a …
-
Towards searching for the best student in a Knowledge Distillation framework
Knowledge Distillation (KD) enables the creation of compact student models that can retain much of the predictive power of larger teacher models. While these student models are invaluable for deployment in resource-constrained environments, the process of identifying an optimal student—balancing …
-
Knowledge Distillation in DMRS CHEST ML to Optimize Radio Performance and Hardware Efficiency
… conditions. This thesis explores using knowledge distillation (KD) to improve machine learning-based channel estimation in 5G systems. A convolutional neural network (CNN) acts as a teacher model, trained on synthetic data, to create a lightweight student CNN through response-based and …
-
Robust and Efficient AI-models for Medical Image Reconstruction, Segmentation, and Multimodal Knowledge Distillation
… semantic segmentation, multimodal knowledge integration, and adversarial robustness. I propose Teach-Former, a multi-teacher knowledge distillation framework that enables lightweight models to absorb rich spatial and contextual representations from multiple large networks, achieving …
-
Toward building more accessible large language models: A preliminary empirical study on data scarcity in knowledge distillation and algorithm complexity in alignment
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms
-
Development of a Fully Automated Al System for Skeletal Maturity Assessment Using Cone Beam CT
… and techniques—including YOLO, SAM, Grad-CAM, knowledge distillation, and demographic conditioning—were incorporated into the development of several frameworks. The highest-performing, fully automated pipeline for SOS fusion staging was ConvNeXt with self-attention, YOLO, and knowledge …
-
Efficient Edge Intelligence in the Era of Big Data
… big data analysis. We propose to leverage knowledge distillation to achieve an ultra-efficient edge-deployable deep learning model. More specifically, through transferring the knowledge from a teacher model to the on-edge student model, the soft target distribution of the teacher model can …
-
REFT: Resource-Efficient Federated Training Framework for Heterogeneous and Resource-Constrained Environments
… Existing works share model parameters or use distillation principles to address the challenges of data heterogeneity. However, these methods ignore some of the other fundamental challenges in FL: device heterogeneity and communication efficiency. In practice, client devices in FL differ …
-
Action Labeling in Images and Video
… framework, multimodal data fusion, and knowledge distillation to improve deep learning models' performance. These methods are assessed for the problems of: (i) recognizing carrying actions in "visible spectrum" and "near-infrared" images, as well as (ii) detecting questionable online …
-
LightMARL : smart swarm coordination in urban spaces
… network quantization, structured pruning, and knowledge distillation tailored for multi-agent policy optimization (MAPPO). These methods decrease model size and computational demands while maintaining essential coordination behaviors. To enhance communication efficiency, vector quantization …
-
Switching State Space Modeling via Constrained Inference for Clinical Outcome Prediction
… outcome prediction. Our method leverages knowledge distillation: a high-capacity LSTM "teacher" model is first trained to predict a target clinical outcome of interest, and its predictive behavior is then transferred to an interpretable AR-HMM "student" model through a similarity …
-
Towards General-purpose Vision via Multiview Contrastive Learning
… part, we discuss other applications (such as knowledge distillation) and improvements of multiview contrastive learning (e.g., how to improve its efficiency on uncurated data).
-
Continual Learning for Deep Dense Prediction
… as it does not allow us to accumulate knowledge sequentially and requires retaining and retraining on all the training data. Existing techniques for mitigating the abrupt performance degradation on previously trained tasks are mainly studied in the context of image classification. In …
-
Distributed Deep Learning in IoT: Splitting Neural Networks in Inference and Training
… rate. We then propose a scheme applying Online Knowledge Distillation via Collaboratively Learning (KDCL) to Split Learning to get a distilled model of server-part model in SplitNN after training. The distilled model could be designed to fit the memory-limited Internet of Things (IoT) devices so …
-
Millimeter-wave radar dataset for multi-modal fusion and keypoint detection in under-canopy soybean and corn row navigation
… (low light, dust, and foliage cover). To our knowledge, this is the first multi-modal dataset of its kind in agricultural settings. Building on this resource, we investigate an end-to-end deep learning framework for radar-based keypoint prediction that leverages unprocessed range–azimuth radar …
Page 1 of 2