Efficient Deep Learning: Model Design and Algorithmic Innovation

The rapid evolution of Artificial Intelligence (AI) and Deep Learning (DL) has revolutionized numerous domains, from computer vision to natural language processing and intelligent recommendation systems. However, this progress has been accompanied by escalating computational demands that challenge the scalability and practical deployment of modern AI systems. This thesis addresses these efficiency challenges through an integrated approach, introducing novel methodologies that co-optimize performance and resource utilization in model design, training, and deployment. Central to this research is the development of advanced Neural Architecture Search (NAS) frameworks. These frameworks enable the automated design of lightweight, task-specific neural networks optimized for resource-constrained scenarios, such as mobile devices and edge computing environments. By employing a semi-supervised NAS strategy, the thesis demonstrates how efficient models can be designed with minimal labeled data, addressing the prevalent challenge of data scarcity in real-world applications. The proposed methods facilitate adaptation to diverse hardware platforms, ensuring high performance while adhering to strict latency, memory, and energy constraints. In addition to model design, the thesis explores the efficiency of generative models, a rapidly growing area of DL research. Novel contributions include hierarchical latent space optimization techniques that accelerate the image generation process and conditional diffusion mechanisms tailored for recommendation systems. These advancements streamline the generative workflows, significantly reducing computational overhead while maintaining state-of-the-art accuracy and output quality. Furthermore, the research introduces segment-wise NAS techniques, which dynamically allocate computational resources across different stages of the diffusion process, achieving optimal efficiency without sacrificing performance. This research offers significant contributions to the field of efficient AI, bridging the gap between cutting-edge algorithmic innovations and practical, resource-aware deployment strategies. It highlights the need for holistic design principles that prioritize both accuracy and efficiency, ensuring the scalability and sustainability of AI in an increasingly resource-constrained world. The findings of this thesis are validated through extensive experimentation on diverse benchmarks.

Grantor dc:publisher

UNSW, Sydney

Year dc:date

2025

Author dc:creator

• Huang, Hongtao

• Resource-efficient Deep Learning
• anzsrc-for: 46 INFORMATION AND COMPUTING SCIENCES

Statement dc:rights

• open access
• CC BY 4.0
• free_to_read

Licence

• https://purl.org/coar/access_right/c_abf2
• https://creativecommons.org/licenses/by/4.0/

Language dc:language

en

Identifier

https://doi.org/10.26190/unsworks/31524

OAI identifier oai:identifier

oai:unsworks.library.unsw.edu.au:1959.4/105642