DYNAMIC NEURAL NETWORKS FOR EFFICIENT VISION MODEL INFERENCE

Recent progress in dynamic neural networks is hindered by high training overhead from full-parameter fine-tuning and limited application beyond visual perception. This thesis addresses these challenges through three key contributions: Parameter-Efficient Fine-Tuning (PEFT): We demonstrate that dynamic networks can be developed with negligible trainable parameters, reducing FLOPs by 30% while significantly lowering training costs. Diffusion Transformers (DiT) Acceleration: We introduce a dynamic architecture for visual generation, achieving a 50% FLOPs reduction and 1.7× speedup with improved quality. Vision-Language Model (VLM) Acceleration: By implementing a dynamic small-large model cooperation mechanism, we achieve up to 3× faster inference while maintaining competitive performance. In conclusion, this research provides a comprehensive solution to reduce training costs and extends dynamic neural networks to visual generation and multi-modal domains, unlocking new possibilities for efficient real-world applications.

Author dc:creator

• ZHAO WANGBO

Advisor dc:contributor.supervisor

• You Yang

• Dynamic Neural Networks; Vision Models; Efficient Inference

Licence

• https://scholarbank.nus.edu.sg/bitstreams/1b1e9b35-ac3a-4afb-a1ca-c03f4e1df669/download

Language dc:language.iso

en

Repository record dc:identifier.uri

https://scholarbank.nus.edu.sg/handle/10635/317970

OAI record oai:identifier

https://scholarbank.nus.edu.sg/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Ascholarbank.nus.edu.sg%3A10635%2F317970