{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/125559"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/125559","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Hardware acceleration of neural graphics","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","abstract_has_math":false,"creators":["Mubarik, Muhammad Husnain"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Kumar, Rakesh","Chen, Deming","Gupta, Saurabh","Iyer, Ravi","Lin, Yingyan"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-07-12","date_published":"2024-07-12","updated_at":"2026-07-22T22:25:02Z","subjects":["Neural Radiance Fields","Latent Diffusion Models","Neural Graphics","Neural Representations","Diffusion Models","Virtual Reality","Augmented Reality","Mixed Reality"],"languages":["en","eng"],"rights":["Copyright 2024 Muhammad Husnain Mubarik"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/125559","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Kumar, Rakesh","Chen, Deming","Gupta, Saurabh","Iyer, Ravi","Lin, Yingyan"]},{"key":"dc:creator","label":"Author","values":["Mubarik, Muhammad Husnain"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-07-12","2024-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Neural Radiance Fields","Latent Diffusion Models","Neural Graphics","Neural Representations","Diffusion Models","Virtual Reality","Augmented Reality","Mixed Reality"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Muhammad Husnain Mubarik"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/125559"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","The student, Muhammad Husnain Mubarik, accepted the attached license on 2024-07-03 at 01:04.","The student, Muhammad Husnain Mubarik, submitted this Dissertation for approval on 2024-07-03 at 01:04.","This Dissertation was approved for publication on 2024-07-12 at 13:59.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20924 on 2025-02-04 at 21:04:11","Neural graphics is a rapidly evolving domain which uses neural networks to replace entire or parts of rendering pipeline. There are two broad classes of neural graphics: 1) Non-generative neural graphics where conventional DNNs are used for rendering. 2) Generative neural graphics where generative models such as latent diffusion models are employed. This dissertation studies the performance characteristic of both non-generative as well as generative neural graphics pipelines on modern hardware platforms, and asks the question: Does neural graphics need hardware support? The research is segmented into three primary areas: non-generative neural graphics via Neural Representations (NRs), generative neural graphics via Latent Diffusion Models (LDMs), and performance enhancements through integrating neural graphics with relatively more mature neural super-resolution techniques. Firstly, we study non-generative neural graphics by focusing on Neural Representations (NRs) which form the backbone of various applications like Neural Radiance and Density Fields (NeRF), Neural Signed Distance Functions (NSDF), Neural Volume Rendering (NVR), and Gigapixel Image Approximation (GIA). We identify significant performance gaps when rendering 4K resolution frames at 60 FPS on current GPUs, with a shortfall ranging from 1.51X to 55.50X. To bridge this gap, we propose the Neural Graphics Processing Cluster (NGPC), an architecture that accelerates input encoding and multi-layer perceptron kernels, achieving up to 58.36× improvement in application-level performance. Secondly, in the realm of generative neural graphics, we explore Latent Diffusion Models (LDMs) which have surpassed all the other generative processes in high-fidelity image generation domain. The computational demands of LDMs are immense, particularly at high resolutions. We performed in-depth breakdown analysis of the primitive compute kernels of the latent diffusion models and used our kernel-level analysis to design a scalable architecture – Latent Diffusion Model Processing Unit (LDPU). Our estimates show that LDPU increasing the energy efficiency by up to 572× compared to traditional GPU baselines. Lastly, the dissertation combines neural graphics with neural super-resolution techniques to further enhance the output quality and computational efficiency of neural rendering processes. By employing these techniques, we can render at lower resolutions and upscale the outputs to achieve desired higher resolutions without the computational cost typically associated with high-resolution rendering. This comprehensive study not only underscores the necessity of specialized hardware for neural graphics but also sets a precedent for future innovations in this domain, potentially revolutionizing applications in virtual reality (VR), augmented reality (AR), mixed reality (XR), and beyond."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Hardware acceleration of neural graphics"]}]}],"canonical_facts":{"dc:contributor":["Kumar, Rakesh","Chen, Deming","Gupta, Saurabh","Iyer, Ravi","Lin, Yingyan"],"dc:creator":["Mubarik, Muhammad Husnain"],"dc:date":["2024-07-12","2024-08"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","The student, Muhammad Husnain Mubarik, accepted the attached license on 2024-07-03 at 01:04.","The student, Muhammad Husnain Mubarik, submitted this Dissertation for approval on 2024-07-03 at 01:04.","This Dissertation was approved for publication on 2024-07-12 at 13:59.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20924 on 2025-02-04 at 21:04:11","Neural graphics is a rapidly evolving domain which uses neural networks to replace entire or parts of rendering pipeline. There are two broad classes of neural graphics: 1) Non-generative neural graphics where conventional DNNs are used for rendering. 2) Generative neural graphics where generative models such as latent diffusion models are employed. This dissertation studies the performance characteristic of both non-generative as well as generative neural graphics pipelines on modern hardware platforms, and asks the question: Does neural graphics need hardware support? The research is segmented into three primary areas: non-generative neural graphics via Neural Representations (NRs), generative neural graphics via Latent Diffusion Models (LDMs), and performance enhancements through integrating neural graphics with relatively more mature neural super-resolution techniques. Firstly, we study non-generative neural graphics by focusing on Neural Representations (NRs) which form the backbone of various applications like Neural Radiance and Density Fields (NeRF), Neural Signed Distance Functions (NSDF), Neural Volume Rendering (NVR), and Gigapixel Image Approximation (GIA). We identify significant performance gaps when rendering 4K resolution frames at 60 FPS on current GPUs, with a shortfall ranging from 1.51X to 55.50X. To bridge this gap, we propose the Neural Graphics Processing Cluster (NGPC), an architecture that accelerates input encoding and multi-layer perceptron kernels, achieving up to 58.36× improvement in application-level performance. Secondly, in the realm of generative neural graphics, we explore Latent Diffusion Models (LDMs) which have surpassed all the other generative processes in high-fidelity image generation domain. The computational demands of LDMs are immense, particularly at high resolutions. We performed in-depth breakdown analysis of the primitive compute kernels of the latent diffusion models and used our kernel-level analysis to design a scalable architecture – Latent Diffusion Model Processing Unit (LDPU). Our estimates show that LDPU increasing the energy efficiency by up to 572× compared to traditional GPU baselines. Lastly, the dissertation combines neural graphics with neural super-resolution techniques to further enhance the output quality and computational efficiency of neural rendering processes. By employing these techniques, we can render at lower resolutions and upscale the outputs to achieve desired higher resolutions without the computational cost typically associated with high-resolution rendering. This comprehensive study not only underscores the necessity of specialized hardware for neural graphics but also sets a precedent for future innovations in this domain, potentially revolutionizing applications in virtual reality (VR), augmented reality (AR), mixed reality (XR), and beyond."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/125559"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Muhammad Husnain Mubarik"],"dc:subject":["Neural Radiance Fields","Latent Diffusion Models","Neural Graphics","Neural Representations","Diffusion Models","Virtual Reality","Augmented Reality","Mixed Reality"],"dc:title":["Hardware acceleration of neural graphics"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:02Z"}