{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/108566"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/108566","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"TensorRT inference performance study in MLModelScope","abstract":"As deep learning has been adopted in various domains, the inference process is of growing importance to ensure the deployment across multiple computing platforms. Within many deep learning frameworks that support freezing and deploying the well-trained models, NVIDIA TensorRT is the leading framework that is exclusively developed for inference. It allows the developer to optimize the model to facilitate high-performance inference. While it has been shown extensively that TensorRT can significantly boost the inference capability, quantitative study is lacking on how assorted optimization strategies can improve the inference compared to other well-known deep learning frameworks such as TensorFlow. This thesis presents such a study that consists of experiments using TensorRT on MLModelScope, a deep learning inference platform that enables standardized inference and multi-level profiling.","abstract_html":"As deep learning has been adopted in various domains, the inference process is of growing importance to ensure the deployment across multiple computing platforms. Within many deep learning frameworks that support freezing and deploying the well-trained models, NVIDIA TensorRT is the leading framework that is exclusively developed for inference. It allows the developer to optimize the model to facilitate high-performance inference. While it has been shown extensively that TensorRT can significantly boost the inference capability, quantitative study is lacking on how assorted optimization strategies can improve the inference compared to other well-known deep learning frameworks such as TensorFlow. This thesis presents such a study that consists of experiments using TensorRT on MLModelScope, a deep learning inference platform that enables standardized inference and multi-level profiling.","abstract_has_math":false,"creators":["Tang, Jingning"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Hwu, Wen-Mei"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-10-07T22:07:11Z","date_published":"2020-10-07T22:07:11Z","updated_at":"2026-07-22T22:24:48Z","subjects":["deep learning","inference","TensorRT","MLModelScope"],"languages":["en"],"rights":["Copyright 2020 Jingning Tang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/108566","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hwu, Wen-Mei"]},{"key":"dc:creator","label":"Author","values":["Tang, Jingning"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-10-07T22:07:11Z","2022-10-07T22:44:53Z","2020-06-23","2020-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["deep learning","inference","TensorRT","MLModelScope"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Jingning Tang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/108566"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["As deep learning has been adopted in various domains, the inference process is of growing importance to ensure the deployment across multiple computing platforms. Within many deep learning frameworks that support freezing and deploying the well-trained models, NVIDIA TensorRT is the leading framework that is exclusively developed for inference. It allows the developer to optimize the model to facilitate high-performance inference. While it has been shown extensively that TensorRT can significantly boost the inference capability, quantitative study is lacking on how assorted optimization strategies can improve the inference compared to other well-known deep learning frameworks such as TensorFlow. This thesis presents such a study that consists of experiments using TensorRT on MLModelScope, a deep learning inference platform that enables standardized inference and multi-level profiling.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-08-01","The student, Jingning Tang, accepted the attached license on 2020-06-19 at 08:35.","The student, Jingning Tang, submitted this Thesis for approval on 2020-06-19 at 08:39.","This Thesis was approved for publication on 2020-06-23 at 09:21.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15457 on 2020-10-02 at 15:30:51","Made available in DSpace on 2020-10-07T22:07:11Z (GMT). No. of bitstreams: 2 TANG-THESIS-2020.pdf: 3133439 bytes, checksum: 4ce2b3e664954ad25e368f41801ba44d (MD5) LICENSE.txt: 4210 bytes, checksum: ee5b3085186c0a68f730b2ad2108ec06 (MD5) Previous issue date: 2020-06-23","Embargo set by: Seth Robbins for item 116192 Lift date: 2022-10-07T22:07:19Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 116192 Lift date: 2022-10-07T22:44:53Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["TensorRT inference performance study in MLModelScope"]}]}],"canonical_facts":{"dc:contributor":["Hwu, Wen-Mei"],"dc:creator":["Tang, Jingning"],"dc:date":["2020-10-07T22:07:11Z","2022-10-07T22:44:53Z","2020-06-23","2020-08"],"dc:description":["As deep learning has been adopted in various domains, the inference process is of growing importance to ensure the deployment across multiple computing platforms. Within many deep learning frameworks that support freezing and deploying the well-trained models, NVIDIA TensorRT is the leading framework that is exclusively developed for inference. It allows the developer to optimize the model to facilitate high-performance inference. While it has been shown extensively that TensorRT can significantly boost the inference capability, quantitative study is lacking on how assorted optimization strategies can improve the inference compared to other well-known deep learning frameworks such as TensorFlow. This thesis presents such a study that consists of experiments using TensorRT on MLModelScope, a deep learning inference platform that enables standardized inference and multi-level profiling.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-08-01","The student, Jingning Tang, accepted the attached license on 2020-06-19 at 08:35.","The student, Jingning Tang, submitted this Thesis for approval on 2020-06-19 at 08:39.","This Thesis was approved for publication on 2020-06-23 at 09:21.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15457 on 2020-10-02 at 15:30:51","Made available in DSpace on 2020-10-07T22:07:11Z (GMT). No. of bitstreams: 2 TANG-THESIS-2020.pdf: 3133439 bytes, checksum: 4ce2b3e664954ad25e368f41801ba44d (MD5) LICENSE.txt: 4210 bytes, checksum: ee5b3085186c0a68f730b2ad2108ec06 (MD5) Previous issue date: 2020-06-23","Embargo set by: Seth Robbins for item 116192 Lift date: 2022-10-07T22:07:19Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 116192 Lift date: 2022-10-07T22:44:53Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/108566"],"dc:language":["en"],"dc:rights":["Copyright 2020 Jingning Tang"],"dc:subject":["deep learning","inference","TensorRT","MLModelScope"],"dc:title":["TensorRT inference performance study in MLModelScope"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:48Z"}