{"id":{"repo_id":"texas","oai_identifier":"oai:repositories.lib.utexas.edu:2152/115131"},"canonical_url":"https://search.dev.ndltd.org/etd/texas/oai:repositories.lib.utexas.edu:2152/115131","repository":{"repo_id":"texas","name":"University of Texas","base_url":"https://repositories.lib.utexas.edu/server/oai/request"},"display":{"title":"Accelerating deep learning training : a storage perspective","abstract":"Deep Learning, specifically Deep Neural Networks (DNNs), is stressing storage systems in new ways, moving the training bottleneck to the data pipeline (fetching, pre-processing data, and writing checkpoints), rather than computation at the GPUs; this leaves the expensive accelerator devices stalled for data. While prior research has explored different ways of accelerating DNN training time, the impact of storage systems, specifically the data pipeline, on ML training has been relatively unexplored. In this dissertation, we study the role of data pipeline in various training scenarios, and based on the insights from our study, we present the design and evaluation of systems that accelerate training. We first present a comprehensive analysis of how the storage subsystem affects the training of the widely used DNN models by building a tool, DS-Analyzer. Our study reveals that in many cases, DNN training time is dominated by data stalls: time spent waiting for data to be fetched from(or written to) storage and pre-processed. We then describe CoorDL, a user-space data loading library to address data stalls in dedicated single-user servers with fixed resource capacities. Next, we design and evaluate Synergy, a work-load aware scheduler for shared GPU clusters that mitigates data stalls by allocating auxiliary resources like CPU and memory cognizant of workload requirements. Finally, we present CheckFreq, a framework that frequently writes model state to storage (checkpoint) for fault-tolerance, thereby reducing wasted GPU work on job interruptions, while also minimizing stalls due to checkpointing. Our dissertation shows that data stalls squander away the improved performance of faster GPUs. Our dissertation further demonstrates that an efficient data pipeline is critical to speeding up end-to-end training, by building and evaluating systems that mitigate data stalls in several training scenarios.","abstract_html":"Deep Learning, specifically Deep Neural Networks (DNNs), is stressing storage systems in new ways, moving the training bottleneck to the data pipeline (fetching, pre-processing data, and writing checkpoints), rather than computation at the GPUs; this leaves the expensive accelerator devices stalled for data. While prior research has explored different ways of accelerating DNN training time, the impact of storage systems, specifically the data pipeline, on ML training has been relatively unexplored. In this dissertation, we study the role of data pipeline in various training scenarios, and based on the insights from our study, we present the design and evaluation of systems that accelerate training. We first present a comprehensive analysis of how the storage subsystem affects the training of the widely used DNN models by building a tool, DS-Analyzer. Our study reveals that in many cases, DNN training time is dominated by data stalls: time spent waiting for data to be fetched from(or written to) storage and pre-processed. We then describe CoorDL, a user-space data loading library to address data stalls in dedicated single-user servers with fixed resource capacities. Next, we design and evaluate Synergy, a work-load aware scheduler for shared GPU clusters that mitigates data stalls by allocating auxiliary resources like CPU and memory cognizant of workload requirements. Finally, we present CheckFreq, a framework that frequently writes model state to storage (checkpoint) for fault-tolerance, thereby reducing wasted GPU work on job interruptions, while also minimizing stalls due to checkpointing. Our dissertation shows that data stalls squander away the improved performance of faster GPUs. Our dissertation further demonstrates that an efficient data pipeline is critical to speeding up end-to-end training, by building and evaluating systems that mitigate data stalls in several training scenarios.","abstract_has_math":false,"creators":["Mohan, Jayashree"],"institution":"The University of Texas at Austin","degree_name":"Doctor of Philosophy","degree_level":"Doctoral","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":[],"advisors":["Chidambaram, Vijay"],"committee_chairs":[],"committee_members":["Phanishayee, Amar","Witchel, Emmett","Rossbach, Christopher J","Krahenbuhl, Philipp"],"year":2021,"date_issued":"2021-12-01","date_published":"2021-12-01","updated_at":"2026-07-24T05:00:56Z","subjects":["Storage","System","Data stall","Deep learning","Training","Checkpoint","Data pipeline","Scheduling","Bottleneck","Cache"],"languages":["en"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://dx.doi.org/10.26153/tsw/42032"],"render_values":[{"text":"http://dx.doi.org/10.26153/tsw/42032","href":"http://dx.doi.org/10.26153/tsw/42032","code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/2152/115131","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Chidambaram, Vijay"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Phanishayee, Amar","Witchel, Emmett","Rossbach, Christopher J","Krahenbuhl, Philipp"]},{"key":"dc:creator","label":"Author","values":["Mohan, Jayashree"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-08-02T23:13:38Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-08-02T23:13:38Z"]},{"key":"dc:date.issued","label":"Date","values":["2021-12-01"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["The University of Texas at Austin"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Storage","System","Data stall","Deep learning","Training","Checkpoint","Data pipeline","Scheduling","Bottleneck","Cache"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/2152/115131","http://dx.doi.org/10.26153/tsw/42032"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Deep Learning, specifically Deep Neural Networks (DNNs), is stressing storage systems in new ways, moving the training bottleneck to the data pipeline (fetching, pre-processing data, and writing checkpoints), rather than computation at the GPUs; this leaves the expensive accelerator devices stalled for data. While prior research has explored different ways of accelerating DNN training time, the impact of storage systems, specifically the data pipeline, on ML training has been relatively unexplored. In this dissertation, we study the role of data pipeline in various training scenarios, and based on the insights from our study, we present the design and evaluation of systems that accelerate training. We first present a comprehensive analysis of how the storage subsystem affects the training of the widely used DNN models by building a tool, DS-Analyzer. Our study reveals that in many cases, DNN training time is dominated by data stalls: time spent waiting for data to be fetched from(or written to) storage and pre-processed. We then describe CoorDL, a user-space data loading library to address data stalls in dedicated single-user servers with fixed resource capacities. Next, we design and evaluate Synergy, a work-load aware scheduler for shared GPU clusters that mitigates data stalls by allocating auxiliary resources like CPU and memory cognizant of workload requirements. Finally, we present CheckFreq, a framework that frequently writes model state to storage (checkpoint) for fault-tolerance, thereby reducing wasted GPU work on job interruptions, while also minimizing stalls due to checkpointing. Our dissertation shows that data stalls squander away the improved performance of faster GPUs. Our dissertation further demonstrates that an efficient data pipeline is critical to speeding up end-to-end training, by building and evaluating systems that mitigate data stalls in several training scenarios."]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Accelerating deep learning training : a storage perspective"]}]}],"canonical_facts":{"dc:contributor.advisor":["Chidambaram, Vijay"],"dc:contributor.committeemember":["Phanishayee, Amar","Witchel, Emmett","Rossbach, Christopher J","Krahenbuhl, Philipp"],"dc:creator":["Mohan, Jayashree"],"dc:date.accessioned":["2022-08-02T23:13:38Z"],"dc:date.available":["2022-08-02T23:13:38Z"],"dc:date.issued":["2021-12-01"],"dc:description.abstract":["Deep Learning, specifically Deep Neural Networks (DNNs), is stressing storage systems in new ways, moving the training bottleneck to the data pipeline (fetching, pre-processing data, and writing checkpoints), rather than computation at the GPUs; this leaves the expensive accelerator devices stalled for data. While prior research has explored different ways of accelerating DNN training time, the impact of storage systems, specifically the data pipeline, on ML training has been relatively unexplored. In this dissertation, we study the role of data pipeline in various training scenarios, and based on the insights from our study, we present the design and evaluation of systems that accelerate training. We first present a comprehensive analysis of how the storage subsystem affects the training of the widely used DNN models by building a tool, DS-Analyzer. Our study reveals that in many cases, DNN training time is dominated by data stalls: time spent waiting for data to be fetched from(or written to) storage and pre-processed. We then describe CoorDL, a user-space data loading library to address data stalls in dedicated single-user servers with fixed resource capacities. Next, we design and evaluate Synergy, a work-load aware scheduler for shared GPU clusters that mitigates data stalls by allocating auxiliary resources like CPU and memory cognizant of workload requirements. Finally, we present CheckFreq, a framework that frequently writes model state to storage (checkpoint) for fault-tolerance, thereby reducing wasted GPU work on job interruptions, while also minimizing stalls due to checkpointing. Our dissertation shows that data stalls squander away the improved performance of faster GPUs. Our dissertation further demonstrates that an efficient data pipeline is critical to speeding up end-to-end training, by building and evaluating systems that mitigate data stalls in several training scenarios."],"dc:format.mimetype":["application/pdf"],"dc:identifier.uri":["https://hdl.handle.net/2152/115131","http://dx.doi.org/10.26153/tsw/42032"],"dc:language.iso":["en"],"dc:subject":["Storage","System","Data stall","Deep learning","Training","Checkpoint","Data pipeline","Scheduling","Bottleneck","Cache"],"dc:title":["Accelerating deep learning training : a storage perspective"],"dc:type":["Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["The University of Texas at Austin"]},"updated_at":"2026-07-24T05:00:56Z"}