{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/138975"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/138975","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Optimizing Graph Neural Network Training on Large Graphs","abstract":"Graphs can be used to represent many important classes of structured real-world data. For this reason, there has been an increase of research interest in various machine learning approaches to solve tasks such as link prediction and node property prediction. Graph Neural Network models demonstrate good performance on such tasks. However, the depth of the models and the size of the graphs they can be trained on is constrained either by the low processing throughput of CPUs or by the limited memory capacity of GPUs. Techniques such as neighborhood sampling are often used to create smaller mini-batch training examples that fit in GPU memory. In this thesis, I provide a systematic performance analysis of GNN training codes written using PyTorch Geometric, the most popular machine learning framework for GNNs. Through this performance analysis, I uncover significant performance bottlenecks related to neighborhood sampling and GPU data transfers. To address these issues, I create FastPyG: a performance-engineered fork of PyTorch Geometric, which achieves a 3-6× speedup over comparable PyTorch Geometric codes without impacting model accuracy. The core contribution included in FastPyG is fast_sampler, an efficient and parallel neighborhood sampling implementation in C++.","abstract_html":"Graphs can be used to represent many important classes of structured real-world data. For this reason, there has been an increase of research interest in various machine learning approaches to solve tasks such as link prediction and node property prediction. Graph Neural Network models demonstrate good performance on such tasks. However, the depth of the models and the size of the graphs they can be trained on is constrained either by the low processing throughput of CPUs or by the limited memory capacity of GPUs. Techniques such as neighborhood sampling are often used to create smaller mini-batch training examples that fit in GPU memory. In this thesis, I provide a systematic performance analysis of GNN training codes written using PyTorch Geometric, the most popular machine learning framework for GNNs. Through this performance analysis, I uncover significant performance bottlenecks related to neighborhood sampling and GPU data transfers. To address these issues, I create FastPyG: a performance-engineered fork of PyTorch Geometric, which achieves a 3-6× speedup over comparable PyTorch Geometric codes without impacting model accuracy. The core contribution included in FastPyG is fast_sampler, an efficient and parallel neighborhood sampling implementation in C++.","abstract_has_math":false,"creators":["Stathas, Nickolas"],"institution":"Massachusetts Institute of Technology","degree_name":"Master","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Leiserson, Charles E.","Kaler, Tim"],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-06","date_published":"2021-06","updated_at":"2026-07-22T22:20:48Z","subjects":[],"languages":[],"rights":["In Copyright - Educational Use Permitted","Copyright MIT"],"rights_urls":["http://rightsstatements.org/page/InC-EDU/1.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/138975","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Leiserson, Charles E.","Kaler, Tim"]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"]},{"key":"dc:creator","label":"Author","values":["Stathas, Nickolas"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-01-14T14:42:12Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-01-14T14:42:12Z"]},{"key":"dc:date.issued","label":"Date","values":["2021-06"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master","Master of Engineering in Electrical Engineering and Computer Science"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright - Educational Use Permitted","Copyright MIT"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://rightsstatements.org/page/InC-EDU/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/138975"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Graphs can be used to represent many important classes of structured real-world data. For this reason, there has been an increase of research interest in various machine learning approaches to solve tasks such as link prediction and node property prediction. Graph Neural Network models demonstrate good performance on such tasks. However, the depth of the models and the size of the graphs they can be trained on is constrained either by the low processing throughput of CPUs or by the limited memory capacity of GPUs. Techniques such as neighborhood sampling are often used to create smaller mini-batch training examples that fit in GPU memory. In this thesis, I provide a systematic performance analysis of GNN training codes written using PyTorch Geometric, the most popular machine learning framework for GNNs. Through this performance analysis, I uncover significant performance bottlenecks related to neighborhood sampling and GPU data transfers. To address these issues, I create FastPyG: a performance-engineered fork of PyTorch Geometric, which achieves a 3-6× speedup over comparable PyTorch Geometric codes without impacting model accuracy. The core contribution included in FastPyG is fast_sampler, an efficient and parallel neighborhood sampling implementation in C++."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["M.Eng."]},{"key":"dc:title","label":"Title","values":["Optimizing Graph Neural Network Training on Large Graphs"]}]}],"canonical_facts":{"dc:contributor.advisor":["Leiserson, Charles E.","Kaler, Tim"],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"],"dc:creator":["Stathas, Nickolas"],"dc:date.accessioned":["2022-01-14T14:42:12Z"],"dc:date.available":["2022-01-14T14:42:12Z"],"dc:date.issued":["2021-06"],"dc:description.abstract":["Graphs can be used to represent many important classes of structured real-world data. For this reason, there has been an increase of research interest in various machine learning approaches to solve tasks such as link prediction and node property prediction. Graph Neural Network models demonstrate good performance on such tasks. However, the depth of the models and the size of the graphs they can be trained on is constrained either by the low processing throughput of CPUs or by the limited memory capacity of GPUs. Techniques such as neighborhood sampling are often used to create smaller mini-batch training examples that fit in GPU memory. In this thesis, I provide a systematic performance analysis of GNN training codes written using PyTorch Geometric, the most popular machine learning framework for GNNs. Through this performance analysis, I uncover significant performance bottlenecks related to neighborhood sampling and GPU data transfers. To address these issues, I create FastPyG: a performance-engineered fork of PyTorch Geometric, which achieves a 3-6× speedup over comparable PyTorch Geometric codes without impacting model accuracy. The core contribution included in FastPyG is fast_sampler, an efficient and parallel neighborhood sampling implementation in C++."],"dc:description.degree":["M.Eng."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/138975"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["In Copyright - Educational Use Permitted","Copyright MIT"],"dc:rights.uri":["http://rightsstatements.org/page/InC-EDU/1.0/"],"dc:title":["Optimizing Graph Neural Network Training on Large Graphs"],"dc:type":["Thesis"],"thesis:degree_name":["Master","Master of Engineering in Electrical Engineering and Computer Science"]},"updated_at":"2026-07-22T22:20:48Z"}