{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/101756"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/101756","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Fault injections on mission-critical computer systems","abstract":"This thesis presents two unique sets of fault injections on mission-critical computer systems with the goal of (1) understanding the impact of faults, errors and failures, and (2) evaluating fault-tolerance and resilience of the targeted systems in the presence of failures. Our first fault injection campaign studies the effects of failures on high-performance computing (HPC) systems. We target the Cray XE Blue Waters JYC testbed at the National Center for Supercomputing Applications, with the goal of improving the understanding of failure causes and propagation observed in the field failure data analysis of Blue Waters. We use data collected from system logs and network performance counters to (1) characterize fault-error-failure sequences and recovery mechanisms in Gemini interconnection networks and in Cray compute elements, (2) understand the impact of failures on the system and user applications at different scales, and (3) identify and recreate fault scenarios that induce unrecoverable failures, to create new tests for system and application design. We utilize HPCArrow, a newly developed software-implemented fault injection tool with the ability to disable and restore user-specified network links, directional connections, compute nodes and blades. We observe failures manifesting in the form of applications not making forward progress and network quiescence operations causing extended system recovery times. Our second fault injection campaign studies the effects of faults, attacks and failures on a smart power grid utilizing software-defined networking (SDN) to orchestrate its data acquisition network. We evaluate our fault models on a smart power grid simulation running Raincoat, an SDN application that reroutes and spoofs network traffic to thwart attackers. Additionally, we propose an application- and data plane-based solution to pro-actively monitor system state and enforce user defined policies. We show that under certain faults, (1) applications orchestrating the network become ineffective, and (2) periodically monitoring the state of the network can identify faults or attacks before they manifest as failures. The results obtained from this work can aid in enhancing the resiliency of future SDN applications.","abstract_html":"This thesis presents two unique sets of fault injections on mission-critical computer systems with the goal of (1) understanding the impact of faults, errors and failures, and (2) evaluating fault-tolerance and resilience of the targeted systems in the presence of failures. Our first fault injection campaign studies the effects of failures on high-performance computing (HPC) systems. We target the Cray XE Blue Waters JYC testbed at the National Center for Supercomputing Applications, with the goal of improving the understanding of failure causes and propagation observed in the field failure data analysis of Blue Waters. We use data collected from system logs and network performance counters to (1) characterize fault-error-failure sequences and recovery mechanisms in Gemini interconnection networks and in Cray compute elements, (2) understand the impact of failures on the system and user applications at different scales, and (3) identify and recreate fault scenarios that induce unrecoverable failures, to create new tests for system and application design. We utilize HPCArrow, a newly developed software-implemented fault injection tool with the ability to disable and restore user-specified network links, directional connections, compute nodes and blades. We observe failures manifesting in the form of applications not making forward progress and network quiescence operations causing extended system recovery times. Our second fault injection campaign studies the effects of faults, attacks and failures on a smart power grid utilizing software-defined networking (SDN) to orchestrate its data acquisition network. We evaluate our fault models on a smart power grid simulation running Raincoat, an SDN application that reroutes and spoofs network traffic to thwart attackers. Additionally, we propose an application- and data plane-based solution to pro-actively monitor system state and enforce user defined policies. We show that under certain faults, (1) applications orchestrating the network become ineffective, and (2) periodically monitoring the state of the network can identify faults or attacks before they manifest as failures. The results obtained from this work can aid in enhancing the resiliency of future SDN applications.","abstract_has_math":false,"creators":["Devnani, Lavin R."],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Iyer, Ravishankar K.","Kalbarczyk, Zbigniew T."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2018,"date_issued":"2018-09-27T16:45:31Z","date_published":"2018-09-27T16:45:31Z","updated_at":"2026-07-22T22:24:40Z","subjects":["Resiliency, Fault Injections, Mission Critical, Power Grid, High-performance Computing, Software-defined Networking, Reliability"],"languages":["en"],"rights":["Copyright 2018 Lavin R. Devnani"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/101756","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Iyer, Ravishankar K.","Kalbarczyk, Zbigniew T."]},{"key":"dc:creator","label":"Author","values":["Devnani, Lavin R."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2018-09-27T16:45:31Z","2020-09-28T09:15:13Z","2018-07-18","2018-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Resiliency, Fault Injections, Mission Critical, Power Grid, High-performance Computing, Software-defined Networking, Reliability"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2018 Lavin R. Devnani"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/101756"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This thesis presents two unique sets of fault injections on mission-critical computer systems with the goal of (1) understanding the impact of faults, errors and failures, and (2) evaluating fault-tolerance and resilience of the targeted systems in the presence of failures. Our first fault injection campaign studies the effects of failures on high-performance computing (HPC) systems. We target the Cray XE Blue Waters JYC testbed at the National Center for Supercomputing Applications, with the goal of improving the understanding of failure causes and propagation observed in the field failure data analysis of Blue Waters. We use data collected from system logs and network performance counters to (1) characterize fault-error-failure sequences and recovery mechanisms in Gemini interconnection networks and in Cray compute elements, (2) understand the impact of failures on the system and user applications at different scales, and (3) identify and recreate fault scenarios that induce unrecoverable failures, to create new tests for system and application design. We utilize HPCArrow, a newly developed software-implemented fault injection tool with the ability to disable and restore user-specified network links, directional connections, compute nodes and blades. We observe failures manifesting in the form of applications not making forward progress and network quiescence operations causing extended system recovery times. Our second fault injection campaign studies the effects of faults, attacks and failures on a smart power grid utilizing software-defined networking (SDN) to orchestrate its data acquisition network. We evaluate our fault models on a smart power grid simulation running Raincoat, an SDN application that reroutes and spoofs network traffic to thwart attackers. Additionally, we propose an application- and data plane-based solution to pro-actively monitor system state and enforce user defined policies. We show that under certain faults, (1) applications orchestrating the network become ineffective, and (2) periodically monitoring the state of the network can identify faults or attacks before they manifest as failures. The results obtained from this work can aid in enhancing the resiliency of future SDN applications.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2020-08-01","The student, Lavin Devnani, accepted the attached license on 2018-07-18 at 13:24.","The student, Lavin Devnani, submitted this Thesis for approval on 2018-07-18 at 13:28.","This Thesis was approved for publication on 2018-07-18 at 14:19.","DSpace SAF Submission Ingestion Package generated from Vireo submission #12626 on 2018-09-27 at 11:33:12","Made available in DSpace on 2018-09-27T16:45:31Z (GMT). No. of bitstreams: 2 DEVNANI-THESIS-2018.pdf: 4875875 bytes, checksum: 61c85eb93aa91fdef4b6320cdf64ac49 (MD5) LICENSE.txt: 4210 bytes, checksum: 55b1b14249cb050fa926429459bdd1a2 (MD5) Previous issue date: 2018-07-18","Embargo set by: Seth Robbins for item 107856 Lift date: 2020-09-27T16:45:39Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 107856 Lift date: 2020-09-27T16:47:41Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited Restriction Lifted for Item 107856 on 2020-09-28T09:15:13Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Fault injections on mission-critical computer systems"]}]}],"canonical_facts":{"dc:contributor":["Iyer, Ravishankar K.","Kalbarczyk, Zbigniew T."],"dc:creator":["Devnani, Lavin R."],"dc:date":["2018-09-27T16:45:31Z","2020-09-28T09:15:13Z","2018-07-18","2018-08"],"dc:description":["This thesis presents two unique sets of fault injections on mission-critical computer systems with the goal of (1) understanding the impact of faults, errors and failures, and (2) evaluating fault-tolerance and resilience of the targeted systems in the presence of failures. Our first fault injection campaign studies the effects of failures on high-performance computing (HPC) systems. We target the Cray XE Blue Waters JYC testbed at the National Center for Supercomputing Applications, with the goal of improving the understanding of failure causes and propagation observed in the field failure data analysis of Blue Waters. We use data collected from system logs and network performance counters to (1) characterize fault-error-failure sequences and recovery mechanisms in Gemini interconnection networks and in Cray compute elements, (2) understand the impact of failures on the system and user applications at different scales, and (3) identify and recreate fault scenarios that induce unrecoverable failures, to create new tests for system and application design. We utilize HPCArrow, a newly developed software-implemented fault injection tool with the ability to disable and restore user-specified network links, directional connections, compute nodes and blades. We observe failures manifesting in the form of applications not making forward progress and network quiescence operations causing extended system recovery times. Our second fault injection campaign studies the effects of faults, attacks and failures on a smart power grid utilizing software-defined networking (SDN) to orchestrate its data acquisition network. We evaluate our fault models on a smart power grid simulation running Raincoat, an SDN application that reroutes and spoofs network traffic to thwart attackers. Additionally, we propose an application- and data plane-based solution to pro-actively monitor system state and enforce user defined policies. We show that under certain faults, (1) applications orchestrating the network become ineffective, and (2) periodically monitoring the state of the network can identify faults or attacks before they manifest as failures. The results obtained from this work can aid in enhancing the resiliency of future SDN applications.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2020-08-01","The student, Lavin Devnani, accepted the attached license on 2018-07-18 at 13:24.","The student, Lavin Devnani, submitted this Thesis for approval on 2018-07-18 at 13:28.","This Thesis was approved for publication on 2018-07-18 at 14:19.","DSpace SAF Submission Ingestion Package generated from Vireo submission #12626 on 2018-09-27 at 11:33:12","Made available in DSpace on 2018-09-27T16:45:31Z (GMT). No. of bitstreams: 2 DEVNANI-THESIS-2018.pdf: 4875875 bytes, checksum: 61c85eb93aa91fdef4b6320cdf64ac49 (MD5) LICENSE.txt: 4210 bytes, checksum: 55b1b14249cb050fa926429459bdd1a2 (MD5) Previous issue date: 2018-07-18","Embargo set by: Seth Robbins for item 107856 Lift date: 2020-09-27T16:45:39Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 107856 Lift date: 2020-09-27T16:47:41Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited Restriction Lifted for Item 107856 on 2020-09-28T09:15:13Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/101756"],"dc:language":["en"],"dc:rights":["Copyright 2018 Lavin R. Devnani"],"dc:subject":["Resiliency, Fault Injections, Mission Critical, Power Grid, High-performance Computing, Software-defined Networking, Reliability"],"dc:title":["Fault injections on mission-critical computer systems"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:40Z"}