{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/18383"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/18383","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Partial fault tolerance in stream processing applications - methods and evaluation techniques","abstract":"Stream processing emerged as a paradigm to continuously process incoming live data streams, such as audio, video, and business feeds. These applications are assembled as dataflow graphs, where each vertex of the graph is a stream operator and each edge is a stream connection. In this environment, a fault in a stream operator can result in massive data loss or in the generation of inaccurate results. Most of the fault tolerance solutions proposed for streaming applications aim at guaranteeing that no data is lost or that no data item is delivered to the application more than once. These techniques result in high performance overhead, given the need to coordinate the state stored in checkpoints of distributed components or maintain consistency between replicas. In this dissertation, we investigate partial fault tolerance methods, which protect only the most critical stream operators of a streaming application. These methods take advantage of the fact that stream processing algorithms are approximate by nature and, as a result, can still achieve acceptable results under data loss and duplicate data delivery. The methods proposed in this dissertation include a checkpoint-based mechanism and a partial graph replication technique. Both techniques were implemented in System S, IBM Research's stream processing middleware. In addition, this dissertation describes two different fault tolerance evaluation techniques. The first technique is based on fault injection and is used to emulate the effects of partial fault tolerance on a streaming application. With the fault injection results, the developers can understand the impact of faults on the application output and identify the most critical operators on their streaming application. The second evaluation technique is a model-based framework which provides generic abstractions for representing streaming applications with the stochastic activity network formalism. The framework allows the comparison of different fault tolerance techniques under varying fault models. Based on the results, the developers can evaluate the trade-offs that a certain technique provides when applied to their target application.","abstract_html":"Stream processing emerged as a paradigm to continuously process incoming live data streams, such as audio, video, and business feeds. These applications are assembled as dataflow graphs, where each vertex of the graph is a stream operator and each edge is a stream connection. In this environment, a fault in a stream operator can result in massive data loss or in the generation of inaccurate results. Most of the fault tolerance solutions proposed for streaming applications aim at guaranteeing that no data is lost or that no data item is delivered to the application more than once. These techniques result in high performance overhead, given the need to coordinate the state stored in checkpoints of distributed components or maintain consistency between replicas. In this dissertation, we investigate partial fault tolerance methods, which protect only the most critical stream operators of a streaming application. These methods take advantage of the fact that stream processing algorithms are approximate by nature and, as a result, can still achieve acceptable results under data loss and duplicate data delivery. The methods proposed in this dissertation include a checkpoint-based mechanism and a partial graph replication technique. Both techniques were implemented in System S, IBM Research's stream processing middleware. In addition, this dissertation describes two different fault tolerance evaluation techniques. The first technique is based on fault injection and is used to emulate the effects of partial fault tolerance on a streaming application. With the fault injection results, the developers can understand the impact of faults on the application output and identify the most critical operators on their streaming application. The second evaluation technique is a model-based framework which provides generic abstractions for representing streaming applications with the stochastic activity network formalism. The framework allows the comparison of different fault tolerance techniques under varying fault models. Based on the results, the developers can evaluate the trade-offs that a certain technique provides when applied to their target application.","abstract_has_math":false,"creators":["Jacques da Silva, Gabriela"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Iyer, Ravishankar K.","Lumetta, Steven S.","Nahrstedt, Klara","Sanders, William H.","Vasudevan, Shobha"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2011,"date_issued":"2011-01-14T22:48:28Z","date_published":"2011-01-14T22:48:28Z","updated_at":"2026-07-22T22:25:11Z","subjects":["fault tolerance","fault injection","stream processing","model-based evaluation","high-availability"],"languages":["en"],"rights":["Copyright 2010 Gabriela Jacques da Silva"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/18383","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Iyer, Ravishankar K.","Lumetta, Steven S.","Nahrstedt, Klara","Sanders, William H.","Vasudevan, Shobha"]},{"key":"dc:creator","label":"Author","values":["Jacques da Silva, Gabriela"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2011-01-14T22:48:28Z","2010-12"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["fault tolerance","fault injection","stream processing","model-based evaluation","high-availability"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2010 Gabriela Jacques da Silva"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/18383"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Stream processing emerged as a paradigm to continuously process incoming live data streams, such as audio, video, and business feeds. These applications are assembled as dataflow graphs, where each vertex of the graph is a stream operator and each edge is a stream connection. In this environment, a fault in a stream operator can result in massive data loss or in the generation of inaccurate results. Most of the fault tolerance solutions proposed for streaming applications aim at guaranteeing that no data is lost or that no data item is delivered to the application more than once. These techniques result in high performance overhead, given the need to coordinate the state stored in checkpoints of distributed components or maintain consistency between replicas. In this dissertation, we investigate partial fault tolerance methods, which protect only the most critical stream operators of a streaming application. These methods take advantage of the fact that stream processing algorithms are approximate by nature and, as a result, can still achieve acceptable results under data loss and duplicate data delivery. The methods proposed in this dissertation include a checkpoint-based mechanism and a partial graph replication technique. Both techniques were implemented in System S, IBM Research's stream processing middleware. In addition, this dissertation describes two different fault tolerance evaluation techniques. The first technique is based on fault injection and is used to emulate the effects of partial fault tolerance on a streaming application. With the fault injection results, the developers can understand the impact of faults on the application output and identify the most critical operators on their streaming application. The second evaluation technique is a model-based framework which provides generic abstractions for representing streaming applications with the stochastic activity network formalism. The framework allows the comparison of different fault tolerance techniques under varying fault models. Based on the results, the developers can evaluate the trade-offs that a certain technique provides when applied to their target application.","Item withdrawn by Mark Zulauf (zulauf@illinois.edu) on 2010-11-17T22:40:58Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 1 JacquesDaSilva_Gabriela.pdf: 2454261 bytes, checksum: 24c464ae689ee20b9762e6c6921dd252 (MD5)","Made available in DSpace on 2011-01-14T22:48:28Z (GMT). No. of bitstreams: 2 JacquesDaSilva_Gabriela.pdf: 2454261 bytes, checksum: 24c464ae689ee20b9762e6c6921dd252 (MD5) license.txt: 4066 bytes, checksum: 12a47b97904d9c68eb3af6c205188721 (MD5)"]},{"key":"dc:title","label":"Title","values":["Partial fault tolerance in stream processing applications - methods and evaluation techniques"]}]}],"canonical_facts":{"dc:contributor":["Iyer, Ravishankar K.","Lumetta, Steven S.","Nahrstedt, Klara","Sanders, William H.","Vasudevan, Shobha"],"dc:creator":["Jacques da Silva, Gabriela"],"dc:date":["2011-01-14T22:48:28Z","2010-12"],"dc:description":["Stream processing emerged as a paradigm to continuously process incoming live data streams, such as audio, video, and business feeds. These applications are assembled as dataflow graphs, where each vertex of the graph is a stream operator and each edge is a stream connection. In this environment, a fault in a stream operator can result in massive data loss or in the generation of inaccurate results. Most of the fault tolerance solutions proposed for streaming applications aim at guaranteeing that no data is lost or that no data item is delivered to the application more than once. These techniques result in high performance overhead, given the need to coordinate the state stored in checkpoints of distributed components or maintain consistency between replicas. In this dissertation, we investigate partial fault tolerance methods, which protect only the most critical stream operators of a streaming application. These methods take advantage of the fact that stream processing algorithms are approximate by nature and, as a result, can still achieve acceptable results under data loss and duplicate data delivery. The methods proposed in this dissertation include a checkpoint-based mechanism and a partial graph replication technique. Both techniques were implemented in System S, IBM Research's stream processing middleware. In addition, this dissertation describes two different fault tolerance evaluation techniques. The first technique is based on fault injection and is used to emulate the effects of partial fault tolerance on a streaming application. With the fault injection results, the developers can understand the impact of faults on the application output and identify the most critical operators on their streaming application. The second evaluation technique is a model-based framework which provides generic abstractions for representing streaming applications with the stochastic activity network formalism. The framework allows the comparison of different fault tolerance techniques under varying fault models. Based on the results, the developers can evaluate the trade-offs that a certain technique provides when applied to their target application.","Item withdrawn by Mark Zulauf (zulauf@illinois.edu) on 2010-11-17T22:40:58Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 1 JacquesDaSilva_Gabriela.pdf: 2454261 bytes, checksum: 24c464ae689ee20b9762e6c6921dd252 (MD5)","Made available in DSpace on 2011-01-14T22:48:28Z (GMT). No. of bitstreams: 2 JacquesDaSilva_Gabriela.pdf: 2454261 bytes, checksum: 24c464ae689ee20b9762e6c6921dd252 (MD5) license.txt: 4066 bytes, checksum: 12a47b97904d9c68eb3af6c205188721 (MD5)"],"dc:identifier":["http://hdl.handle.net/2142/18383"],"dc:language":["en"],"dc:rights":["Copyright 2010 Gabriela Jacques da Silva"],"dc:subject":["fault tolerance","fault injection","stream processing","model-based evaluation","high-availability"],"dc:title":["Partial fault tolerance in stream processing applications - methods and evaluation techniques"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:11Z"}