Segment backup: datacenter-disaster tolerance for stream processing systems

We design a datacenter-disaster tolerance solution called segment backup for stream processing systems. During regular running, segment backup inserts barriers into the normal tuple stream to indicate the backup versions and ensure node-level synchronization of multiple input streams. Additionally, segment backup materializes the partial results at some intermediate nodes to reduce reprocessing work after failures. The simple restarting logic can construct global consistency by connecting the segment consistency with the help of the materialized partial results. This method gives each segment flexibility to roll back to its latest version. We implement a prototype stream processing system with segment backup. Our experimental results show that segment backup can shorten the recovery delay with acceptable overhead during normal execution.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2017

Author dc:creator

• Shen, Juanli

Contributors dc:contributor

• Winslett, Marianne Southall

• Stream processing systems
• Fault tolerance

Statement dc:rights

• Copyright 2017 Juanli Shen

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/97603

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/97603