Deniz Altinbüken on Chain Replication (old and new) & Wes Chow (Mini) on Tiered Replication

New York - July 27, 2016

Deniz Altinbüken on Chain Replication (old and new) & a PWL-Mini with Wes Chow

Description

Chain Replication

Chain Replication (CR) is a variant of Primary-Backup Replication that supports high throughput and fast recovery from failures. CR has been widely used in both commercial systems and academic research prototypes. In so doing, various shortcomings of the original CR protocol have come to light. In this talk, I will summarize these findings and present a new version of CR that addresses the shortcomings. Our improved CR protocol supports different consistency guarantees, avoids the tail bottleneck for reads, and introduces autonomous reconfiguration of the system without requiring an external master. Additionally, we have developed a formal end-to-end specification of the protocol, including the actions of clients, detailing reconfiguration and linearizable execution of client requests. Through this specification, we are able to reason about the new protocol more precisely and implement the protocol effortlessly. Lastly, I will contrast our approach to the related work.

Tiered Replication

Tiered Replication, by Cidon et. al, explores the problem of effective data replication strategies first introduced in the Copysets paper, awarded 2013 Usenix ATC Best Student Paper. While Copysets introduced a randomized algorithm for solving NP Hard redundancy and load balancing constraints around placement of data in distributed filesystem, Tiered Replication proposes a greedy algorithm for solving the same problem and also adding the ability to bake in real world constraints such as rack awareness. Wes will summarize the problem Copysets proposed, show Tiered Replication’s solution, and examine a real world deployment of the algorithm at Chartbeat.

Bios

Deniz Altinbüken (@denizaltinbuken) is a Ph.D. candidate in Distributed Systems at Cornell University, working with Robbert van Renesse. Her interests are in distributed systems and the theory of distributed computing with a focus on building infrastructure services for large-scale distributed systems. She loves breaking distributed systems down to their basic components, understanding the bits and pieces of what makes them work and explaining these in precise ways with the objective of making complicated systems easier to understand and implement.

Wes Chow (@weschow) has a B.S. in Electrical Engineering & Computer Science from UC Berkeley. He spent eight years building technical infrastructure for high frequency trading shops. One day, he stared into his dark soul and realized he needed to move into the startup light. Thus S7 Labs sprang into being, and he led teams that built Storybox, a Seedcamp NY finalist, and Songza Radio, subsumed by Google Music. He's now at Chartbeat serving out his term as CTO.

Audio

Slides

TwoSigma The New York Chapter would like to thank TwoSigma for helping to make this meetup possible.