Leveraging Sharding in the Design of Scalable Replication Protocols
Robbert van Renesse
Cornell University
SWS Distinguished Lecture Series
Robbert van Renesse obtained his Ph.D. in 1989 at the Vrije Universiteit in Amsterdam, and subsequently worked on the Plan 9 operating system at AT&T Bell Labs. Since 1991 he and his students have been working at Cornell University on the theory and practice of scalable distributed systems. Van Renesse co-founded two companies in distributed systems technology.
Most if not all datacenter services use sharding and replication for scalability and reliability. Shards are more-or-less independent of one another and individually replicated. We challenge this design philosophy and present a replication protocol where the shards interact with one another: A protocol running within shards ensures linearizable consistency, while the shards interact in order to improve availability. We analyze its availability properties and evaluate a working implementation.