Megastore: Providing Scalable, Highly Available Storage for Interactive Services
đź“ś Abstract
Megastore blends the scalability of a NoSQL datastore with the convenience of a traditional RDBMS. It provides fully serializable ACID semantics across wide areas, even when partitions happen. Tables are divided into partitions that are replicated across data centers, thus allowing reads to be served locally but ensuring strong consistency through a synchronization protocol on writes. Our system has been in production at Google on a large scale for years.
✨ Summary
The paper titled “Megastore: Providing Scalable, Highly Available Storage for Interactive Services” discusses the design and implementation of Megastore, a storage solution by Google that combines the scalability of NoSQL with the transactional consistency of relational databases. The authors present how Megastore achieves high availability and geographic data replication while maintaining strong ACID semantics.
Through a careful mix of partitioning and replication strategies across data centers, Megastore serves the demanding requirements of Google’s interactive services. It allows local read access while ensuring global consistency via a synchronization protocol for writes. At the time of publication, Megastore was already operational at Google at a significant scale.
The paper has influenced subsequent research and industrial developments in scalable and geo-replicated storage systems. It is referenced by several research articles discussing distributed databases and cloud computing systems. “Megastore” inspired other cloud-based services like Google Spanner. For example, in the paper titled “Spanner: Google’s Globally-Distributed Database,” published by the 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI), the authors mention how “Megastore” was a precursor to Spanner (source). The impact of Megastore extends to several other cloud-based storage systems exploring similar consistency and scaling challenges.