Dynamo: Amazon’s Highly Available Key-value Store
📜 Abstract
Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems. This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon’s core services use to provide an “always-on” experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.
✨ Summary
Summary
Dynamo presents a highly available, decentralized key-value store designed for Amazon services that require continuous reads and writes, low tail latency, incremental scaling, and replication across data centers. It deliberately favors availability over strong consistency and provides eventual consistency, allowing writes to succeed during temporary failures and network partitions.
The system combines consistent hashing and virtual nodes for partitioning, replication across a configurable number of nodes, vector clocks for tracking causal relationships between object versions, application-assisted conflict resolution, sloppy quorums, hinted handoff, Merkle-tree-based anti-entropy, read repair, and gossip-based membership management. Applications tune the replication and quorum parameters N, R, and W to balance availability, durability, consistency, latency, and cost. The paper also emphasizes production-oriented concerns such as tail latency at the 99.9th percentile, background-task admission control, load balancing, bootstrapping, and operational scalability.
In its reported production deployment, Dynamo supported Amazon services including shopping carts, session management, product catalogs, and customer preferences. The paper reports that 99.94% of profiled shopping-cart reads returned exactly one version, while divergent versions were uncommon. It also reports successful responses for 99.9995% of requests across the described deployments and no data-loss event during the stated operational period.
Influence
The paper became a foundational reference for eventually consistent and highly available NoSQL storage design. Apache Cassandra explicitly documents its use of Dynamo-style consistent hashing, multi-master replication, tunable consistency, gossip-based membership, failure detection, and incremental scale-out, while adapting the data model and storage engine. (cassandra.apache.org) Amazon later stated that DynamoDB was developed from Dynamo’s principles, combining Dynamo’s scalability and predictable low latency with a managed operational model. (aws.amazon.com) The paper’s bibliographic record identifies it as a 2007 SOSP publication by the nine listed authors. (cir.nii.ac.jp)