ZooKeeper: Wait-free coordination for Internet-scale systems
📜 Abstract
In this paper, we describe ZooKeeper, a service for coordinating processes of distributed applications. Since ZooKeeper is part of critical infrastructure, ZooKeeper aims to provide a simple and high performance kernel for building more complex coordination primitives at the client. It incorporates elements from group messaging, shared registers, and distributed lock services in a replicated, centralized service. The interface exposed by ZooKeeper has the wait-free aspects of shared registers with an event-driven mechanism similar to cache invalidations of distributed file systems to provide a simple, yet powerful coordination service. The ZooKeeper interface enables a high-performance service implementation. In addition to the wait-free property, ZooKeeper provides a per client guarantee of FIFO execution of requests and linearizability for all requests that change the ZooKeeper state. These design decisions enable the implementation of a high performance processing pipeline with read requests being satisfied by local servers. We show for the target workloads, 2:1 to 100:1 read to write ratio, that ZooKeeper can handle tens to hundreds of thousands of transactions per second. This performance allows ZooKeeper to be used extensively by client applications.
✨ Summary
Summary
The paper presents ZooKeeper as a replicated coordination kernel for distributed applications. Rather than implementing locks, leader election, group membership, barriers, or configuration services directly in the server, ZooKeeper exposes a small hierarchical data model and client API from which these higher-level primitives can be constructed.
Its principal abstractions are regular, ephemeral, and sequential znodes; conditional updates using version numbers; sessions; and one-time watches for event-driven notification. Ephemeral znodes support failure-sensitive membership and lock recipes, while sequential znodes provide ordering for contention-management algorithms. Watches allow clients to react to changes without polling and avoid blocking the server on slow clients.
The consistency model deliberately separates reads from updates. State-changing requests are totally ordered and linearizable, with FIFO ordering for requests from each client. Reads are served locally by replicas and may be stale; clients can use sync when they require a read to reflect preceding updates. This design enables high read throughput while preserving the ordering properties needed to implement coordination protocols.
The service uses a leader-based atomic broadcast protocol, replicated in-memory state, write-ahead logging, and periodic snapshots. It remains available when a majority of servers are functioning and preserves acknowledged updates across failures provided that a quorum can eventually recover. The evaluation reports particularly strong performance for read-dominant workloads, reaching hundreds of thousands of operations per second in the tested configurations, while write throughput is lower because writes require agreement and durable logging.
Influence and adoption
The design became the foundation of the Apache ZooKeeper project, which describes ZooKeeper as an open-source coordination service for naming, configuration management, synchronization, and group services. Apache’s documentation also preserves the paper’s client-side recipe model for queues, locks, barriers, group membership, and leader election. (zookeeper.apache.org)
Apache identifies ZooKeeper as a coordination component used by distributed systems including Kafka, Hadoop, HBase, Flink, Pulsar, Solr, and Spark. Documented uses include leader election, failure detection, service discovery, configuration propagation, metadata storage, and cluster coordination. These deployments demonstrate the paper’s practical influence on widely used data-processing, messaging, search, and stream-processing systems. (zookeeper.apache.org)
The paper was published in the 2010 USENIX Annual Technical Conference and is catalogued with the four authors listed above. (usenix.org)