The Google File System
📜 Abstract
We have designed and implemented the Google File System, a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients. While sharing many of the same goals as previous distributed file systems, our design has been driven by observations of our application workloads and technological environment, both current and anticipated, that reflect a marked departure from some earlier file system assumptions. This has led us to reexamine traditional choices and explore radically different design points. The file system has successfully met our storage needs. It is widely deployed within Google as the storage platform for the generation and processing of data used by our service as well as research and development efforts that require large data sets. The largest cluster to date provides hundreds of terabytes of storage across thousands of disks on over a thousand machines, and it is concurrently accessed by hundreds of clients. In this paper, we present file system interface extensions designed to support distributed applications, discuss many aspects of our design, and report measurements from both micro-benchmarks and real world use.
✨ Summary
Summary
The paper presents GFS, a distributed file system designed specifically for Google’s large-scale, data-intensive workloads. Its design assumes frequent failures, very large files, predominantly sequential reads, append-heavy writes, and a need for high sustained bandwidth rather than low individual-operation latency.
GFS uses a centralized master for namespace and chunk metadata, while clients transfer file data directly to replicated chunkservers. Files are divided into large, fixed-size chunks, normally replicated three ways across machines and racks. The master minimizes its involvement in data operations through client-side metadata caching and chunk leases; a primary replica serializes mutations and propagates them to secondary replicas. The system also provides atomic record append, copy-on-write snapshots, lazy garbage collection, checksums for detecting corruption, stale-replica detection, automatic re-replication, and fast recovery.
The consistency model is deliberately relaxed. Successful non-concurrent mutations produce defined regions, while concurrent writes may produce consistent but undefined regions. Record append guarantees that a record is atomically appended at least once, allowing applications to tolerate padding and duplicates through checksums and application-level identifiers. This model shifts some responsibility to applications but simplifies the file system and fits Google’s append-oriented workloads.
The measurements show that GFS achieved high aggregate throughput and that the master was not a bottleneck for the evaluated workloads. The principal limitations were write throughput for individual clients, network contention, and hotspots caused by heavily accessed small files. Operational experience further demonstrated the importance of checksumming, detailed diagnostic logs, automatic repair, and adapting the implementation to real hardware and operating-system behavior.
Influence
GFS directly informed Google’s subsequent distributed-data infrastructure. The MapReduce system used GFS for input and output storage, making the file system a foundational storage layer for large-scale data processing. (usenix.org) Bigtable stored its persistent tablet state and related files in GFS, using it as the storage substrate for a scalable structured-data system. (usenix.org)
The paper also influenced open-source infrastructure. HDFS was developed as an open-source system modeled after GFS and adopted several related design principles, including large blocks, centralized namespace management, replication, commodity hardware, and optimization for high-throughput processing of large datasets. (usenix.org)