paper

VL2: A Scalable and Flexible Data Center Network

  • Authors:
Read the paper

📜 Abstract

To be agile and cost effective, data centers should allow dynamic resource allocation across large server pools. In particular, the data center network should enable any server to be assigned to any service. To meet these goals, we present VL2, a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics. VL2 uses (1) flat addressing to allow service instances to be placed anywhere in the network, (2) Valiant Load Balancing to spread traffic uniformly across network paths, and (3) end-system based address resolution to scale to large server pools, without introducing complexity to the network control plane. VL2’s design is driven by detailed measurements of traffic and fault data from a large operational cloud service provider. VL2’s implementation leverages proven network technologies, already available at low cost in high-speed hardware implementations, to build a scalable and reliable network architecture. As a result, VL2 networks can be deployed today, and we have built a working prototype. We evaluate the merits of the VL2 design using measurement, analysis, and experiments. Our VL2 prototype shuffles 2.7 TB of data among 75 servers in 395 seconds – sustaining a rate that is 94% of the maximum possible.

✨ Summary

Summary

VL2 addresses the need for agility in large cloud data centers: the ability to assign any server to any service as workloads and failures change. The paper argues that conventional hierarchical data-center networks impede this goal through oversubscription, limited path diversity, VLAN and address-space fragmentation, congestion, and weak utilization of otherwise available servers.

The proposed architecture combines several mechanisms:

  • A Clos topology built from commodity switch ASICs to provide extensive path diversity and high aggregate capacity.
  • Flat addressing that separates service identity from a server’s physical location, allowing services and virtual machines to move without requiring topology-dependent address changes.
  • Valiant Load Balancing, implemented with end-host path selection and ECMP, to distribute flows across available network paths without centralized traffic engineering.
  • An end-system directory and address-resolution mechanism that avoids placing large, rapidly changing host state in the network control plane.
  • A host-side layer-2.5 shim that preserves Ethernet layer-2 semantics and permits legacy applications to operate without modification.

The authors support the design with traffic and fault measurements from a production cloud-service data center. Their prototype demonstrates high throughput and isolation: it transfers 2.7 TB among 75 servers in 395 seconds, achieving 94% of the maximum possible rate, and reports a TCP fairness index of 0.995 in all-to-all shuffle experiments. The evaluation also indicates that random flow spreading provides effective utilization and that network churn has limited impact on TCP goodput.

The paper became an influential reference point for data-center networking. Microsoft later described VL2’s core principles—Clos-based scaling, randomized load-balanced routing, and separation of endpoint addresses from physical locations—as foundational to modern data-center networks, and VL2 received the ACM SIGCOMM Test of Time Paper Award in 2019. (microsoft.com) Subsequent data-center virtualization work, including IETF documents concerning NVGRE, Geneve, and network-virtualization overlays, cites VL2 as prior work on scalable layer-2 semantics, encapsulation, and host- or edge-based virtualization. (datatracker.ietf.org) A USENIX profile of coauthor Sudipta Sengupta states that VL2 was deployed in Microsoft’s cloud data centers, providing evidence of industry technology transfer. (usenix.org)