Live Migration of Virtual Machines
📜 Abstract
Migrating operating system instances across distinct physical hosts is a useful tool for administrators of data centers and clusters: It allows a clean separation between hardware and software, and facilitates fault management, load balancing, and low-level system maintenance. By carrying out the majority of migration while OSes continue to run, we achieve impressive performance with minimal service downtimes; we demonstrate the migration of entire OS instances on a commodity cluster, recording service downtimes as low as 60ms. We show that that our performance is sufficient to make live migration a practical tool even for servers running interactive loads. In this paper we consider the design options for migrating OSes running services with liveness constraints, focusing on data center and cluster environments. We introduce and analyze the concept of writable working set, and present the design, implementation and evaluation of high-performance OS migration built on top of the Xen VMM.
✨ Summary
Summary
The paper presents a live migration mechanism for Xen that moves a running operating-system instance between physical hosts while keeping service interruption brief. Its central technique is iterative pre-copy: memory pages are copied while the virtual machine continues executing, modified pages are recopied in later rounds, and the VM is paused only briefly to transfer the remaining state and CPU registers. The authors introduce the writable working set (WWS) as a way to characterize frequently modified pages that limit the effectiveness of pre-copy migration.
The system combines dirty-page tracking through Xen shadow page tables, adaptive migration bandwidth, transactional failure handling, and network redirection using unsolicited ARP replies. It assumes a well-connected cluster and network-attached storage, while leaving wide-area migration and local-disk migration as future work. In the evaluation, an 800 MB SPECweb99 VM migrated in 71 seconds with 210 ms of downtime, while a 64 MB Quake 3 server experienced 60 ms of downtime. A synthetic workload that dirtied memory faster than the network could transfer it produced 3.5 seconds of downtime, demonstrating a fundamental limitation of pre-copy migration.
Influence
The paper helped establish iterative pre-copy migration, dirty-page tracking, workload-dependent writable sets, and bandwidth/downtime trade-offs as standard topics in virtual-machine migration research. Later work explicitly treats pre-copy as the traditional baseline when developing post-copy migration, and subsequent surveys identify pre-copy as a commonly used approach across Xen, VMware, KVM, Hyper-V, and other virtualization platforms. (cir.nii.ac.jp)
Research following this work expanded the problem space to post-copy migration, hybrid pre/post-copy schemes, migration-cost modeling, downtime prediction, and network-contention management. These directions directly address limitations identified by the paper, especially repeated copying of hot pages, unpredictable migration duration, and competition between migration traffic and application traffic. (sciencedirect.com)
The operational goals demonstrated in the paper—moving running VMs for maintenance, load balancing, host evacuation, and resource management—are reflected in current virtualization platforms. Xen documents live migration as a supported mechanism for moving running VMs between physical servers, while Microsoft Hyper-V and Google Compute Engine document live migration for maintenance, balancing, and infrastructure management. These deployments show that live migration became an established industry capability, although the available sources do not by themselves establish that each implementation directly derives from this paper. (xenbits.xen.org)