Decoupling Cores, Kernels, and Operating Systems
📜 Abstract
We present Barrelfish/DC, an extension to the Barrelfish OS which decouples physical cores from a native OS kernel, and furthermore the kernel itself from the rest of the OS and application state. In Barrelfish/DC, native kernel code on any core can be quickly replaced, kernel state moved between cores, and cores added and removed from the system transparently to applications and OS processes, which continue to execute. Barrelfish/DC is a multikernel with two novel ideas: the use of boot drivers to abstract cores as regular devices, and a partitioned capability system for memory management which externalizes core-local kernel state. We show by performance measurements of real applications and device drivers that the approach is practical enough to be used for a number of purposes, such as online kernel upgrades, and temporarily delivering hard real-time performance by executing a process under a specialized, single-application kernel.
✨ Summary
Contributions and design
The paper presents Barrelfish/DC, an extension of the Barrelfish multikernel that treats physical cores, kernel instances, and per-core operating-system state as independently replaceable and movable entities. Its central abstraction is the OSnode, which contains the state associated with a core, including capabilities, process-control information, interrupt state, scheduling state, and timer information. A Kernel Control Block (KCB) provides the kernel with a compact, stable handle to the state that must survive kernel replacement.
Two mechanisms enable the design:
- Boot drivers represent cores similarly to peripheral devices and execute on another core. They encapsulate platform-specific operations for booting, suspending, resuming, and powering down target cores while leaving kernel-selection policy outside the kernel.
- Partitioned capabilities externalize core-local memory and resource state. Because the kernel does not directly own most persistent OS state, a kernel can be stopped and replaced, then reconstructed from the existing OSnode and KCB.
The system supports kernel replacement, OSnode migration between cores, temporary co-location of multiple OSnodes on one core, and deployment of specialized kernels. The authors demonstrate applications including online kernel updates, core reconfiguration, device-driver continuity, and hard-real-time execution using a dedicated kernel with no scheduler or external interrupts.
Results and limitations
On the evaluated x86 systems, stopping a core in Barrelfish/DC took approximately 0.8–3.5 microseconds and was largely unaffected by system load, whereas Linux 3.13 required approximately 46–131 milliseconds when idle and substantially longer under load. Core startup in Barrelfish/DC was dominated by kernel-image preparation and ranged from approximately 7–29 milliseconds when idle, comparable to Linux on the tested systems. Restarting a kernel caused only brief, application-specific disturbances for the Ethernet driver, web server, and PostgreSQL experiments. Parking an OSnode introduced expected latency and bandwidth variation because multiple OSnodes shared a core through time slicing.
A specialized Barrelfish/DC kernel eliminated the large execution-time outliers observed with ordinary Barrelfish and Linux configurations in the synthetic real-time benchmark. The approach nevertheless depends on maintaining a stable OSnode format, currently assumes cache-coherent cores and shared physical memory for practical migration, and leaves scheduling, power-management, migration policy, fault tolerance, and stronger isolation as open issues.
Influence on subsequent work
The paper’s ideas have been cited in later research on decentralized operating-system structures for manycore processors. For example, work on MedusaVM+ cites it alongside the multikernel literature when motivating decentralized virtual-memory systems for many-core machines. (sciencedirect.com) A later study of distributed operating systems for lightweight manycores also cites the paper when discussing distributed OS instances, OS-level abstractions, and reducing OS memory overhead. (sciencedirect.com) The available evidence indicates research influence primarily through the multikernel and dynamic-core-management design space; no concrete widespread industry deployment of Barrelfish/DC itself was identified in the search. The paper appeared in the OSDI 2014 proceedings and is listed as pages 17–31. (usenix.org)