paper

Flexible Operating System Internals: The Design and Implementation of the Anykernel and Rump Kernels

  • Authors:

📜 Abstract

The monolithic kernel architecture is significant in the real world due to the large amount of working and proven code. However, the architecture is not without problems: testing and development is difficult, virtualizing kernel services can be done only by duplicating the entire kernel, and security is weak due to a single domain where all code has direct access to everything. Alternate kernel architectures such as the microkernel and exokernel have been proposed to rectify these problems with monolithic kernels. However, alternate system structures do not address the common desire of using a monolithic kernel when the abovementioned problems do not apply. We propose a flexible anykernel architecture which enables running kernel drivers in a variety of configurations, examples of which include microkernel-style servers and application libraries. A monolithic kernel is shown to be convertible into an anykernel with a reasonable amount of effort and without introducing performance-hindering indirection layers. The original monolithic mode of operation is preserved after the anykernel adjustments, and alternate modes of operation for drivers are available as a runtime choice. For non-monolithic modes of operation, the rump kernel is introduced as a lightweight container for drivers. A rump kernel runs on top of a hypervisor which offers high-level primitives such as thread scheduling and virtual memory. A production quality implementation for the NetBSD open source OS has been done. The anykernel architecture and rump kernels are evaluated both against four years of real-world experience from daily NetBSD development as well as against synthetic benchmarks.

✨ Summary

Summary

The dissertation proposes the anykernel architecture, in which kernel drivers are structured so that the same largely unmodified code can operate in multiple execution models: as part of a conventional monolithic kernel, as an application library, or as an isolated microkernel-style or remote service. Its implementation vehicle is the rump kernel, a lightweight virtualized kernel instance containing only the support layers and driver components required for a particular use case.

The design partitions NetBSD kernel functionality into a minimally dependent base, orthogonal subsystem layers called factions, and drivers. Support not essential to the driver—such as memory management, host scheduling, and I/O primitives—is delegated to the host through a hypercall interface. The implementation avoids pervasive indirection by using source-code restructuring, function pointers, weak symbols, generated interfaces, and symbol-namespace rewriting. It also provides local, microkernel, and remote client models, enabling direct library calls, host-kernel-mediated services, and system-call-style access over local or network sockets.

The evaluation demonstrates three principal benefits. First, rump kernels isolate potentially faulty drivers, particularly file-system drivers processing untrusted disk images, while preserving ordinary user-facing file-system interfaces. Second, they enable reuse of kernel implementations in applications such as file-system utilities and makefs, substantially reducing duplicate implementation effort. Third, they support rapid, isolated kernel testing and development: NetBSD used rump kernels in close to 1,000 test cases, allowing crashes to terminate individual test processes rather than the test host. The reported implementation typically bootstrapped in approximately 10 milliseconds with roughly 700 kB–1.5 MB of memory overhead, although costs varied by configuration. Local system calls were faster than native system calls in the reported experiments, while remote calls were dominated by IPC and copy-in/copy-out overhead.

The work’s principal limitation is that rump kernels do not provide a complete independent virtual-memory environment: page faults, memory protection, and much scheduling are delegated to the host. Remote clients also face ABI and data-structure compatibility constraints. Consequently, the architecture is best understood as selective driver virtualization rather than full operating-system virtualization.

Subsequent influence and use

Concrete evidence of subsequent uptake includes continued use of rump kernels in NetBSD’s automated testing infrastructure. The NetBSD project identifies rump kernels as backends for continuous tests, directly connecting the dissertation’s testing model to ongoing project practice. (wiki.netbsd.org)

The rumpkernel project describes the dissertation as the technical reference for the operating principles and terminology of rump kernels, indicating that the work became foundational documentation for the surrounding software ecosystem. (rumpkernel.github.io)

The concepts were subsequently applied beyond the original NetBSD testing and file-system use cases. A Xen Project account describes the development of Rumprun, a unikernel environment built around reusable rump-kernel components and the anykernel approach. (xenproject.org) The NetBSD TCP/IP stack was also integrated with DPDK, demonstrating use of rump-derived networking components with a high-performance packet-processing framework. (dpdk.org)

Later systems research has cited the dissertation in work on flexible OS isolation and related componentized or library-based operating-system designs, including FlexOS. This establishes scholarly influence through citation, although the cited sources do not by themselves demonstrate that the dissertation was the sole origin of those later systems. (owl.eu.com)

The bibliographic metadata and abstract were cross-checked against Aalto University’s research portal, which identifies the work as a 2012 doctoral thesis by Antti Kantee. (research.aalto.fi)