Embracing diversity in the Barrelfish manycore operating system
📜 Abstract
We discuss diversity and heterogeneity in manycore computer systems, and identify three distinct types of diversity, all of which present challenges to operating system designers and application writers alike. We observe that most current research work has concentrated on a narrow form of one of these (non-uniform memory access) to the exclusion of the others, and show with measurement why this makes sense in the short term. However, we claim that this is not viable in the long term given current processor and system roadmaps, and present our approach to dealing with both heterogeneous hardware within a single system, and the increasing diversity of complete system configurations: we directly represent detailed system information in an expressive “system knowledge base” accessible to applications and OS subsystems alike, and use this to control tasks such as scheduling and resource allocation.
✨ Summary
Summary
The paper argues that manycore operating systems must address three forms of hardware diversity: non-uniformity within a machine, diversity among processor cores, and diversity across complete system configurations. Its measurements on contemporary Intel and AMD systems show that memory-access latency can vary substantially with topology, while differences in interprocessor-interrupt and device-access latency were smaller. The authors therefore propose moving beyond abstractions that present hardware as a uniform SMP machine.
The central design proposal is the system knowledge base (SKB): a queryable representation of hardware topology, device characteristics, runtime measurements, and prior hardware knowledge. The SKB is intended to support operating-system policy and application-level decisions, including scheduling, resource allocation, device-buffer placement, and cache-aware thread placement. The design separates policy representations from performance-critical mechanism data structures, keeping SKB queries off the fast path while allowing the system to continue operating if the SKB is temporarily unavailable.
The paper’s subsequent influence is documented primarily within the Barrelfish research project. Later Barrelfish work incorporated the system knowledge base into end-to-end multicore scheduling and used it for tasks including multicast routing, device configuration, interrupt routing, and scheduling. (barrelfish.org) The SKB also supported follow-on work on database–operating-system co-design, adaptive scheduling, and performance discovery on Barrelfish. (research-collection.ethz.ch) The broader Barrelfish project subsequently developed the multikernel architecture, which treated multicore hardware as a distributed system and became a reference point for later multicore operating-system research. (barrelfish.org)