IX: A Protected Dataplane Operating System for High Throughput and Low Latency

Abstract

📜 Abstract

The conventional wisdom is that aggressive networking requirements, such as high packet rates for small messages and microsecond-scale tail latency, are best addressed outside the kernel, in a user-level networking stack. We present IX, a dataplane operating system that provides high I/O performance, while maintaining the key advantage of strong protection offered by existing kernels. IX uses hardware virtualization to separate management and scheduling functions of the kernel (control plane) from network processing (dataplane). The dataplane architecture builds upon a native, zero-copy API and optimizes for both bandwidth and latency by dedicating hardware threads and networking queues to dataplane instances, processing bounded batches of packets to completion, and by eliminating coherence traffic and multi-core synchronization. We demonstrate that IX outperforms Linux and state-of-the-art, user-space network stacks significantly in both throughput and end-to-end latency. Moreover, IX improves the throughput of a widely deployed, key-value store by up to 3.6× and reduces tail latency by more than 2×.

Description

✨ Summary

IX proposes a protected dataplane operating system for datacenter workloads that require both very high packet rates and microsecond-scale tail latency. Its central architectural separation places conventional management, resource allocation, and compatibility services in a Linux-based control plane, while dedicated dataplane instances execute network processing and application logic on assigned hardware threads and NIC queues. Hardware virtualization provides isolation among the control plane, dataplanes, and application code.

The dataplane combines four main techniques: run-to-completion processing, bounded adaptive batching, a native asynchronous zero-copy API, and flow-consistent assignment of traffic to queues and cores. These choices reduce intermediate buffering, kernel-transition overhead, cache inefficiency, synchronization, and cache-coherence traffic. The API exposes network flow-control state directly to applications and supports batched operations through shared-memory request and event arrays.

In the authors’ evaluation on Intel 10-GbE hardware, IX achieved 5.7 μs one-way latency for 64-byte messages, saturated a 10-GbE link with three cores, reached 8.8 million 64-byte round trips per second in the tested configuration, and scaled to four bonded 10-GbE interfaces. It also sustained useful performance with up to 250,000 concurrent connections. A memcached port improved throughput by up to 3.6× at a 500 μs 99th-percentile latency target, while reducing unloaded latency by approximately half. The prototype’s limitations included reliance on trusted dataplane code, lack of IOMMU and SR-IOV support, static rather than dynamic resource allocation in the evaluation, and the requirement that latency-sensitive application threads avoid long-running or blocking work.

The work was subsequently developed into a broader IX operating-system study published in ACM Transactions on Computer Systems, adding dynamic control-plane resource and energy-management experiments and reporting larger memcached gains in the extended system. (mast.stanford.edu) The authors also released an open-source IX implementation, making the dataplane architecture and supporting code available for further experimentation. (github.com) Later systems research continued to investigate related combinations of dedicated data paths, zero-copy communication, kernel protection, and low-latency networking; for example, StackMap explicitly presents a protected OS network service with a zero-copy data path and cites IX as relevant prior work. (usenix.org) Bibliographic aggregation records substantial subsequent scholarly citation of the paper, although citation counts vary by database and should not be interpreted as a direct measure of practical deployment. (rankless.org)