Fixed-Rate Compressed Floating-Point Arrays
📜 Abstract
Current compression schemes for floating-point data commonly take fixed-precision values and compress them to a variable-length bit stream, complicating memory management and random access. We present a fixed-rate, near-lossless compression scheme that maps small blocks of 4^d values in d dimensions to a fixed, user-specified number of bits per block, thereby allowing read and write random access to compressed floating-point data at block granularity. Our approach is inspired by fixed-rate texture compression methods widely adopted in graphics hardware, but has been tailored to the high dynamic range and precision demands of scientific applications. Our compressor is based on a new, lifted, orthogonal block transform and embedded coding, allowing each per-block bit stream to be truncated at any point if desired, thus facilitating bit rate selection using a single compression scheme. To avoid compression or decompression upon every data access, we employ a software write-back cache of uncompressed blocks. Our compressor has been designed with computational simplicity and speed in mind to allow for the possibility of a hardware implementation, and uses only a small number of fixed-point arithmetic operations per compressed value. We demonstrate the viability and benefits of lossy compression in several applications, including visualization, quantitative data analysis, and numerical simulation.
✨ Summary
Contributions
The paper introduces a fixed-rate, near-lossless representation for multidimensional floating-point arrays. Data are divided into independent 4 × 4 × 4 blocks, with each block assigned an exact, user-selected number of bits. This design enables block-granularity random reads and writes and avoids the memory-management difficulties of variable-length streams.
The compression pipeline aligns values in a block to a common exponent, converts them to fixed-point form, applies a separable orthogonal transform, orders coefficients by sequency, and encodes them using embedded bit-plane coding. The transform is implemented through lifting steps, emphasizing integer arithmetic, bit shifts, low computational cost, and possible hardware implementation. A write-back cache of decompressed blocks reduces repeated compression and decompression during localized access and update patterns.
Experiments on scientific simulation fields show that the method—later known as ZFP—can provide substantially higher compression than lossless floating-point methods, while retaining useful absolute and relative accuracy. The reported applications include volume rendering, Fourier-spectrum analysis, feature segmentation, streamline visualization, and an evolving Sedov blast-wave simulation. In the presented tests, 4 bits per double often produced visually or analytically acceptable results, while the blast-wave simulation retained close agreement with the uncompressed result at 16 bits per double and remained usable at lower rates.
Limitations
The approach is designed primarily for regularly gridded, spatially correlated data. It does not naturally provide a strict maximum-error bound in its fixed-rate form, and noisy or unstructured data may compress less effectively. The paper also identifies thread safety, cache design, block layout, time-varying data, and hardware optimization as areas requiring further work.
Influence
The paper directly established the algorithmic foundation for the open-source ZFP library. Current ZFP documentation states that the library is based on this work, while also noting that later versions added functionality such as support for integer data, additional array dimensions, and multiple execution backends. (github.com)
Subsequent research developed formal error analyses for ZFP and examined its behavior in iterative and time-stepping numerical methods. The later error-analysis work explicitly identifies this paper as the original description of ZFP, and subsequent stability research extends that analysis to repeated compression and decompression during computation. (arxiv.org)
The available evidence therefore shows concrete influence on scientific-computing software and follow-on numerical-analysis research. No specific commercial-industry deployment is established by the sources reviewed.