QEM compression compared with HDF5, 2026-10-23

Historical six-acquisition evidence. The current five-acquisition selection excludes 96 × 96 detector inputs from future runs and active examples; the completed measurements below are retained unchanged.

For the six default acquisitions, ANS QEM saves 29.0% of the total HDF5 storage: 76.570 GB becomes 54.360 GB. Five acquisitions become 29.8–32.4% smaller; acquisition-06 becomes 25.5% larger. These are complete-file sizes for the exact original counts, not a claim that ANS always compresses better.

The subsequent index and CUDA profile now isolates the spatial-index bytes and measures storage-read/checksum intervals. Acquisition-06 still exceeds HDF5 without its spatial index; its ANS payload alone is larger than the HDF5 acquisition.

Measurement and lossless coverage

This analysis uses the completed six-acquisition comparison and private experiment 20261023-six-default-io. There are three backends and six acquisitions. No conversion or timing was rerun for this analysis. The measured source fingerprint is 2873ac0ffac23c49c89a7d5311ffbcc8836ead6af640acf6df10f5a292290df8, on base revision d2db0bdb4c9859230d3e9ba2f6b23fcfae8eb0f1.

HDF5 is a container. Every linked detector dataset in these six inputs uses bitshuffle plus LZ4, confirmed from filter ID 32008 and compression parameter 2. QEM saves the resident ANS streams together with decoding models, offsets, spatial indexes, metadata and block checksums. This compares these two complete storage layouts; it does not isolate the entropy coders from their indexes or container overhead.

HDF5 size includes the master and all unique linked detector files. QEM size is the entire generated file, measured before temporary-file cleanup. The table uses the MPS-produced QEM size consistently. Among the available CUDA, MPS and Swift producers, the maximum size spread for any acquisition is below 0.064%; exact per-producer byte counts are retained in the private audit. There is no native Swift-produced acquisition-06 because that HDF5 layout is unsupported; its Swift loading test used an MPS-produced QEM.

Original size means the logical tensor at uint16: all scan positions and detector pixels, including flagged pixels, with no correction, cropping or binning. Full comparisons covered 217,432,719,360 values across the six inputs, with zero differences for each completed conversion. The Swift workflow uses MPS for the exhaustive source comparison, as described in the timing report.

Compression results

GB means 1,000,000,000 bytes. Compression ratio is original uint16 bytes divided by complete stored-file bytes; a larger ratio means a smaller file. Space saved is 100 × (1 − QEM bytes / HDF5 bytes). Negative savings mean expansion. All ratios use exact byte counts before display rounding.

Acquisition · scan × detector	Original uint16 (GB)	HDF5 (GB)	QEM (GB)	HDF5 ratio	QEM ratio	Space saved vs HDF5
03 · 256² × 192²	4.832	0.717	0.489	6.74×	9.87×	31.8%
06 · 512² × 96²	4.832	1.367	1.715	3.54×	2.82×	−25.5%
01 · 512² × 192²	19.327	3.040	2.054	6.36×	9.41×	32.4%
04 · 512² × 192²	19.327	3.374	2.355	5.73×	8.21×	30.2%
08 · 1024² × 192²	77.309	13.476	9.413	5.74×	8.21×	30.2%
10 · 2048² × 192²	309.238	54.595	38.333	5.66×	8.07×	29.8%
Total	434.865	76.570	54.360	5.68×	8.00×	29.0%

For the 2048² input, QEM is 16.262 GB smaller. HDF5 is 1.42 times the QEM size: the 8.07× QEM compression ratio is relative to uncompressed uint16, not relative to HDF5. Across all six, the HDF5-to-QEM size ratio is 1.41×, weighted by actual bytes rather than an average of the six percentages.

Acquisition-06 retains less compactly in this QEM layout. Its 25.5% expansion is measured. The initial comparison did not isolate the components; the follow-up above measures the index contribution, while count statistics and model selection remain unisolated. The production converter declines this larger copy. Its reported QEM size comes from the disposable, fully verified benchmark retry, not a change to that production policy.

Why the loading gain differs by backend

For the 2048² input, repeated HDF5/QEM loading is 11.96/10.47 seconds on CUDA (1.14×), 12.11/2.20 seconds on MPS (5.49×), and 24.28/4.54 seconds on Swift/Metal (5.35×). These medians use trials two and three, with uncontrolled OS caches, and stop at encoded GPU residency. The machines and storage differ.

QEM avoids HDF5 decoding, ANS re-encoding and spatial-index construction, but still reads and authenticates approximately 38.3 GB for this acquisition. CUDA then transfers the encoded arrays into device memory. Its loader already overlaps bounded reads and SHA-256 checks with uploads using four 64 MiB pinned staging buffers. In the repeated HDF5 trials, the recorded ANS encoding time is about 1.20 seconds and index construction about 0.43 seconds. These stage counters do not independently partition overlapping wall time; they do show that skipping ANS encoding alone cannot explain an order-of-magnitude loading improvement.

The MPS loader reads and hashes blocks directly into shared Metal buffers using up to eight workers. Native Swift verifies a file mapping in parallel, then copies its arrays into shared Metal buffers. These different allocation, copying and transfer paths matter independently of compression ratio.

The original QEM load measurements do not separately time storage reads, SHA-256, allocation, host copies and device transfers. Those costs are confirmed by source inspection, but assigning the CUDA 10.47 seconds to one of them requires stage profiling under controlled cache conditions. The observed throughput alone does not establish a disk, checksum or PCIe limit. Normal QEM reload checks are distinct from the exhaustive source-count verification performed during conversion; that full comparison is not repeated inside each load timing. The follow-up above adds interval profiling and process storage-read counters without replacing these original observations.

Evidence

The private compression-audit.json retains source identities, every linked dataset’s filter metadata, exact source-file totals, per-producer QEM sizes, formulas and verification coverage. Source masters and current file totals were checked against the retained conversion records. Original timing observations and source data remain unchanged. This is documentation of existing verified artifacts, not a new codec optimization or release qualification.