zxc

Release Notes

v0.14.1 is a hardening release: every decode path got stricter, block decompression is 10-13 % faster, reusable contexts reach all five wrappers, and seekable archives can verify per-block checksums.

Drop-in for v0.14.0. Same format v8, byte-identical output at every level (verified on Silesia), soname still libzxc.so.4. One symbol added, zxc_seekable_set_checksum; none removed or changed.

Decoder hardening

• A forged block size passed off a truncated decode as success (#428). A payload size spanning the EOF marker made the frame walk run off the archive and return what it had, with checksums off (the default). A data block's payload can no longer exceed the header's block size, the walk must end on the EOF block, and a seek-table entry spans at most one block. The check sits in the block decoder shared by every reader. Both rules are now normative in FORMAT.md.
• Stack address stored in heap memory (#429). Multi-threaded seekable jobs held the caller's output pointer. This was harmless, since workers are joined before return, but a scanner can't prove that. Jobs now carry offsets from one base pointer on the caller's stack.
• zxc_stream_decompress ignored the dictionary size limit (#430). A dictionary above ZXC_DICT_SIZE_MAX was accepted, or rejected with a misleading ZXC_ERROR_DICT_MISMATCH. The check moved into the shared stream engine.
• Contexts (#417, #422). Static contexts reject oversized blocks up front. An empty input through a context now yields the canonical 36-byte archive, like the one-shot API.

API

• Reusable contexts in every wrapper (#424, #414, #415). Cctx and Dctx now ship in Rust, Go, Python, Node.js and WASM. A dictionary given to the factory applies to every call that doesn't override it. The block API also takes a dictionary's shared literal Huffman table (#415), so small blocks no longer spend bytes describing their own.
• Seekable checksum verification (#425). zxc_seekable_decompress_range can verify per-block checksums through zxc_seekable_set_checksum. It is off by default and exposed in all five wrappers. Also fixed: an overflowing offset plus length slipped past the bounds check, and an incomplete range returned a silent partial read instead of ZXC_ERROR_CORRUPT_DATA.

CLI

• -l and -t read checksum status from the HAS_CHECKSUM header flag (#435). -j adds checksum_method; exit codes are unchanged. Thanks to @Vladexy88x for the fix.

Performance

• zxc_decompress_block writes straight to the destination (#418). Given the zxc_decompress_block_bound() capacity, it skips the bounce-buffer copy and runs 10-13 % faster, from 64 KB to 2 MB blocks (Apple M3, level 3, Silesia). An exact-size destination falls back to the bounce buffer instead of failing with ZXC_ERROR_OVERFLOW.

Build & Packaging

• Trusted publishing for the release workflows (#421), replacing long-lived tokens.
• Rapidhash is resolved at configure time (#420). The vendored copy is used unless -DZXC_USE_SYSTEM_RAPIDHASH=ON, and a missing copy fails at configure rather than mid-build. Thanks to @jeanga for the issue.
• Wrapper packaging and release integrity tightened (#409). Publish jobs no longer install an unpinned global npm (#426, #427).
• One seeded checksum entry point (#423), tidier nudge tuning macros (#412), and SARIF uploads skipped on fork PRs.

Documentation

FORMAT.md gains the two decoder rules and the dictionary ID derivation (#413): a 32-bit value that binds a dictionary to an archive without authenticating it. The Rust crate's zxc-compress / zxc naming is clarified (#410), with README refinements (#408).

Changelog

• api: Enforces strict block size and EOF validation (#428)
• api: Implements reusable compression contexts (#424)
• api: Enhances seekable API with checksum control and robust range validation (#425)
• api: Fix empty frame compressio

Release Notes

v0.14.0 is a decode-speed release built on a format break.

This release boosts decompression speed by up to 19% over v0.13.x, up to doubling our performance lead over LZ4 on ARM64 (+85% to +105%) and up to tripling it on x86_64 (+50% to +175%).

Three independent changes get there: blocks stop describing what the decoder can derive (format v8), short back-references are copied 32 bytes at a time instead of 16, and at levels 1-5 the encoder keeps its matches far enough back that the widest SIMD copy path stays on. The PivCo Huffman kernels also picked up AVX-512 VBMI, 256-bit AVX2 and NEON work.

Nothing in the public API moved: no symbol added, none removed, ABI unchanged (libzxc.so.4). The break is in the container format.

⚠️ Breaking change (format). The container format is now v8. v0.14.0 writes v8 archives that v0.13.x and earlier decoders reject, and it cannot read v7 or older: the header version byte is checked for exact equality (ZXC_ERROR_BAD_VERSION). The ABI is unchanged (soname stays libzxc.so.4), so no re-link is required — but upgrading the library alone makes newly written archives v8.

Migrating v7 -> v8 is a one-time transcode — decompress with your old build, recompress with the new one:

zxc-old -dc old.zxc | zxc-new -z -c > new.zxc

Keep a v7 build around until your data at rest is converted; if the old archive used a dictionary, pass it to the decompress side (zxc-old -dc -D dict.zxd). Full guide (bulk migration, dictionaries, verification): docs/MIGRATION.md.

ZXC format v8

Compression is untouched — same parser, same match finder, same entropy coders. What changed is how a block describes itself, and every change pulls the same way: stop writing down what the decoder can work out. (#374)

• Block sub-header: 16 -> 12 bytes. Four reserved bytes nobody used.
• Descriptors only when they carry information. v7 wrote a fixed 32-byte table of section sizes; most of them follow from the header. v8 keeps the two that don't: the literal section's compressed size when it is RLE- or entropy-coded, and the token section's at level 7. GLO now carries 0, 4 or 8 descriptor bytes, GHI none. The extras section is the leftover payload.
• enc_litlen -> enc_tok. The field always described the token section, never literal lengths.
• 32 readable bytes must follow the literal section. Literals are copied with a wild copy that overshoots, and those bytes keep the last copy inside the payload. Real blocks clear the bar without trying; the encoder pads into the extras when a tiny block doesn't. Always relied on, never written down — now normative, and decoders must reject blocks that break it.
• enc_lit, enc_tok and enc_off are closed value sets. Decoders reject unknown values instead of defaulting, which keeps the spare values free for a future version.

Per block this saves 25 to 34 bytes: invisible at the 512 KB default (levels 6 and 7 land 0.02 % smaller), worth about 2 % at the 4 KB blocks where dictionaries live.

Performance

Decompression on Silesia (202 MB), single-threaded, v0.13.3 -> v0.14.0 on the same reference hardware:

In absolute terms, level 3 decodes at 8.4 GB/s on Apple M2 and 6.7 GB/s on Zen 5; level 6 holds 6.7 GB/s on M2 at a 36.28 % ratio, ahead of lz4hc -9 on both axes on every machine

Release Notes

v0.13.3 is a hardening and packaging release.

The headline isn't really in the code: ZXC is now enrolled in Google OSS-Fuzz, which runs five harnesses around the clock. It has already turned up two decoder edge cases, both fixed here. Neither is reachable from an archive ZXC itself produced — you need a hand-crafted file to hit them — but a decompressor that can be walked out of its buffer is a decompressor worth fixing.

The rest is housekeeping. The GHI header's enc_off field is now formally unused, which is the ambiguity that let the first bug through; match-offset validation lost a counter it never needed; and static linking on Windows works again.

Nothing changes in the format or the ABI. v0.13.3 reads and writes the same format v7 archives as the rest of the v0.13.x line, and the soname is still libzxc.so.4.

Drop-in for v0.13.x. No format break, no ABI change, nothing to re-link. Archives move both ways between v0.13.3 and any earlier v0.13.x release. One detail worth knowing: at levels 1 and 2 the encoder now writes the unused enc_off GHI header field as 0 rather than 1, so output at those levels is no longer byte-for-byte what v0.13.2 produced. It is still standard format v7, and every v0.13.x release reads it. Levels 3-7 are byte-identical.

Reliability & Security Fixes

Out-of-bounds read on forged GHI offsets (decoder)

The GHI safe loop took its offset-validation threshold from the enc_off header field: 256 bytes when enc_off == 1. That field never described GHI offsets, which live inline in the sequence word and are always 16 bits wide. So a crafted archive could claim enc_off = 1, carry a much larger offset, and once 256 bytes had been written the loop simply stopped checking — and the copy read from before the start of the output buffer. The threshold is now the 16-bit bound, unconditionally. Found by OSS-Fuzz (544800535). (#367)

Output-buffer overflow on escaped literal lengths (decoder)

When a sequence escapes its literal length into a varint, the GLO and GHI decode macros used to check that the literals fit in the remaining output space — and nothing else. They ignored the match length that follows, and the output still owed by the rest of the batch. A crafted archive could clear that check and then write past the end of the destination buffer. Both macros now weigh literals, match length and the batch remainder together, and the trailing-literal copy has an explicit guard in front of it. Found by OSS-Fuzz (546426939). (#369)

enc_off is unused in GHI blocks and must be ignored

#367 was really a spec problem, so the field is now nailed down on both sides. The compressor always writes enc_off = 0 in GHI headers, and FORMAT.md says decoders must ignore it. GHI has no offset stream and its sequence words always carry 16-bit offsets, so there is nothing for the field to bound. (#368)

Stale dictionary size on a reused compression context

zxc_compress_cctx() never reset dict_size. Reuse a context after a dictionary compression and the old dictionary size came along for the ride into the next, plain one. It is reset on every call now. (#351)

Performance

Leaner match-offset validation

Match-offset validation used to lean on a running count of bytes written, updated on every single sequence. That counter is gone. The decoder now compares the match source directly to the start of the output buffer, or of the dictionary prefix when one is attached. Same guarantee, less bookkeeping in the hot loop. (#368)

Build, Packaging & Portability

Windows static-library linking

__declspec(dllimport) and static libraries don't get along on Windows: the link fails outright. ZXC_DLL_IMPORT is opt-in from now on, so plain declarations link against both the DLL and the static library. CMake, M

Release Notes

Release v0.13.2 is a performance and hardening patch release. The headline change: levels 6 and 7 now decompress +2–3 % and +10–11 % faster respectively, thanks to a decode-oriented nudge of the Huffman code lengths, at a near-zero ratio cost: compressed output grows by just 0.05 % at level 6 and 0.27 % at level 7. It also fixes two decoder edge cases (an out-of-bounds read on short RAW literal sections, and an in-place bound that could under-reserve the trailing framing), makes ZXC embeddable as a CMake subproject, and shrinks the shared library by removing duplicate SIMD variants. There are no changes to the container format or the ABI: v0.13.2 reads and writes the same format v7 archives as v0.13.0 and v0.13.1, and the shared-library soname stays libzxc.so.4.

Drop-in for v0.13.0 / v0.13.1. No format break, no ABI change, no re-link required. Every archive written by any v0.13.x release reads back identically in v0.13.2 and vice versa. One nuance: at levels 6 and 7 the encoder now emits slightly flatter Huffman trees, so archives written at those levels are no longer byte-for-byte identical to v0.13.1's output but they remain standard format v7, readable by every v0.13.x release. Levels 1-5 produce byte-identical output.

Performance

Faster decompression at levels 6 and 7

The encoder now applies a joint flat/length nudge to Huffman code lengths (#336): trees are flattened wherever it costs almost nothing in compressed size, so the PivCo decoder spends fewer cycles per symbol. At level 7, a dynamic-programming tier picks the best nudge per block; level 6 gets the same nudge in its lighter form (#343). Idea by @dougallj.

Levels 1-5 are unaffected.

Reliability Fixes

Out-of-bounds read on short RAW literal sections (decoder)

RAW literal sections decoded directly from the input buffer could read a few bytes past the end of the compressed input when the section tail was shorter than the decoder's padding requirement and no data followed it. Such sections are now staged through a padded internal buffer, and the conformance tests cover these layouts. (#341)

In-place bound now reserves the trailing framing

zxc_inplace_margin() ignored what the encoder writes after the last data block — the EOF block header and, for seekable archives, the seek table — so the returned bound could fall below the compressed size for seekable archives of incompressible data with small blocks. The worst-case seek table is now reserved unconditionally (4 bytes per block, at most 0.1 % of the payload at the minimum block size, negligible at the default). (#337)

Embedding & Portability

Vendor ZXC as a CMake subproject

ZXC can now be embedded directly into another CMake build via add_subdirectory() or FetchContent (#338). When built as a subproject: the CLI, tests, LTO and -march=native default to OFF; install, doc and formatting targets are disabled; compiler flags are inherited from the parent project instead of being overridden; and a zxc::zxc_lib ALIAS target is provided for consistent linking. A new ZXC_USE_SYSTEM_RAPIDHASH option controls whether the vendored or system rapidhash.h is used. The README documents the vendoring workflow.

No more compiler-rt dependency for x86 CPU detection

CPUID and XGETBV are now issued through direct intrinsics and inline assembly instead of compiler-specific builtins, removing the implicit compiler-rt dependency and making the runtime dispatch robust across more toolchains. (#339)

Build, Binary Size & CI

• Duplicate SIMD variants removed: the SSE2 and NEON function-multi-versioning variants duplicated the baseline code paths and are gone, shrinking the shared library by roughly a third