Release Notes

v0.15.0 is an integrity release built on a format break, and a speed release on top of it.

Decompression gains up to 24 % at levels 3-5 and 9-14 % at level 7 over v0.14.x; compression gains up to 39 % at levels 3-7. Level 3 now decodes 2.03x faster than LZ4 on Apple M2 (1.75x in v0.14), at a smaller size.

Format v9 redoes every integrity field: checksums that leave no header bit unprotected, block checksums over the decoded bytes, an optional archive digest, a seek table that validates itself group by group. On top of it, ZXC archives become multi-frame: cat a.zxc b.zxc is a valid archive, and the seekable reader spans every frame.

⚠️ Breaking change (format and ABI). The container format is now v9, and dictionaries move to .zxd version 2. v0.15.0 cannot read v8 archives or version-1 dictionaries, and v0.14.x cannot read what v0.15.0 writes (ZXC_ERROR_BAD_VERSION). The soname moves to libzxc.so.5: seekable block counts are 64-bit (#438), so programs linked against libzxc.so.4 must be rebuilt.

Migrating v8 -> v9 is the same one-time transcode as before:

zxc-old -dc old.zxc | zxc-new -z -c > new.zxc

A dictionary is migrated with its archives: retrain it with the new build, or convert it byte for byte to keep its dict_id. Full guide, including the CLI output changes and the new Windows archive layout: docs/MIGRATION.md.

ZXC format v9

The compressed data is unchanged: same blocks, same parser, same entropy coders. The break is in what protects them and in how a reader finds its way. (#452)

• Multiplicative header checksums (#432). The XOR/shift construction left part of the compressed size unprotected; the same 2 bytes now cover every header bit.
• Block checksums cover the decoded bytes, seeded by position (#434). A block moved or duplicated inside an archive no longer verifies. The whole-archive hash gives way to an optional 8-byte archive digest folded from the block checksums.
• A seek table that validates itself (#436, #438). Groups of one 64-bit anchor and 64 block sizes, checked one group at a time and loaded lazily, so opening a large archive reads only what a range needs. Block counts are 64-bit.
• HAS_SEEK_TABLE header flag (#448). The table is announced instead of probed for. In v0.14, about one footer in 65 536 parsed as a seek-table header and failed an intact archive with ZXC_ERROR_IO; that ambiguity is gone.
• A variable-length footer (#489). Source size and compressed frame size are stored on the bytes they need, behind the optional digest, with a last byte giving their lengths: 3 to 17 bytes, plus 8 with the digest. ZXC_FILE_FOOTER_SIZE is replaced by ZXC_FILE_FOOTER_MIN_SIZE / ZXC_FILE_FOOTER_MAX_SIZE.
• One valid form for the EOF and SEK headers. Reserved bytes must be zero there and decoders check them, so a lying flag reads as corrupt data rather than a truncated table.
• .zxd dictionaries version 2, signed with the new header checksum.

Multi-frame archives (#490)

A ZXC file is now one or more frames back to back. After each footer comes the end of the input, the next frame's magic, or an error.

• Concatenation just works. cat a.zxc b.zxc > ab.zxc decodes to both sources in order, through the buffer, context, FILE* and push-stream APIs, and the size queries add the frames up. Frames keep their own block size, checksum flag and dictionary.
• Seekable across frames. When every frame carries a seek table, zxc_seekable_open spans them all (up to 2^20 frames), with a dictionary per frame.
• Frame info API (#489). zxc_get_frame_info(), zxc_get_last_frame_info() and their FILE* versions report a frame's sizes and flags; the compressed frame size in each footer lets a reader walk the frames back from the end.
• Wrappers. Go, Rust and Python readers decode concatenations as one stream by default, with an opt-out (Multistream(false), .multistream(false), multistream=False) for a source that stays open after one frame.
• CLI. zxc -d and -t decode every frame. zxc -l gains Frames and Seekable columns and reports enabled / disabled / mixed; the JSON keys changed (see MIGRATION).

Performance

Silesia (202 MB), single-threaded, v0.14.x -> v0.15.0 on the same reference hardware.

Decompression:

Compression:

In absolute terms, level 3 decodes at 9.7 GB/s on Apple M2 and 8.3 GB/s on Zen 5, and level 1 at 13.5 GB/s on M2. Full tables and effective throughput: README and WHITEPAPER.

v0.14.x figures are the v0.14.0 run (lzbench 2.3.1): v0.14.1 writes byte-identical output and its decode path only changed for the block API. v0.15.0 was measured with lzbench 2.4.1.

Long GLO matches split into inline sequences at levels 3-5 (#441)

Long matches leave the decoder's slow path: the encoder splits them into sequences the inline copy handles, which is where most of the level 3-5 gain comes from. The trade is paid in ratio, at those three levels only:

Decoder and encoder kernels

• Wild-copy loops (#439) and GHI batch register pressure (#460) tightened in the decoder's hot loops.
• PivCo Huffman tree simplified (#451).
• Faster LZ77 parsing (#466): lazy evaluation exits early, and the optimal parser carries a match over instead of re-evaluating it, skipping redundant comparisons and DP relaxations.

Parallel FILE* decompression (#495)

From a regular file, workers read their own blocks with positioned reads and small blocks travel in batches, so the reader no longer copies every byte. Concatenated frames of the same geometry share one engine instead of restarting threads and buffers per frame. Pipes and sockets still decode block by block, so a live stream is never held back.

API

• Zero-allocation in-place decompression (#461): zxc_decompress_inplace_dctx() decodes in place through a reusable context, with no allocation per call.
• Seekable archives from the context API (#487): the compression context can write a seek table.
• Frame info (#489): zxc_get_frame_info(), zxc_get_last_frame_info(), zxc_stream_get_frame_info(), zxc_stream_get_last_frame_info(), zxc_frame_info_size().
• 64-bit seekable block counts (#438): zxc_seekable_get_num_blocks() and zxc_seek_table_size() take and return uint64_t (the ABI break above).
• Linux kernel support (#454): the library builds as a kernel module, scalar paths only, with its own CI workflow.

Wrappers

• Python (#505): zxc.compress() and zxc.Cctx take seekable=True, as stream_compress() already did.
• Go (#499): Compress and CompressTo honour WithSeekable(true); they returned an archive without a seek table. CompressTo accepts empty input and writes a minimal frame, as Compress does.

Reliability & Security

• In-place decompression (#450): strict read/write separation; a write could overtake unread input under a forged archive.
• Type-safe stream callback (#449): the processor callback no longer goes through an incompatible function-pointer cast (UBSan -fsanitize=function).
• MSVC 19.51 ARM64 (#458): worked around a miscompile of the seek-table size fold.
• Go seekable reader (#486): a panic in a user reader is recovered and reported as an error instead of crashing across the C trampoline.
• WASM (#459): a failed heap allocation throws instead of returning a null pointer.
• Frame validation and dictionary edge cases: truncated or forged footers, stale FILE* error indicators and dictionaries that no frame needs are handled consistently across the buffer, stream and seekable paths.
• Fuzzing (#463): a block-API fuzzer and coverage of the dictionary Huffman table; OSS-Fuzz line coverage of the v9 decoder went from 0.9 % to 84.8 % once the corpus was rebuilt.

CLI

• zxccat (#503): an alias for zxc -dc that decompresses every file it is given, or standard input, to standard output, one after the other: zxccat logs-*.zxc | grep ERROR. Installed next to unzxc on POSIX systems.
• -c appends to a >> redirect (#503): decompressing to standard output reopened it and truncated a file it should have appended to.
• zxcless (#506): pages files in less(1), decompressing .zxc archives on the fly (through LESSOPEN, less 451 or later); other files and standard input display unchanged. With it, -f now copies input t...

Release Notes

v0.14.1 is a hardening release: every decode path got stricter, block decompression is 10-13 % faster, reusable contexts reach all five wrappers, and seekable archives can verify per-block checksums.

Drop-in for v0.14.0. Same format v8, byte-identical output at every level (verified on Silesia), soname still libzxc.so.4. One symbol added, zxc_seekable_set_checksum; none removed or changed.

Decoder hardening

• A forged block size passed off a truncated decode as success (#428). A payload size spanning the EOF marker made the frame walk run off the archive and return what it had, with checksums off (the default). A data block's payload can no longer exceed the header's block size, the walk must end on the EOF block, and a seek-table entry spans at most one block. The check sits in the block decoder shared by every reader. Both rules are now normative in FORMAT.md.
• Stack address stored in heap memory (#429). Multi-threaded seekable jobs held the caller's output pointer. This was harmless, since workers are joined before return, but a scanner can't prove that. Jobs now carry offsets from one base pointer on the caller's stack.
• zxc_stream_decompress ignored the dictionary size limit (#430). A dictionary above ZXC_DICT_SIZE_MAX was accepted, or rejected with a misleading ZXC_ERROR_DICT_MISMATCH. The check moved into the shared stream engine.
• Contexts (#417, #422). Static contexts reject oversized blocks up front. An empty input through a context now yields the canonical 36-byte archive, like the one-shot API.

API

• Reusable contexts in every wrapper (#424, #414, #415). Cctx and Dctx now ship in Rust, Go, Python, Node.js and WASM. A dictionary given to the factory applies to every call that doesn't override it. The block API also takes a dictionary's shared literal Huffman table (#415), so small blocks no longer spend bytes describing their own.
• Seekable checksum verification (#425). zxc_seekable_decompress_range can verify per-block checksums through zxc_seekable_set_checksum. It is off by default and exposed in all five wrappers.
  Also fixed: an overflowing offset plus length slipped past the bounds check, and an incomplete range returned a silent partial read instead of ZXC_ERROR_CORRUPT_DATA.

CLI

• -l and -t read checksum status from the HAS_CHECKSUM header flag (#435). -j adds checksum_method; exit codes are unchanged. Thanks to @Vladexy88x for the fix.

Performance

• zxc_decompress_block writes straight to the destination (#418). Given the zxc_decompress_block_bound() capacity, it skips the bounce-buffer copy and runs 10-13 % faster, from 64 KB to 2 MB blocks (Apple M3, level 3, Silesia). An exact-size destination falls back to the bounce buffer instead of failing with ZXC_ERROR_OVERFLOW.

Build & Packaging

• Trusted publishing for the release workflows (#421), replacing long-lived tokens.
• Rapidhash is resolved at configure time (#420). The vendored copy is used unless -DZXC_USE_SYSTEM_RAPIDHASH=ON, and a missing copy fails at configure rather than mid-build. Thanks to @jeanga for the issue.
• Wrapper packaging and release integrity tightened (#409). Publish jobs no longer install an unpinned global npm (#426, #427).
• One seeded checksum entry point (#423), tidier nudge tuning macros (#412), and SARIF uploads skipped on fork PRs.

Documentation

FORMAT.md gains the two decoder rules and the dictionary ID derivation (#413): a 32-bit value that binds a dictionary to an archive without authenticating it. The Rust crate's zxc-compress / zxc naming is clarified (#410), with README refinements (#408).

Changelog

• api: Enforces strict block size and EOF validation (#428)
• api: Implements reusable compression contexts (#424)
• api: Enhances seekable API with checksum control and robust range validation (#425)
• api: Fix empty frame compression in contexts (#422)
• api: Enforce strict block size bounds for static decompression contexts (#417)
• api: Adds shared literal table support to block API (#415)
• api: Extends dictionary support for reusable contexts (#414)
• api: Refines README documentation (#408)
• perf: Optimizes block decompression for direct and flexible output (#418)
• cli: Report checksum presence from the archive header in -l and -t (#435)
• build: Migrate release workflows to trusted publishing (#421)
• build: Enforce rapidhash resolution at configure/setup time (#420)
• build: Resolve dependency resolution in wrapper test (#410) (#411)
• build: Enhances wrapper packaging and release integrity (#409)
• doc: Clarify crate branding and improve documentation metadata (#410)
• misc: Compile library objects separately for fuzzers (#440)
• misc: Add fuzzer for streaming API (#443)
• misc: Skip SARIF uploads for fork pull requests
• misc: Enforce dictionary size limits on decompression path (#430)
• misc: Prevent stack memory escape in multi-threaded decompression (#429)
• misc: Drop unpinned global npm install from publish job (#426)
• misc: Drop unpinned global npm install from publish job (#427)
• misc: Consolidates checksums (#423)
• misc: Formalizes dictionary ID calculation and enhances fuzzer (#ossf-557281704) (#413)
• misc: Tidy the nudge tuning macros (#412)

Full Changelog: v0.14.0...v0.14.1

Release Notes

v0.14.0 is a decode-speed release built on a format break.

This release boosts decompression speed by up to 19% over v0.13.x, up to doubling our performance lead over LZ4 on ARM64 (+85% to +105%) and up to tripling it on x86_64 (+50% to +175%).

Three independent changes get there: blocks stop describing what the decoder can derive (format v8), short back-references are copied 32 bytes at a time instead of 16, and at levels 1-5 the encoder keeps its matches far enough back that the widest SIMD copy path stays on. The PivCo Huffman kernels also picked up AVX-512 VBMI, 256-bit AVX2 and NEON work.

Nothing in the public API moved: no symbol added, none removed, ABI unchanged (libzxc.so.4). The break is in the container format.

⚠️ Breaking change (format). The container format is now v8. v0.14.0 writes v8 archives that v0.13.x and earlier decoders reject, and it cannot read v7 or older: the header version byte is checked for exact equality (ZXC_ERROR_BAD_VERSION). The ABI is unchanged (soname stays libzxc.so.4), so no re-link is required — but upgrading the library alone makes newly written archives v8.

Migrating v7 -> v8 is a one-time transcode — decompress with your old build, recompress with the new one:

zxc-old -dc old.zxc | zxc-new -z -c > new.zxc

Keep a v7 build around until your data at rest is converted; if the old archive used a dictionary, pass it to the decompress side (zxc-old -dc -D dict.zxd). Full guide (bulk migration, dictionaries, verification): docs/MIGRATION.md.

ZXC format v8

Compression is untouched — same parser, same match finder, same entropy coders. What changed is how a block describes itself, and every change pulls the same way: stop writing down what the decoder can work out. (#374)

• Block sub-header: 16 -> 12 bytes. Four reserved bytes nobody used.
• Descriptors only when they carry information. v7 wrote a fixed 32-byte table of section sizes; most of them follow from the header. v8 keeps the two that don't: the literal section's compressed size when it is RLE- or entropy-coded, and the token section's at level 7. GLO now carries 0, 4 or 8 descriptor bytes, GHI none. The extras section is the leftover payload.
• enc_litlen -> enc_tok. The field always described the token section, never literal lengths.
• 32 readable bytes must follow the literal section. Literals are copied with a wild copy that overshoots, and those bytes keep the last copy inside the payload. Real blocks clear the bar without trying; the encoder pads into the extras when a tiny block doesn't. Always relied on, never written down — now normative, and decoders must reject blocks that break it.
• enc_lit, enc_tok and enc_off are closed value sets. Decoders reject unknown values instead of defaulting, which keeps the spare values free for a future version.

Per block this saves 25 to 34 bytes: invisible at the 512 KB default (levels 6 and 7 land 0.02 % smaller), worth about 2 % at the 4 KB blocks where dictionaries live.

Performance

Decompression on Silesia (202 MB), single-threaded, v0.13.3 -> v0.14.0 on the same reference hardware:

In absolute terms, level 3 decodes at 8.4 GB/s on Apple M2 and 6.7 GB/s on Zen 5; level 6 holds 6.7 GB/s on M2 at a 36.28 % ratio, ahead of lz4hc -9 on both axes on every machine measured. Full tables, cycles/byte and effective throughput: README and WHITEPAPER.

Three levers get there, all independent.

Overlap copies run 32 bytes at a time (#374)

Short back-references took one of two paths, and one re-read bytes it had just written at a distance straddling two stores: no store forwarding, a stall every iteration. A 32-byte pattern covers distances 2 to 31 in a single arm — the stall goes, four distance tests become three, and the old 16-byte kernel (four SIMD variants, two lookup tables) is deleted. Short GLO sequences also get their own copy paths, specialized on branches the decoder was already taking.

A minimum match distance at levels 1-5 (#376)

Levels 1 to 5 now keep matches at least 32 bytes back, so the decoder stays on its widest SIMD copy path instead of dropping into overlap handling. On periodic data that floor would cost ratio and speed, so the call is made per block: a probe samples about one position per KB (16 to 64 samples) for 4-byte repeats inside the forbidden window, and past 20 % hits the block keeps its short matches. Levels 6 and 7 are untouched.

The trade is paid in ratio:

Levels 1-5 buy 10-19 % of decode speed for under 1 % of size. Levels 6 and 7 keep their ratio and get slightly smaller, from the v8 header savings.

Wider PivCo Huffman kernels (#373, #377)

• AVX2 merge widened to 256 bits. The AVX2 tier was still running the SSSE3 kernel: two 128-bit selects, an OR and two stores per 32 outputs. Now one pair of 256-bit selects, a blend and a single store — 16 vector instructions instead of 24. (#377)
• AVX-512. _mm512_mask_expandloadu_epi8 folds the masked load and the byte expand into one fault-safe instruction. With VBMI, vpmultishiftqb does the unaligned D-bit read in a single instruction, 64 codes per step, and leaf emission uses the control word directly as its blend mask. The dispatch gate now checks VBMI, which the variant is already built with.
• AVX2 unpack. A 32-codes-per-step path for D = 2..6, plus a 32-byte leaf emitter.
• NEON. vld1q_u8_x2 / vld1q_u8_x4 for the c2s tables, instead of filling the lanes one load at a time.
• Encoder. A symbol's bit-reversed residual never changes, so it is computed once per symbol instead of once per literal.

Reliability & Security

In-place decompression: forged footers and buffer separation (#375)

• Forged footer sizes. The decompressed size from the footer is checked for plausibility before anything is sized from it, so untrusted input can't drive an outsized allocation. The check is centralized and shared by every path that reads the footer.
• Buffer separation floor. Attacker-controlled padding could shrink the read/write separation in-place decoding depends on. A minimum floor is enforced, so the margin holds whatever the archive claims.
• A forged footer caught during in-place decompression returns ZXC_ERROR_CORRUPT_DATA rather than a vaguer code.

Stricter block validation (#374)

Blocks that break the 32-readable-bytes rule are rejected. Unknown enc_lit / enc_tok / enc_off values are rejected instead of silently defaulting. The stream layer reports distinct error codes where it used to return one generic I/O error.

Library internals: hardening and consolidation (#391)

What changes behaviour:

• Destination capacity below a block sub-header. The size check wrapped around and the encoder wrote the sub-header out of bounds. Rejected up front now, on the block and frame APIs.
• Arithmetic that could wrap. Block-header validation moved to 64 bits; the seekable entry count is checked against its 32-bit field; the block count uses division-and-remainder instead of a ceiling addition.
• A truncated stream with an agreeing footer decompressed as a clean short read. An unreadable block header is corruption now, and the stream layer stops swallowing allocation failures.
• Measuring runs are exact. Compressing with a NULL output wrote the file header and the seek table without counting them, under-reporting by 16 bytes, 52 with a seek table.

The rest is consolidation: one threading shim instead of two, one RLE lookahead instead of one per SIMD backend, centralized cleanup on the engine and dictionary-training failure paths, and a read buffer held on the seekable handle instead of reallocated per range request.

Format testing: annotated vectors, frozen corpus (#407)

The format moved this release under a test suite that could only say the bytes differ.

• Golden archives ship an annotated dump. Each tests/format/golden/*.zxc has a .zxc.txt beside it: every header field decoded and named, plus a payload hash. A format change reviews as a readable diff instead of Binary files differ, and the hash catches an encoder change that leaves the fields intact.
• The conformance corpus is versioned and frozen. Vectors live in conformance/v8/, one directory per format version, each with a vectors.sha256 manifest CI verifies byte for byte. v8 ships 15 valid and 28 invalid vectors.
• Valid vectors have a written recipe. valid_cases.h and zxc_valid_gen make the construction parameters explicit and the corpus regenerable, and the suite is one entry per decoder-visible trait rather than one per level, so a failure points at a single format rule.

The golden and conformance workflows m...

Release Notes

v0.13.3 is a hardening and packaging release.

The headline isn't really in the code: ZXC is now enrolled in Google OSS-Fuzz, which runs five harnesses around the clock. It has already turned up two decoder edge cases, both fixed here. Neither is reachable from an archive ZXC itself produced — you need a hand-crafted file to hit them — but a decompressor that can be walked out of its buffer is a decompressor worth fixing.

The rest is housekeeping. The GHI header's enc_off field is now formally unused, which is the ambiguity that let the first bug through; match-offset validation lost a counter it never needed; and static linking on Windows works again.

Nothing changes in the format or the ABI. v0.13.3 reads and writes the same format v7 archives as the rest of the v0.13.x line, and the soname is still libzxc.so.4.

Drop-in for v0.13.x. No format break, no ABI change, nothing to re-link. Archives move both ways between v0.13.3 and any earlier v0.13.x release. One detail worth knowing: at levels 1 and 2 the encoder now writes the unused enc_off GHI header field as 0 rather than 1, so output at those levels is no longer byte-for-byte what v0.13.2 produced. It is still standard format v7, and every v0.13.x release reads it. Levels 3-7 are byte-identical.

Reliability & Security Fixes

Out-of-bounds read on forged GHI offsets (decoder)

The GHI safe loop took its offset-validation threshold from the enc_off header field: 256 bytes when enc_off == 1. That field never described GHI offsets, which live inline in the sequence word and are always 16 bits wide. So a crafted archive could claim enc_off = 1, carry a much larger offset, and once 256 bytes had been written the loop simply stopped checking — and the copy read from before the start of the output buffer. The threshold is now the 16-bit bound, unconditionally. Found by OSS-Fuzz (544800535). (#367)

Output-buffer overflow on escaped literal lengths (decoder)

When a sequence escapes its literal length into a varint, the GLO and GHI decode macros used to check that the literals fit in the remaining output space — and nothing else. They ignored the match length that follows, and the output still owed by the rest of the batch. A crafted archive could clear that check and then write past the end of the destination buffer. Both macros now weigh literals, match length and the batch remainder together, and the trailing-literal copy has an explicit guard in front of it. Found by OSS-Fuzz (546426939). (#369)

enc_off is unused in GHI blocks and must be ignored

#367 was really a spec problem, so the field is now nailed down on both sides. The compressor always writes enc_off = 0 in GHI headers, and FORMAT.md says decoders must ignore it. GHI has no offset stream and its sequence words always carry 16-bit offsets, so there is nothing for the field to bound. (#368)

Stale dictionary size on a reused compression context

zxc_compress_cctx() never reset dict_size. Reuse a context after a dictionary compression and the old dictionary size came along for the ride into the next, plain one. It is reset on every call now. (#351)

Performance

Leaner match-offset validation

Match-offset validation used to lean on a running count of bytes written, updated on every single sequence. That counter is gone. The decoder now compares the match source directly to the start of the output buffer, or of the dictionary prefix when one is attached. Same guarantee, less bookkeeping in the hot loop. (#368)

Build, Packaging & Portability

Windows static-library linking

__declspec(dllimport) and static libraries don't get along on Windows: the link fails outright. ZXC_DLL_IMPORT is opt-in from now on, so plain declarations link against both the DLL and the static library. CMake, Meson and pkg-config each propagate the right define — ZXC_DLL_IMPORT for shared, ZXC_STATIC_DEFINE for static — and a new packaging workflow builds a real consumer against every combination on Linux and Windows, so it stays fixed. (#350)

CMake build modularized

CMakeLists.txt had grown past 700 lines. It is now split into focused modules under cmake/ — version, options, platform detection, compiler flags, dependencies, SIMD variants, tests, docs, formatting, WASM, install, summary — with the top-level file left as the orchestrator. Consumers see no difference. (#347)

Meson CLI parity

The Meson build now passes the MSVC compiler and linker flags CMake already had, and installs the unzxc symlink on POSIX systems. (#348)

Quality & Docs

• Continuous fuzzing moved to Google OSS-Fuzz. Five harnesses — roundtrip, decompress, streaming, seekable, dictionary — run there around the clock. The same harnesses still run under ClusterFuzzLite (ASan + UBSan) on every pull request that touches the library. README badge and workflow triggers updated to match.
• Public headers and internal comments got a pass: Doxygen blocks tightened, internal explanations corrected where they had drifted from the code, clang-format extended to tests/. (#366)
• The level-ordered layout behind the PivCo-Huffman section codec is credited to Marcin Żukowski (@MarcinZukowski), and the joint flat/length nudge to Dougall Johnson (@dougallj). Both are implemented independently here.
• Dependency bumps across the Rust, Python and Node.js wrappers, the fuzzing base image and the GitHub Actions workflows.

Changelog

• api: Refine comments for improved clarity (#366)
• perf: Optimizes decode offset validation and clarifies GHI format (#368)
• fix: Reset dict_size in compression context (#351)
• fix: Ensure correct offset validation bounds for GHI decompression (ossfuzz #544800535) (#367)
• fix: Enhance output buffer overflow checks during decompression (ossfuzz #546426939) (#369)
• cli: Add MSVC CLI build configuration and unzxc alias (#348)
• build: Tag the Go submodule on release (#371)
• build: Publish the WebAssembly build to npm (#372)
• build: Update node.js wrapper dependencies
• build: bump scikit-build-core from 0.12.2 to 1.0.3 in /wrappers/python (#362)
• build: bump cibuildwheel from 4.1.0 to 4.1.1 in /wrappers/python (#360)
• build: bump setuptools-scm from 10.2.0 to 10.2.1 in /wrappers/python (#361)
• build: Remove scheduled workflow trigger
• build: bump cc from 1.2.65 to 1.4.0 in /wrappers/rust (#355)
• build: bump thiserror from 2.0.18 to 2.0.19 in /wrappers/rust (#354)
• build: bump libc from 0.2.186 to 0.2.189 in /wrappers/rust (#357)
• build: bump vitest from 4.1.9 to 4.1.10 in /wrappers/nodejs (#358)
• build: Prevent Windows static library linking failures (#350)
• build: Modularize CMakeLists.txt into dedicated modules (#347)
• doc: Update fuzzing status and details in README
• doc: Add PivCo-Huffman attribution for algorithm elements
• misc: bump meson from 1.11.1 to 1.11.2 in /.github/requirements (#363)
• misc: bump oss-fuzz-base/base-builder in /.clusterfuzzlite (#359)
• misc: bump cross-platform-actions/action (#365)
• misc: bump github/codeql-action/upload-sarif (#364)
• misc: bump actions/checkout from 6.0.2 to 7.0.1 (#356)
• misc: bump softprops/action-gh-release from 3.0.1 to 3.0.2 (#353)
• misc: bump actions/setup-python from 6.3.0 to 7.0.0 (#352)

Full Changelog: v0.13.2...v0.13.3

Release Notes

Release v0.13.2 is a performance and hardening patch release. The headline change: levels 6 and 7 now decompress +2–3 % and +10–11 % faster respectively, thanks to a decode-oriented nudge of the Huffman code lengths, at a near-zero ratio cost: compressed output grows by just 0.05 % at level 6 and 0.27 % at level 7.
It also fixes two decoder edge cases (an out-of-bounds read on short RAW literal sections, and an in-place bound that could under-reserve the trailing framing), makes ZXC embeddable as a CMake subproject, and shrinks the shared library by removing duplicate SIMD variants.
There are no changes to the container format or the ABI: v0.13.2 reads and writes the same format v7 archives as v0.13.0 and v0.13.1, and the shared-library soname stays libzxc.so.4.

Drop-in for v0.13.0 / v0.13.1. No format break, no ABI change, no re-link required. Every archive written by any v0.13.x release reads back identically in v0.13.2 and vice versa. One nuance: at levels 6 and 7 the encoder now emits slightly flatter Huffman trees, so archives written at those levels are no longer byte-for-byte identical to v0.13.1's output but they remain standard format v7, readable by every v0.13.x release. Levels 1-5 produce byte-identical output.

Performance

Faster decompression at levels 6 and 7

The encoder now applies a joint flat/length nudge to Huffman code lengths (#336): trees are flattened wherever it costs almost nothing in compressed size, so the PivCo decoder spends fewer cycles per symbol. At level 7, a dynamic-programming tier picks the best nudge per block; level 6 gets the same nudge in its lighter form (#343). Idea by @dougallj.

Levels 1-5 are unaffected.

Reliability Fixes

Out-of-bounds read on short RAW literal sections (decoder)

RAW literal sections decoded directly from the input buffer could read a few bytes past the end of the compressed input when the section tail was shorter than the decoder's padding requirement and no data followed it. Such sections are now staged through a padded internal buffer, and the conformance tests cover these layouts. (#341)

In-place bound now reserves the trailing framing

zxc_inplace_margin() ignored what the encoder writes after the last data block — the EOF block header and, for seekable archives, the seek table — so the returned bound could fall below the compressed size for seekable archives of incompressible data with small blocks. The worst-case seek table is now reserved unconditionally (4 bytes per block, at most 0.1 % of the payload at the minimum block size, negligible at the default). (#337)

Embedding & Portability

Vendor ZXC as a CMake subproject

ZXC can now be embedded directly into another CMake build via add_subdirectory() or FetchContent (#338). When built as a subproject: the CLI, tests, LTO and -march=native default to OFF; install, doc and formatting targets are disabled; compiler flags are inherited from the parent project instead of being overridden; and a zxc::zxc_lib ALIAS target is provided for consistent linking. A new ZXC_USE_SYSTEM_RAPIDHASH option controls whether the vendored or system rapidhash.h is used. The README documents the vendoring workflow.

No more compiler-rt dependency for x86 CPU detection

CPUID and XGETBV are now issued through direct intrinsics and inline assembly instead of compiler-specific builtins, removing the implicit compiler-rt dependency and making the runtime dispatch robust across more toolchains. (#339)

Build, Binary Size & CI

• Duplicate SIMD variants removed: the SSE2 and NEON function-multi-versioning variants duplicated the baseline code paths and are gone, shrinking the shared library by roughly a third on arm64 and a quarter on amd64. -mfma is dropped, and the x86 dispatch gates now also check BMI1/BMI2/LZCNT before selecting a variant that uses them. (#331)
• The ISA-independent dictionary tree setup is compiled once instead of once per SIMD variant. (#327)
• MSVC /arch mapping and /wd4244 flags fixed, silencing the wrapdb CI warnings. (#326)
• Wrapper sources are formatted and the formatting is enforced in CI. (#330)
• Remaining compiler warnings resolved. (#342)
• The AVX-512 tier is now tested under Intel SDE instead of QEMU, which cannot emulate AVX-512. (#325)
• npm dev dependencies updated to address audit alerts. (#218, #335)

Internal

• The GLO/GHI decode loops are deduplicated and macro-folded (−666 lines); the hot paths are byte-identical. (#328)
• Typo fixes in the documentation.

Changelog

• fix: fix typos
• fix: Fix MSVC /arch + /wd4244 flags (wrapdb CI warnings) (#326)
• perf: Extend code length nudging to level 6 (#343)
• perf: Optimizes Huffman code lengths for faster PivCo decode (#336)
• cli: Make project embeddable as a subproject (#338)
• cli: Resolve compiler warnings (#342)
• build: Format wrapper sources and guard it in CI (#330)
• portability: Remove compiler-rt dependency for x86 CPUID (#339)
• misc: Stage short RAW literal sections to prevent overread (#341)
• misc: Reserve the trailing framing in the in-place bound (#337)
• misc: Deduplicate the GLO/GHI decode loops (#328)
• misc: Update npm dev dependencies to address alerts (#218) (#335)
• misc: Remove duplicate SSE2/NEON FMV variants, drop -mfma, harden x86 dispatch gates (#331)
• misc: Compile ISA-independent dict tree setup once, not per-variant (#327)
• misc: Test the AVX-512 tier under Intel SDE instead of QEMU (#325)

Full Changelog: v0.13.1...v0.13.2

Release Notes

v0.13.1 is a build-only patch release. It fixes the Meson build so the Windows DLL correctly exports its public API, raises the minimum Meson version so c_std is honoured in subproject builds, and corrects a README heading. There are no changes to the codec, the container format, or the ABI: v0.13.1 reads and writes the exact same format v7 archives as v0.13.0, and the shared-library soname stays libzxc.so.4.

Drop-in for v0.13.0. No format break, no ABI change, no re-link required. Archives written by v0.13.0 and v0.13.1 are byte-for-byte interchangeable. If you build ZXC from source with Meson on Windows, or vendor it as a Meson subproject, this is the release to use.

Build Fixes

Windows DLL now exports its API (Meson)

The Meson build did not define the export-control macro, so zxc.dll was built with its public symbols hidden — leaving downstream projects unable to link against it. The build now defines -Dzxc_lib_EXPORTS for shared builds (and -DZXC_STATIC_DEFINE for static builds and consumers), so the Windows DLL exports its API exactly as the CMake build already did.

c_std honoured in subproject builds (Meson)

The minimum Meson version is raised to >= 0.63.0 so that the project's c_std=c17 from default_options is applied when ZXC is consumed as a Meson subproject. On older Meson, that option was silently ignored in the subproject case, which could downgrade the C standard used to build the vendored library.

Documentation

• README: the Windows install section (Option 6) is now correctly titled Winget, and the download example is bumped to 0.13.1.
• API.md: library version references updated to 0.13.1 (SOVERSION unchanged at 4).

Changelog

• api: Define zxc_lib_EXPORTS in Meson so the Windows DLL exports its API
• misc: Require Meson >= 0.63.0 so c_std applies in subproject builds
• docs: Update Option 6 to specify Winget for installation

Full Changelog: v0.13.0...v0.13.1

Release Notes

This release introduces ZXC format v7 and a new Level 7 (ULTRA) compression tier (Huffman-coded literals and sequence tokens over the new PivCo SIMD-decodable bitstream layout), that pushes the ratio past zstd -1 while still decoding roughly 2x faster than it. The entropy-decode path was reworked for a +5-13 % Level 6 decode speedup across every reference CPU, and decompression can now run in-place in a single buffer. The container format advances to v7; the shared-library ABI is unchanged (libzxc.so.4).

⚠️Breaking change (format). The container format is now v7. v0.13.0 writes v7 archives that v0.12.0 and earlier decoders reject, and it cannot read v6/v5 archives: the header version byte is checked for exact equality (ZXC_ERROR_BAD_VERSION). The shared-library ABI is unchanged (soname stays libzxc.so.4), so no re-link is required, but note that upgrading the library alone makes newly written archives v7.

Migrating v6 → v7 archives is a one-time transcode — decompress with your old (v6) build and recompress with the new one:

zxc-v6 -dc old.zxc | zxc-v7 -z -c > new.zxc

Keep a v6 build around until your data at rest is converted. Full guide (bulk migration, dictionaries, verification):
docs/MIGRATION.md.

New Features

Level 7 (ULTRA) Maximum Density

A new top compression level for storage and bandwidth-bound, write-once / read-many workloads. On top of Level 6's Huffman-coded literals, Level 7 additionally Huffman-codes the sequence-token stream (length-limited canonical codes up to 11 bits) and runs a deeper optimal parse selected by a space-speed Lagrangian (J = size + λ · decoded_bytes).

• On the Silesia corpus it reaches a 33.00 % ratio : 9 % denser than Level 6 (36.27 %) and denser than zstd -1 (34.53 %), while still decoding 2x faster than zstd -1. It fills the gap between lz4hc's speed and zstd's density. (#315)
• Exposed as -7 / ZXC_LEVEL_ULTRA in the CLI and library, and across Go, Rust, Python, Node.js, and WASM. (#317)

Pivot-Coded Huffman Bitstream (format v7)

Huffman-coded sections now place their bits on the wire in the PivCo level-ordered layout: the same length-limited canonical Huffman code, but the bits are transposed by tree level into per-node branch bitmaps so the decoder runs data-parallel SIMD list merges instead of a serial bit chain. This is the source of the Level 6 decode speedup and what makes Level 7's token coding practical. (#315)

In-Place Decompression

Decompression can now run in a single buffer: the compressed input sits flush-right and is decoded left-to-right within the same allocation; for lower peak memory in tightly-budgeted integrations. (#315)

CLI: Progress, Robust Parsing, and File I/O

The zxc CLI gains a progress indicator, more robust argument parsing, and hardened file-I/O handling. (#318) A new CLI install option (incl. winget on Windows) is documented in the README. (#316)

Performance & Memory

Measured versus v0.12.0 (Silesia, single-thread):

• Level 6 decode: +5% to +13% faster on every reference CPU: Apple M2 +6%, Google Axion / Neoverse-V2 +6%, AMD EPYC Zen 5 +13%, AMD EPYC Zen 3 +5%, from the reworked PivCo Huffman decoder.
• New Level 7 (ULTRA): 33.00% ratio : +9% denser than Level 6, denser than zstd -1, and decoding ~2x faster than it.
• Ratio at Levels 1-6 is unchanged (Levels 1-5 are byte-identical to v0.12.0).

• Reworked Huffman decoder and refined compression margins across the entropy path. (#301, #319)
• Space-speed Lagrangian selection at ULTRA trades ratio against decode cost per block. (#315)
• Cached dictionary Huffman tree: the dictionary's shared literal table is decoded into its ready-to-use tree once, at attach time, instead of per block: a large small-block dictionary-decode speedup. (#320)
• Robustness and memory-management refinements across the library. (#321)

Bug Fixes & Robustness

• Improved robustness and memory management across compression and decompression. (#321)
• Refined LZ77 search-loop conditions and general hardening across components. (#296, #298)
• Type-safety and modernization passes over the JS and Go wrappers. (#297)

Development & CI

• OS & libc compatibility CI for musl / Alpine Linux and FreeBSD, validating the freestanding core beyond glibc. (#291)
• ABI-check accuracy improved and the libabigail baseline updated for 0.13.0: opaque-handle layout changes are now correctly recognised as ABI-compatible (no soname bump).
• Refined code-coverage reporting with vendored code excluded. (#322)
• Meson project version and SOVERSION are now statically defined. (#290)
• Benchmark tooling updated to lzbench 2.3 with the ZXC 0.13.0 codec.

Documentation

• README: refreshed Silesia benchmarks (lzbench 2.3) across Apple M2, Google Axion (Neoverse-V2), AMD EPYC 9B45 (Zen 5) and 7B13 (Zen 3), plus a new Ultra (Level 7 vs zstd -1) comparison tier.
• MIGRATION: v6 -> v7 transcode guide.
• API / FORMAT / man page: Level 7 (ZXC_LEVEL_ULTRA), the PivCo layout, and the v7 header documented.

Acknowledgements

ZXC's new v7 Huffman bitstream is built on PivCo-Huffman. Sincere thanks to @MarcinZukowski for the technique and its paper, and to @rygorous whose breakdown of the PivCo merge operations made the SIMD decoder practical.

Thanks also to @inikep (lzbench) and @powturbo (TurboBench) for keeping ZXC in the independent benchmark suites, to @guglovich for packaging ZXC for the Arch Linux AUR, and to @jeanga for continued testing and packaging help.

Changelog

• api: Refines code coverage reporting (#322)
• api: Adds Level 7 (Ultra) compression to wrappers (#317)
• api: ZXC format v7: PivCo Huffman, level 7 (ULTRA), space-speed selection, in-place decode (#315)
• api: bump node-addon-api from 8.8.0 to 8.9.0 in /wrappers/nodejs (#308)
• api: Refines documentation and API option descriptions (#299)
• api: Enhances code quality and analysis setup (#294)
• perf: Optimizes compression and decompression performance (#319)
• perf: Optimizes Huffman decoding and compression margins (#301)
• cli: Refines CLI with progress, robust parsing, and file I/O (#318)
• cli: Update README with zxc CLI installation option (#316)
• cli: Enhances code quality and robustness across components (#296)
• cli: Enhance README clarity and quick start
• build: bump pytest from 9.0.3 to 9.1.1 in /wrappers/python (#310)
• build: bump setuptools-scm in /wrappers/python (#311)
• build: bump cibuildwheel from 3.4.1 to 4.1.0 in /wrappers/python (#309)
• build: bump vitest from 4.1.7 to 4.1.9 in /wrappers/nodejs (#307)
• build: Refines code style, type safety, and modernizes JS/Go wrappers (#297)
• build: Refines CI/CD workflows and test stability (#292)
• portability: Add OS and libc compatibility CI for musl Linux and FreeBSD (#291)
• doc: Refactor LZ77 search loop conditions (#298)
• doc: Update README badges
• misc: Ignore vendors code in coverage reports
• misc: Improves robustness and refines memory management (#321)
• misc: Update lzbench benchmark source to v0.13.0
• misc: Caches dict Huffman tree upon attach (#320)
• misc: bump actions/setup-python from 6.2.0 to 6.3.0 (#305)
• misc: bump oss-fuzz-base/base-builder in /.clusterfuzzlite (#312)
• misc: bump msys2/setup-msys2 from 2.31.1 to 2.32.0 (#304)
• misc: bump actions-rust-lang/setup-rust-toolchain (#306)
• misc: bump actions/attest-build-provenance from 4.1.0 to 4.1.1 (#303)
• misc: bump softprops/action-gh-release from 3.0.0 to 3.0.1 (#302)
• misc: Refines code quality and addresses warnings (#295)
• misc: Update bash conditional expressions to '[[ ... ]]' (#293)
• misc: Statically define Meson project version and SOVERSION (#290)

Full Changelog: v0.12.0...v0.13.0

Release Notes

This release introduces pre-trained dictionary compression - Train a dictionary from representative samples and ship dramatically smaller archives for small-block / many-small-file workloads - together with a shared dictionary Huffman table, a static context API for zero-allocation caller-managed workspaces, and a custom reader interface for seekable archives. The container format advances to v6 and the shared library SOVERSION bumps from 3 to 4.

⚠️ Breaking change. This is an ABI and format break. The format version is now v6 (the legacy NUM block type is removed), so v0.12.0 cannot read archives written by v0.11.0 and earlier, and older decoders reject v6 archives. Re-link against libzxc.so.4.

Migrating v5 → v6 archives is a one-time transcode, decompress with your old (v5) build and recompress with the new one:

zxc-v5 -dc old.zxc | zxc-v6 -z -c > new.zxc

Keep a v5 build around until your data at rest is converted (it is the only thing that can read v5). Full guide: bulk migration, dictionaries, verification: docs/MIGRATION.md.

New Features

Pre-Trained Dictionary Compression

For workloads compressed in small blocks (4 KB–128 KB), a pre-trained dictionary prefills the LZ77 sliding window at the start of every block, so even the earliest bytes have representative history to match against. This helps any time the block size is small enough that early bytes would otherwise lack history: a single small payload, or a large payload split into many small blocks.

• Train from a corpus, serialize to a self-describing .zxd, and compress/decompress against it. Dictionaries are capped at 64 KB (ZXC_DICT_SIZE_MAX).
• New public API in zxc_dict.h: zxc_train_dict / zxc_train_dict_huf (train), zxc_dict_save / zxc_dict_save_bound (serialize), zxc_dict_train (one-shot train + serialize), zxc_dict_load / zxc_dict_huf (load), and zxc_dict_id / zxc_dict_get_id (identity).
• Each archive records its 32-bit dict_id; supplying the wrong dictionary (or none) fails cleanly with a dictionary-required / mismatch error instead of silently corrupting output. (#261)

Shared Dictionary Huffman Table

The .zxd carries a 128-byte packed literal Huffman code-lengths table trained on the same corpus. Small blocks no longer each pay to embed their own literal table — the decoder reuses the dictionary's shared table, which is where most of the small-block ratio gain comes from. The dict_id covers both the content and the table, so a single id pins the complete decode state. (#275)

Static Context API (Caller-Managed Workspaces)

New zero-allocation path for embedded and tightly-budgeted integrators: query the required workspace size up front, then hand libzxc a buffer you own.

• zxc_static_cctx_workspace_size / zxc_static_dctx_workspace_size and zxc_init_static_cctx / zxc_init_static_dctx initialise compression/decompression contexts in caller-provided memory: no malloc on the hot path. (#242)

Custom Reader Interface for Seekable Archives

zxc_seekable_open_reader lets seekable random-access decompression run over an arbitrary user-supplied reader callback (network, memory map, custom VFS), not just FILE*. zxc_seekable_set_dict wires dictionary support into the seekable path. (#240)

Dictionary API in Every Wrapper

Full dictionary training, serialization, and dictionary-backed compress/decompress exposed idiomatically across Go, Rust, Python, Node.js, and WASM, backed by an overhauled, hardened dictionary fuzzer. (#269, #270)
The seekable random-access decompression API is now exposed across Go, Rust, Node.js, and WASM via the umbrella header, and added to the Python wrapper. (#224, #250)

CLI: Dictionaries and the unzxc Alias

• --train trains a dictionary from the input files (output path via -o, defaulting to dictionary_<dict_id>.zxd). This renames the former --train-dict PATH.
• -D, --dict FILE compresses or decompresses against a .zxd; -l, --list reports an archive's Dict ID and inspects .zxd files.
• unzxc is a new decompression alias: installed as a symlink to zxc, it defaults to decompress mode (equivalent to zxc -d). (#272)
• Native wildcard expansion on Windows: the CLI expands glob patterns itself on Windows (where the shell doesn't), so zxc *.log behaves as on POSIX. (#284)

Performance & Memory

Measured versus v0.11.0:

• ~10 % faster compression at level 6, from the repeat-offset seed feeding the optimal parser (see below).
• ~3 % faster decompression at levels 1 and 2.

• Repeat-offset seed at L6: the optimal parser seeds match finding with the current repeat offset, accelerating LZ77 search on repetitive data at maximum density. (#257)
• Bucket-sort Huffman leaf ordering: replaces comparison sorting of literal frequencies with a bucket sort during table construction. (#244)
• Dedicated SSE2 SIMD path for x86-64: a sub-AVX2 tier so older / baseline x86-64 CPUs get vectorised decode and match finding instead of falling back to scalar. (#259)
• Short-offset LZ run decoding: faster run/overlap copies for small match offsets, improving decode on highly repetitive data (neutral on Silesia). (#276)

Bug Fixes & Robustness

• Harden decompressor buffer bounds checks against malformed input. (#229)
• Decompressor output-buffer tail padding adjusted to match the fast decoder's overwrite contract. (#249)
• Empty-data compression/decompression handled correctly end-to-end across the wrappers. (#265)
• Enhanced format validation with stricter rejection of malformed frames. (#271)
• Snyk findings addressed and general robustness improvements. (#273)
• Gate AVX2/AVX-512 detection on OS-enabled vector state: feature detection now also checks OSXSAVE/XCR0, so the AVX2/AVX-512 decode paths are used only when the OS has enabled the YMM/ZMM state, preventing illegal-instruction faults on misconfigured systems. (#283)

Development & CI

• Decoder conformance test suite plus a make target to run it, locking in cross-version decode behaviour. (#246)
• Golden-file format-stability tests to catch unintended on-disk format drift. (#256)
• Native Meson build system support alongside CMake. (#245)
• Automated ABI stability check workflow diffs every change against a committed libabigail baseline. (#222)
• Multi-compiler matrix extended with GCC 15 and 16. (#274)
• Format advanced to v6 with the NUM block type removed (#264); the Sans-IO API and frame primitives are now internalised (#225). Codecov action updated to v7 (#267).
• Consolidated ClusterFuzzLite fuzzer builds and runs into a single workflow, simplifying continuous fuzzing CI. (#287)

Documentation

• README: dictionary guide (when small-block workloads benefit and why), specific decode-performance claims, and a format conformance & stability overview.
• FORMAT / man page / EXAMPLES: document the .zxd dictionary format, --train, -D/--dict, and the unzxc alias.
• Doxygen output now includes the README and brief per-symbol descriptions.

Acknowledgements

Special thanks to @jeanga for the help, support, and testing that went into the new pre-trained dictionary mode.

Changelog

• api: Implements shared dictionary Huffman table (#275)
• api: Enhances format validation (#271)
• api: Overhauls dictionary fuzzer for robust testing (#270)
• api: Adds comprehensive dictionary API to all wrappers (#269)
• api: Introduces pre-trained dictionary compression (#261)
• api: bump node-addon-api from 8.7.0 to 8.8.0 in /wrappers/nodejs (#253)
• api: Adds conformance test suite and improves empty frame handling (#246)
• api: Harden decompressor buffer bounds checks (#229)
• api: Introduces static context API for caller-managed workspaces (#242)
• api: Adds custom reader interface for seekable archives (#240)
• api: wrappers: Adds seekable random-access decompression API (#224)
• api: Internalizes Sans-IO API and frame primitives (#225)
• perf: Optimize LZ run decoding for short offsets (#276)
• perf: Add make target to run decoder conformance suite
• perf: Improve Huffman leaf sorting with bucket sort (#244)
• cli: enable native wildcard expansion for CLI on Windows (#284)
• cli: Adds unzxc alias and renames dictionary training option (#272)
• cli: Remove Snyk policy ignore
• cli: Addresses Snyk scan findings and improves robustness (#273)
• cli: Adds native Meson build system support (#245)
• build: bump tar from 7.5.13 to 7.5.16 in /wrappers/nodejs (#286)
• build: bump vite from 8.0.14 to 8.0.16 in /wrappers/nodejs (#285)
• build: Add GCC 15 and 16 to CI multi-compiler matrix (#274)
• build: Update LZbench branch for benchmark workflow
• build: Enables empty data compression/decompression (#265)
• build: Update package descriptions
• build: Introduce dedicated SSE2 SIMD optimization path for x86-64 (#259)
• build: tests: Enforces golden file format stability (#256)
• build: bump vitest from 4.1.6 to 4.1.7 in /wrappers/nodejs (#254)
• build: Restrict SBOM generation to tag pushes
• build: Add qemu cpu targeting for simd dispatch coverage (#238)
• build: Bump cibuildwheel from 3.3.1 to 3.4.1 in /wrappers/python (#233)
• build: Bump vitest from 4.1.5 to 4.1.6 in /wrappers/nodejs (#234)
• build: Use upstream LZbench for benchmarks
• build: Add automated ABI stability check workflow (#222)
• build: Generates SBOM for GitHub releases (#226)
• build: Automate CHANGELOG generation for releases (#220)
• build: Standardizes release artifact structure and naming (#223)
• build: Pins Python wrapper build dependencies (#221)
• build: Pin Python wrapper dependencies with ...

Release Notes

This release introduces Level 6 (ZXC_LEVEL_DENSITY) with Huffman-coded literals and an optimal LZ77 parser, a push-based streaming API for non-blocking integrations, and idiomatic streaming I/O adapters across every official wrapper. It raises the default block size from 256 KB to 512 KB for better ratio and decode throughput.
No ABI breaks: SOVERSION stays at 3.

New Features

Level 6: ZXC_LEVEL_DENSITY

New maximum compression preset combining:

• Canonical Huffman literal encoding (8-bit length-limited). Decoder uses a 2048-entry multi-symbol lookup table over an 11-bit window, each entry decodes 1 or 2 symbols, with a 4-way interleaved bitstream. Selected only when Huffman saves >= 3 % over RAW/RLE.
• Optimal LZ77 parser based on dynamic programming over per-position cost estimates. Scratch is allocated lazily per cctx and reused across blocks.

The Huffman codec is integrated into the function-multi-versioning system (default / NEON / AVX2 / AVX-512). Format spec updated with enc_lit=2. (#208, #214)

Push-Based Streaming API (zxc_pstream.h)

Caller-driven, single-threaded streaming API for environments where blocking on FILE* is not possible: async event loops, callback-driven libraries, network protocols. Uses explicit zxc_inbuf_t / zxc_outbuf_t descriptors, allowing input/output in arbitrary chunks. Output is bit-identical to the Buffer and Stream APIs. Bindings provided in Go, Rust, Python, Node.js, and WASM, with a dedicated fuzzer. (#204)

Streaming I/O Adapters in Every Wrapper

Idiomatic streaming adapters across all wrappers, plus a magic-word detection utility for content-type sniffing.

• Go: io.Reader / io.Writer / io.Closer. (#209)
• Python: ZxcReader / ZxcWriter (io.RawIOBase). (#210)
• WHATWG TransformStream: for fetch().body.pipeThrough(), Deno, Bun, Node.js. (#211)
• Node.js stream.Transform. (#212)
• Rust std::io: Encoder / Decoder. (#213)

Block API Safety Helpers

Introduces zxc_decompress_block_bound (minimum destination capacity required by the fast decoder) and zxc_decompress_block_safe (strict-sized variant for page-aligned or exactly-sized buffers where tail padding is impossible). (#198)

Compression Context Memory Estimation

zxc_estimate_cctx_size lets integrators with tight memory budgets compute the full context footprint before allocation. Exposed too in the Rust and Go wrappers. (#198)

Automatic Block Size in Block API

zxc_compress_block now automatically rounds the effective block size up to the next power of two that fits the source when no explicit block_size is provided. (#198)

Benchmarks vs v0.10.0

Silesia corpus, lzbench 2.2.1, single-threaded, -march=native.

Compression Ratio

Smaller output at every level:

• -1: 61.56 % → 61.50 % (−0.10 %)
• -2: 54.00 % → 53.61 % (−0.72 %)
• -3: 46.35 % → 45.79 % (−1.21 %)
• -4: 43.14 % → 42.65 % (−1.14 %)
• -5: 40.67 % → 40.27 % (−0.98 %)
• -6 (new): 36.28 %. ~10.8 % smaller than L5

Decompression Throughput

• Apple M2 (ARM64): L1 12 195 → 12 530 MB/s (+2.7 %), L2 +3.1 %, L3–L5 +0.6 to +1.4 %.
• Google Axion (ARM64): L1 8 924 → 9 067 MB/s (+1.6 %), L2 +0.8 %, L3–L5 within ±0.3 %.
• AMD EPYC 9B45 (x86_64): L1 10 803 → 10 844 MB/s (+0.4 %), L2 +0.6 %, L4–L5 −1.1 to −1.3 %.
• AMD EPYC 7763 (x86_64): L1 6 921 → 7 077 MB/s (+2.3 %), L2 +2.1 %, L5 +1.7 %.

Compression Throughput

Fast levels accelerate sharply on Axion / x86 thanks to the tag-first hash filter; mid levels trade a few % for the ratio gains:

• L1: M2 -3.1 %, Axion +7.2 %, EPYC 9B45 +9.1 %, EPYC 7763 +6.5 %.
• L2: M2 -2.3 %, Axion +12.0 %, EPYC 9B45 +16.1 %, EPYC 7763 +11.1 %.
• L3–L5: -1 to -5 % across all platforms, cost of the tighter ratio target and DP-based decisions feeding the parser.
• L6 (new): ~9-12 MB/s. ~10× slower than L5 by design, paying compression CPU once to ship a smaller artifact to millions of decoders.

Performance & Memory

• Tag-first hash filter for fast levels: at ZXC_LEVEL_FAST and below, an early hash-tag check short-circuits the position lookup on tag mismatch, with speculative prefetch of the next entries. (#215)
• NEON nibble masks via SHRN: 128-bit byte comparison masks consolidated into 64-bit nibble masks. (#208)
• Vectorized optimal-parser DP updates: AVX-512 / AVX2 / NEON for constant-cost match-length updates. (#208)
• Bulk-flush Huffman bit writer + cache-line-aligned tables. (#208)
• Chain table as a 64 KB ring buffer: entries beyond the 64 KB uint16_t delta reach were never read back. Saves over 3.8 MB per cctx at the 2 MB block size with no change in ratio or throughput. (#198)
• Aliased sequence/offset/token buffers: GLO and GHI paths are mutually exclusive per block; the three pointers now share a single region. (#198)
• Tighter max-sequence bound: chunk_size / ZXC_LZ_MIN_MATCH_LEN. (#198)
• Unified fast/safe decoders via a compile-time constant, eliminating duplicated logic with zero-cost rollback in the fast path. (#198)

Bug Fixes & Robustness

• TSan data race in stream engine shutdown: the async writer read job->result_sz outside the lock while the main thread wrote the shutdown sentinel under it. Fixed by snapshotting result_sz and in_sz under the lock in the writer. (#201)
• Varint truncation for chunk > 2 MB: zxc_write_varint was capped at 3 bytes, silently truncating literal runs or match lengths above 2 MB. Extended to 5 bytes to match the decoder. (#198)
• Block API auto-resize: no longer rejects inputs larger than the explicit block_size; rounds up to the next power of two. (#198)
• Block API re-init safety: guards against zero-capacity destinations to prevent underflow. (#198)
• Harden decompression logic against integer overflows on 32-bit platforms. (#202)
• Pstream input validation: comprehensive bounds and NULL-pointer checks; rejects malformed buffer descriptors; footer validation distinguishes corruption from checksum failures. (#204)
• Wrapper decompression flush: Python, Node.js, and WASM dstream loops now drain internal buffers after user input is exhausted. Output buffer growth checks max-capacity overflow. (#204)

Development & CI

• Seekable API fuzzer (fuzz_seekable.c) exercising range reads, metadata getters, multi-threaded decompression, and raw SEK block parsing. All fuzzers now use persistent static buffers and a 4 MB input cap. (#196)
• Pstream API fuzzer targeting the state machine, chunk boundaries, and parser logic of the push-based streaming API. (#204)
• Windows ARM64 Go CI: windows-11-arm runner with llvm-mingw toolchain (aarch64-w64-mingw32-gcc). (#195)
• Monolithic test suite split: per-API files, individual CTest entries (96 instead of one aggregate). (#200)

Documentation

• Block API safety helpers: documents zxc_decompress_block_bound, zxc_decompress_block_safe, and zxc_estimate_cctx_size, explaining the tail-pad requirement of the fast decoder. (#198)
• Format spec: enc_lit=2 Huffman literal section.
• API: documents the push-based streaming API and the new ZXC_LEVEL_DENSITY preset.
• Nim bindings: community-maintained zxc-nim added to the README's examples and language support table, thanks to @georgelemon
• Free Pascal bindings: community-maintained Free-Pascal-port-of-ZXC-compressor-decompressor added to the README, thanks to @Xelitan
• TurboBench verification: documents TurboBench as an industry-standard benchmark suite that has officially merged zxc into its master branch.

Changes

Full Changelog: v0.10.0...v0.11.0

Release Notes

This release introduces seekable archives for random-access decompression, WebAssembly support for browser and server-side JavaScript, a block-level API for filesystem integrations, a runtime library information API, and an official Go wrapper. It improves stream engine robustness with thread-safety fixes and includes several breaking CLI changes (default extension, checksum behavior, pkg-config module name). It also delivers significant encoder optimizations (split hash table, semi-branchless varint decoding, unified literal copy fast path, BMI/LZCNT extensions, parameterized lazy matching, SIMD register optimization) and internal code quality improvements. SOVERSION is incremented to 3.

Breaking Changes

• Default file extension changed to .zxc: Compressed files now use the .zxc extension instead of the previous .xc suffix. File detection, recursive processing, and documentation have been updated accordingly. (#143)
• pkg-config module renamed to libzxc: Follows the naming convention used by libzstd, liblz4, liblzma, libbrotli. Consumers should update their build files to use pkg_check_modules(LIBZXC IMPORTED_TARGET libzxc). (#144)
• Checksums enabled by default: The CLI now enables checksums for all operation modes except --bench, where they remain disabled to avoid measuring checksum overhead. (#142)
• SOVERSION incremented to 3: The shared library SOVERSION is bumped from 2 to 3 due to ABI-breaking changes (new seekable API, block-level API, runtime info API). Consumers linking against the shared library (libzxc.so.3 / libzxc.3.dylib) must rebuild. Static library users are unaffected.

New Features

Seekable Archives

Introduces seekable ZXC archives for random-access decompression. When seekable = 1 is set in zxc_compress_opts_t, the compressor appends a seek table block (ZXC_BLOCK_SEK) containing per-block compressed sizes, enabling O(1) block lookup and byte-range decompression without scanning the full stream.

New APIs: zxc_seekable_open, zxc_seekable_open_file, zxc_seekable_read, zxc_seekable_close, zxc_seekable_get_num_blocks, zxc_seekable_get_decompressed_size. (#188)

WebAssembly (WASM) Support

Adds an Emscripten-based WebAssembly build target, enabling browser and server-side JavaScript compression. Includes a JavaScript API wrapper for buffer-based operations, Node.js roundtrip tests, and a GitHub Actions workflow for automated WASM builds on push and release events. (#189)

Block-Level API

Introduces zxc_compress_block, zxc_decompress_block, and zxc_compress_block_bound for single-block compression and decompression without file headers, footers, or EOF blocks. This API is designed for filesystem integrations (DwarFS, EROFS, SquashFS) where the caller manages its own block indexing. (#148)

Runtime Library Information API

Introduces zxc_min_level, zxc_max_level, zxc_default_level, and zxc_version_string to query the supported compression level range and library version string at runtime. This allows callers, such as filesystem integrations, to discover metadata and capabilities without relying on compile-time constants alone. (#163)

Go Wrapper

Introduces the official Go package for zxc, located in wrappers/go. Built on CGo, it exposes two idiomatic APIs:

• Buffer API: for in-memory compression and decompression.
• Streaming API: for large files, backed by the multi-threaded stream engine with separate reader, worker, and writer threads.

Typed sentinel errors map every C error code to a named Go error value. Cross-platform CI (ubuntu-latest, ubuntu-24.04-arm, macos-latest, windows-latest) runs both tests and benchmarks on every release via the Test Go Package workflow. (#149)

Scalar-Only Build Option

Introduces the ZXC_DISABLE_SIMD CMake option to bypass all hand-written SIMD intrinsics and inline assembly, forcing the library into a scalar-only execution path. Compiler auto-vectorization remains unaffected. This is useful for baseline performance benchmarking, portability testing, and security auditing of the scalar code paths. (#174)

Performance

• Split hash table architecture: Separates the LZ77 hash table into a position array (uint32_t, 128 KB) and an 8-bit tag array (uint8_t, 32 KB) with 15-bit addressing (32 768 buckets, up from 16 384). The compact tag array fits in L1 cache and rejects mismatches before loading positions, while the doubled bucket count shortens chain walks. (#179)
• Semi-branchless varint decoding: Replaces conditional branching in the variable-byte integer decoder with look-up tables and bitwise operations, reducing branch mispredictions in the decompression hot path. Also standardizes bit-width constants to use CHAR_BIT from <limits.h>. (#186)
• Unified 32-byte literal copy: Replaces conditional 16-byte and 32-byte literal copies with a single 32-byte copy when buffer padding allows, reducing branching in the compression hot path. (#181)
• Varint encoding simplification: Removes unreachable 4-byte and 5-byte varint encoding paths (bounded by the 2 MB block size limit), replaces hardcoded integer limits with bit-shift constants, and simplifies lazy evaluation naming for clarity. (#183)
• Decoupled LZ77 lazy evaluation: Improves the parallelism of lazy evaluation checks within zxc_lz77_find_best_match by evaluating ip+1 and ip+2 positions independently before a single decision, enhancing match finding efficiency. (#177)
• Refined numeric data detection: The zxc_probe_is_numeric function now employs multi-region sampling, analyzing both the start and middle of a block to more accurately determine if data is suitable for numeric compression. (#177)
• Optimized decompression loop: The safe loop for sequence decoding in zxc_decode_block_ghi is now 4x unrolled to process multiple sequences concurrently, significantly boosting decompression speed. (#177)
• SIMD register optimization: Eliminates GPR round-trips in SIMD batch loops across all platforms. On ARM64, uses vdupq_laneq_u32 for direct NEON lane broadcast. On AVX2, uses _mm256_permute2x128_si256 + _mm256_shuffle_epi32. On AVX-512, uses _mm512_shuffle_i32x4 + _mm512_shuffle_epi32. This keeps the running sum in vector registers throughout the decode batch loop, reducing instruction latency. (#164)
• ARM64 encoder vectorization: Leverages vminvq_u8 and vmaxvq_u8 vector reduction intrinsics for 16-byte block uniformity checks during RLE and literal scanning, avoiding premature extraction to general-purpose registers. Also removes an unnecessary prefetch in the LZ77 match-finding loop. (#164)
• Encoder micro-optimization: Extracts offset_bits and offset_mask from the context structure into local constants in zxc_encode_block_glo and zxc_encode_block_ghi, reducing repeated member access in performance-critical paths. (#141)
• BMI/LZCNT extensions: Optimizes zxc_log2_u32 with platform-specific intrinsics (_BitScanReverse, __builtin_clz) and enables BMI1/BMI2/LZCNT instruction set flags for AVX2 and AVX512 build variants. Fixes potential undefined behavior in 64-bit shift calculations for offset mask and epoch values. (#159)
• Parameterized lazy matching threshold: Replaces the hardcoded match length limit (128) for lazy evaluation with a configurable parameter in zxc_lz77_params_t, allowing each compression level to independently tune the trade-off between search depth and encoding speed. (#158)
• I/O buffer alignment: Replaces hardcoded buffer constants in the CLI with the ZXC_IO_BUFFER_SIZE macro (1 MB), ensuring consistent I/O buffer sizing during file processing. (#162)
• Decompression macro refactor: Introduces DECODE_COPY_LITERALS and DECODE_COPY_MATCH sub-macros to deduplicate the literal and match copying logic across SAFE and FAST decoding paths in both glo and ghi block decoders. (#150)

Bug Fixes & Robustness

• Improved bit reader robustness: Corrects edge-case handling in the bit reader's zxc_br_ensure function, ensuring proper behavior when working with 64-bit masks and shifts.
• Stream engine synchronization: Destroys mutexes and condition variables after worker threads have joined to prevent resource leaks. (#146)
• Thread creation failure handling: Properly destroys synchronization primitives and joins started threads when thread creation or context initialization fails, preventing resource leaks and deadlocks. (#146)
• Worker thread data races: Job status and result size updates in worker threads are now performed while holding the ...