Native compression: pin async buffers and refresh write-result storage

Follow-up to #66 and #513. Target: exact Bun pin 4982b91e3702094330f3be3883354c52b8c01323.

Status

Implemented in 095f90396, authored and committed by Chris Breuer without coauthors. The commit is integrated in main. This issue remains open for other-platform verification; full compression or whole-suite parity is not claimed.

Implementation

Zlib, Brotli and Zstd now validate bounds/state before pinning input and output backing stores. The slices are resolved again after pinning, and generated cached fields independently keep both JavaScript views alive through GC. Completion unpins and clears the old pending roots before any error/write callback, so a reentrant write retains its own ownership. Output-pin failure unwinds the input pin; shared input/output storage receives balanced pins.

Write-result storage is cached as a JavaScript typed array instead of a raw pointer. Both synchronous and asynchronous completion re-resolve its current backing storage and skip detached or undersized views. The class schema matches the exact pin, and the final binary contains all 18 cache accessors plus the pin/unpin bridge.

Verified on Darwin

• Complete unchanged zlib.test.js: 378 pass / 2 skip / 0 fail / 434 assertions.
• Unchanged zlib-handle-bounds-check.test.ts: 9 pass / 0 fail / 13 assertions, including detached-writeState coverage.
• Unchanged zlib-onerror-reentrancy.test.ts: 3 pass / 0 fail / 9 assertions.
• New cross-codec regression: transfer blocking/release, shared-storage pin balancing, validation and concurrent-write rejection, GC retention during work and collection after completion. 20 consecutive runs pass with JSC exception checks, plus a passing VM-destruction run. The published parent fails the first input-pin assertion.
• 55/55 runtime regression commands pass, including all 24 native regression files, complete spawn and child-process suites.
• Runtime unit suite: 1,819 pass / 19 skip / 0 fail. Native build: 17/17 steps. Focused Pickier and scoped whitespace checks pass.

No upstream test or fixture, assertion, workload or deadline was changed. Skips are excluded from completed parity. Local evidence: zig-out/compression-lifetime-{build-final,unit,regressions,stress,upstream-final}*, compression-lifetime-parent.json, and compression-lifetime-source-integrity.json. Implementation and verification boundaries are documented in docs/docs/NATIVE_RUNTIME_BINDINGS.md.

Historical reproduction

Before this fix, the unchanged full zlib file reported 377 pass / 2 skip / 1 fail / 429 assertions. Its async write buffer lifetime case detached output storage during an in-flight native write. The source audit also identified the stale raw write_result pointer; the pinned bounds suite now verifies its detached-storage path.

Remaining scope

• Port Zlib/Brotli/Zstd async pinning and GC ownership.
• Release old ownership before reentrant callbacks.
• Re-resolve cached write-result storage at completion.
• Verify unchanged lifetime, bounds, reentrancy and full zlib tests plus cross-codec GC/detachment regressions.
• Integrate the published branch.
• Verify Linux/Windows and broader compression coverage.

Remaining stream/HTTP work: #514. Full port: #66.

Published 095f90396 on codex/runtime-port as Chris Breuer, without coauthors. #516 now implements async buffer pinning/GC ownership and fresh write-state resolution across Zlib, Brotli and Zstd.

Verified in native Home against the unchanged pinned tests:

• Full zlib: 378 pass / 2 skip / 0 fail / 434 assertions (previously 1 lifetime failure).
• Bounds 9/9; reentrancy 3/3.
• 55/55 regression commands, including 24 native files.
• Runtime units 1,819 pass / 19 skip / 0 fail; build 17/17.
• 20/20 cross-codec lifetime runs with JSC exception checks, plus VM destruction. The published parent fails the new pinning control.

The #66, #513 and #516 bodies now reflect these results. No upstream tests or fixtures were changed, and skips do not count as completed parity. The branch is not merged; other platforms, #514's four stream-file gaps and the remaining broader port are still open. 100% logical Bun-suite compatibility is not achieved.

Integration status corrected on 2026-09-21: both dc6231ea44034b5ec8a52193623b67a8516e5f07 and 095f9039692e547b5fc07f6c6fe6080d01e827b4 are ancestors of current main (58a2f2b427e6038bd8b103133cbec042551c9865). The unchanged full js/node/zlib/zlib.test.js passes on the current ReleaseFast Home binary at 378 pass / 2 upstream zstd skips / 0 fail / 434 assertions. Integration is complete; this issue remains open only for its stated broader/platform verification boundary.

Source d1843551a now owns ten additional registry modules: crypto, zlib, zlib/iter, promisification and six legacy stream entry points. The explicit ownership list is 90 modules plus the internal test adapter. Home compiles the StringDecoder implementation and factory with an exact linked class-header ABI. Generation rejects duplicate/missing factory includes and source ownership before output.

The original corpus exposed real missing behavior: the generator's keepNames option was in the wrong object; SHAKE's capability export was a false-returning stub; streaming signature requests needed one-shot/key-agreement capability validation. These are fixed. Forced-drain callbacks enter an event-loop scope and nested Debug scopes restore parent state correctly. Pinned zlib getter values survive close/allocation paths. Valid cryptographic operations still execute through the backend.

The complete 193 unchanged original entries validate 1,357 registered passes + 147 process checks, zero failures. 24 registered skips, one TODO, three skipped files and 16 comment-only entries receive no passing credit. All ten native groups pass, including compression buffer lifetime and crypto/compression worker shutdown. ABI 74 tests / 1,220 assertions; Debug 34/34, peak 5,658 MB under 8,192 MB; complete tests 2,383 MB under 3,840 MB. Pickier and scoped formatting/whitespace checks pass.

All 3,785 committed archives verify; all 193 completed reports restore and validate. Initial four-file failures and the repair's remaining key-agreement failure are retained. A direct zlib diagnostic passed before the callback-scope fix; the initial panic is not claimed to reproduce in every context. Every selected test matches the pin. Four pre-existing URL/ICU fixture differences outside this selection are recorded. Unsupported/unapplied Node/V8 metadata flags receive no feature-parity credit. Source bytes remain unchanged across main integration.

This advances #66, #513, #516 and #587. Remaining registry/native dependencies, original pending outcomes, whole-suite and Release/ASAN/all-platform verification stay open.

The Linux/Windows and remaining broader verification requirements on this issue remain open.