ReviewOS

home-lang/home

Port Node zlib stream accounting and round-trip contracts

#204
Closed chris opened this 11 days ago ·
11 days ago

Tracks the ordered Bun corpus block beginning at position 3904, js/node/zlib/bytesWritten.test.ts.

Current executable boundary after abec8c035:

  • first failing contract: createGzip piped into createGunzip emits an empty result instead of the original payload
  • focused result: 1 file, 0 passed, 1 failed, 0 TODO, 0 unsupported
  • preceding checkpoint: positions 3659-3903 are green at 245 files, 1,263 passed, 113 upstream TODO, 0 failed, 0 unsupported, 165 allowed-empty

Acceptance:

  • implement logical zlib transform output and bytesWritten accounting
  • advance through the ordered zlib files without Home-injected skips or TODOs
  • add permanent regression coverage for completed contracts
  • keep master tracker #66, ordered tracker #201, and TODO/skip tracker #202 cross-linked and current
11 days ago

Zlib stream accounting checkpoint landed in https://github.com/home-lang/home/commit/76d8be478.

Implemented:

  • reset-safe max-output-length handling for gzip/zstd decompression
  • single coherent deflate/deflateRaw/Brotli frame emission across repeated flushes
  • exact decompressor bytesWritten accounting with trailing input, including bounded Brotli consumed-prefix detection
  • permanent zlib regression gate covering bytesWritten.test.ts (5), deflate-streaming.test.ts (1), and leak.test.ts (8)

Verification:

  • permanent timer/TLS, URL, util, V8, watch, worker, zlib, and VM gates all pass
  • positions 3659-3906: 248 files, 1,277 passed, 113 upstream TODO, 0 failed, 0 unsupported, 165 allowed-empty

Next boundary is position 3907, js/node/zlib/zlib-estimated-size-gc.test.ts. The first failure is Brotli construction-mode memory accounting: estimated size is 176 but the upstream contract requires greater than 5,000. Keeping this issue open for that native memory-accounting work.

11 days ago

Second zlib checkpoint landed in https://github.com/home-lang/home/commit/52b661f39.

Implemented:

  • construction-mode native footprint accounting for Brotli and Zstd handles, stable across close and concurrent GC
  • zlib handle writeSync input/output bounds validation
  • detached _writeState backing-store safety and completion accounting
  • global Buffer.kMaxLength enforcement for opaque standard DEFLATE and Home-framed output
  • expanded permanent zlib gate through position 3911

Newly green files:

  • zlib-estimated-size-gc.test.ts: 6
  • zlib-handle-bounds-check.test.ts: 9
  • zlib-onerror-reentrancy.test.ts: 3
  • zlib-reset-race.test.ts: 3
  • zlib.kMaxLength.global.test.js: 8

Cumulative positions 3659-3911: 253 files, 1,306 passed, 113 upstream TODO, 0 failed, 0 unsupported, 165 allowed-empty.

Next boundary is position 3912, js/node/zlib/zlib.test.js. The first failure is constructor prototype inheritance: Gzip.prototype is not currently instanceof Gzip.proto.

11 days ago

Completed by commit 05e1acbf8.

This finishes the Node zlib checkpoint end to end:

  • restored the exact Zlib/Brotli/Zstd constructor and Transform inheritance graph
  • added safe integer/range/default handling for classic, Brotli, and Zstd options
  • made decompressors stop at the first complete frame and preserve exact bytesWritten across trailing-write variants
  • implemented strict truncated-frame errors plus Z_SYNC_FLUSH partial-prefix decoding
  • pinned input/output ArrayBuffers during asynchronous native writes and released them before completion callbacks
  • routed filesystem streams through a binary-safe native byte bridge
  • moved Brotli decompression to the linked C streaming decoder with bounded output
  • wired real libzstd compression/decompression into the JSC bridge, including external frame interoperability and output limits
  • added js/node/zlib/zlib.test.js to the permanent zlib gate

Verification:

  • focused file: 378 passed, 2 upstream TODO, 0 failed, 0 unsupported
  • cumulative positions 3659–3912: 254 files, 1,684 passed, 115 TODO, 0 failed, 0 unsupported, 165 allowed-empty
  • all permanent timers/TLS, URL, util, V8, watch, worker, zlib, and VM gates passed

The two focused TODOs are the source file's explicit Zstd skipIf variants (documented there as failing in Node too), not Home-injected skips. Closing this focused issue as complete; overall parity continues in #66 and injected-skip accounting remains tracked in #202.

Sign in to comment on this issue.