ReviewOS

home-lang/home

Bun parity: preserve TextEncoderStream split-surrogate state

#334
Closed chris opened this 10 days ago ·
10 days ago

Scope

Port the remaining Bun/WHATWG TextEncoderStream behavior at corpus position 4103 (js/web/encoding/text-encoder-stream.test.ts).

Current failure

a character split between chunks should be correctly encoded expects one reassembled UTF-8 output chunk, but Home’s corpus stream shim emits two replacement-character chunks.

Acceptance

  • preserve incremental encoder state across split UTF-16 surrogate chunks
  • emit the correct chunk boundaries and UTF-8 bytes
  • retain structured stream errors with operation/cause/stack context
  • pin positions 4102 and 4103 in the permanent HTTP/web tail queue
  • focused corpus, permanent queue, and clean build pass

Tracking from #66.

10 days ago

Completed in e58dcb2 (fix(streams): preserve encoder surrogate state) and pushed directly to main as Chris Breuer.

What landed:

  • faithful pending-lead-surrogate state across TextEncoderStream chunks
  • valid cross-chunk surrogate reassembly
  • correct U+FFFD placement for invalid/unmatched sequences
  • empty-chunk carry behavior and final flush replacement
  • original stream error identity retained while adding ERR_TEXT_ENCODER_STREAM, operation, and source stack context
  • positions 4102 and 4103 pinned in the permanent HTTP/web tail queue

Verification:

  • position 4102 text-decoder.test.js: 77 passed
  • position 4103 text-encoder-stream.test.ts: 19 passed
  • permanent HTTP/web tail queue: passed
  • clean project build: passed

Cumulative contiguous corpus through position 4103: 444 files, 5,994 passed, 270 todo, 172 allowed-empty, 0 failed, 0 unsupported.

Sign in to comment on this issue.