ReviewOS

home-lang/home

Bootstrap corpus runner: stop recursive microtask growth in expansion-tail encoding cases

#626
Closed chris opened this 10 days ago ·
10 days ago

Problem

The full zig build test -Denable_jsc=true umbrella run completes the native runtime count but then hangs in bootstrap runner mirrors expansion queue tail mini-suite. The process remained live for more than one hour, consumed two cores, and retained 16.3 GiB (17.8 GiB peak) before being stopped.

A macOS sample identifies the active path as:

  • corpus_runner.test.bootstrap runner mirrors expansion queue tail mini-suite
  • runFile / runRelativeFile
  • jsc_bootstrap.Runtime.runFile
  • JSEvaluateScript
  • JSLock::willReleaseLock
  • VM::drainMicrotasks
  • a deeply recursive JavaScript microtask chain ending in repeated Number valueOf errors and GC

Isolation

The first expansion-tail case that does not complete is the unchanged corpus copy:

zig-out/bin/home-debug test packages/runtime/test/bun-corpus/js/web/encoding/text-encoder.test.js

It exceeds 12 seconds and rapidly grows memory, while the same source addressed at the pinned Bun checkout completes normally:

zig-out/bin/home-debug test /Users/chris/Code/bun/test/js/web/encoding/text-encoder.test.js

The later unchanged js/node/util/node-inspect-tests/parallel/util-inspect-long-running.test.mjs corpus-root path also exceeds 12 seconds, although its pinned-checkout path completes in about 0.3 seconds. This suggests a bootstrap/corpus path or realm teardown distinction, not a fixture defect.

Acceptance criteria

  • bootstrap runner mirrors expansion queue tail mini-suite completes with its current expected counts.
  • The unchanged text-encoder file completes with 42 passes through the corpus/bootstrap path.
  • The unchanged util-inspect long-running file completes with 1 pass through the same path.
  • Microtask draining is finite and runtime memory returns after each file.
  • No corpus file, fixture, assertion, deadline, depth, or workload is reduced.
  • Re-run the wider umbrella suite beyond this previously blocking point.

Parent parity tracker: #66.

10 days ago

Progress landed in 6ce4fe1 (fix(runtime): accelerate encoding and circular inspection).\n\nImplemented:\n- Native TextEncoder.encodeInto over UTF-16 input and Uint8Array backing storage, including astral boundary semantics.\n- Native non-streaming, non-fatal UTF-8 TextDecoder path with byte-view normalization for ArrayBuffer, DataView, and typed arrays; NUL and replacement characters are preserved through UTF-16 JSC string creation.\n- Bun-compatible indirect circular-reference collapsing in the shared util.inspect implementation.\n\nUnchanged corpus verification:\n- text-encoder.test.js: 42 pass, 0 fail\n- text-decoder.test.js: 77 pass, 0 fail\n- text-decoder-cjk.test.ts: 30 pass, 0 fail\n- text-decoder-single-byte.test.ts: 13 pass, 0 fail\n- deno encoding.test.ts: 21 pass, 0 fail, 2 existing todo\n- util-inspect-long-running.test.mjs: 1 pass, 0 fail\n- broader internal-inspect, util-inspect, and util-format focused files: 0 failures\n\nThe inspect stress case improved from 31.94 seconds / 7.35 GB max RSS to about 1.6 seconds / 616 MB max RSS. The encoder case now finishes in about 19 seconds instead of running for more than an hour in the umbrella process.\n\nThe production debug build also passes. No corpus sources, fixtures, assertions, deadlines, depths, or workloads changed.\n\nKeeping this issue open: the build command ignored the requested Zig test-name filter and launched the full home_test artifact, so the complete expansion-tail and wider umbrella acceptance runs still need an honest traversal. That accidental run was stopped after sampling showed it had moved into the unrelated node URL-contract test.

10 days ago

Completed by 6ce4fe1 and 399bb5f.\n\nThe full unfiltered home_test artifact now traverses the expansion-tail mini-suite and all later corpus-contract tests instead of hanging. Final aggregate: 1,330 passed, 1 skipped, 118 failed out of 1,449 Zig tests in 24m18s. The remaining failures are concrete independent parity gaps.\n\nDuring the run, live RSS repeatedly returned to roughly 230-300 MB between high-allocation stress files after restoring JSGlobalContextRelease in test builds. The prior test-only release suppression kept every corpus realm alive and caused monotonic growth.\n\nAcceptance evidence:\n- expansion-tail suite completed in the wider artifact\n- unchanged text encoder: 42 pass, 0 fail\n- unchanged long-running util.inspect: 1 pass, 0 fail in about 1.6 seconds\n- all 183 focused encoding assertions pass, with 2 existing todos\n- no corpus source, fixture, assertion, deadline, depth, or workload changed\n- production debug JSC build passes\n\nThe full artifact still has high transient peak allocation in exhaustive stress files, but it no longer stalls or retains each completed realm. Closing this focused blocker; remaining failures continue under #66 and focused follow-ups.

Sign in to comment on this issue.