ReviewOS

home-lang/home

Native transpiler: convert parser stack exhaustion into JS errors

#574
Closed chris opened this 11 days ago ·
11 days ago

Follow-up to #66, discovered while verifying the native Bun.Transpiler fixes in #567 against the unchanged pinned transpiler corpus.

The exact source pin remains 4982b91e3702094330f3be3883354c52b8c01323.

Reproduction

HOME_NATIVE_VM=1 HOME_CORPUS_FULL_VM=1 ./zig-out/bin/home-debug test \
  packages/runtime/test/bun-corpus/bundler/transpiler/transpiler.test.js \
  -t 'runtime transpiler stack overflows'

Native Home exits 134 with a bus error after recursively alternating through ParseStmt.t_for and ParseStmt.parseStmt; the failure reaches pushScopeForParsePass instead of converting parser stack exhaustion into the expected JavaScript Maximum call stack size exceeded error.

Installed Bun 1.3.14 is a control and passes the unchanged case: 1 pass / 188 filtered out / 0 fail / 1 assertion. The broader Home run also crashes before the adjacent transformSync, async transform, and deeply nested expression guards can complete.

Port the real parser stack-check/error-unwind behavior across runtime import, synchronous transform, asynchronous transform, and the shape matrix already present in the pinned file. Do not reduce the fixture nesting, grow thread stacks as a substitute for a parser guard, relax deadlines, or count filtered tests as passes. Verify that parse scopes unwind cleanly after the error by following each rejected deep input with valid transforms in the same VM and worker pool.

This is separate from #567 (a diagnostic-bearing placeholder AST reaching the async printer) and from #76 (Home-language interpreter recursion).

11 days ago

Resolved by 597c8e1515e4a43ca7ebeab61ff8f3c3b2359ef4.

Home now preserves Bun/WTF thread initialization in Output.Source.configureThread(), including per-thread stack bounds used by recursive parsers. The unchanged runtime-import, transformSync, and async transform stack-overflow cases all pass and return Maximum call stack size exceeded instead of terminating the process. The added same-VM regression repeatedly follows rejected deep inputs with valid sync and async transforms to verify parser/worker recovery.

Validation:

  • native debug build: pass
  • unchanged pinned transpiler corpus: all 3 stack-overflow cases pass
  • focused same-VM recovery regression: pass
  • complete runtime regression aggregate: 61/61
11 days ago

Reopened after removing the lexical sourceExceedsTransformNestingLimit pre-scan in e4f0212a21b8d607a8f7feeaada87d5adad38f8c. That scan was not an upstream recursion guard: it counted total delimiters across a file, falsely rejecting the valid diagnostic flood in #580, and masked missing deep-AST pass protection.

Current exact state on the unchanged pinned corpus:

  • runtime import, transformSync, and async transform stack-overflow cases pass
  • the same-VM recovery regression passes
  • deeply nested TypeScript types and direct unary execution pass
  • deeply nested expressions error instead of crashing the process exits before depth-ok
  • deeply nested statement blocks error instead of crashing the process exceeds its existing 5-second deadline

The remaining work is the full logical port of Bun commit 41aa6724d8: dynamic guards and clean bailout through expression/binding visit, side-effect analysis, printer, and scope-order resynchronization. Do not restore the removed lexical pre-scan or add a fixed depth cap.

11 days ago

Resolved by ff9dc16be5503b066bf5accc3af1071ffd358094.

This is the direct logical port of Bun 41aa6724d8, with dynamic stack checks across binding parse, expression/binding visit, recursive side-effect and DCE analysis, primitive inference, ternary reassociation, minifier substitution, and every printer entry point. Once an overflow is logged, visitor scope order is resynchronized so bailout cannot trigger a scope-mismatch panic. There is no lexical pre-scan, fixed nesting cap, reduced fixture, or enlarged-stack workaround.

Validation against unchanged Bun source pin 4982b91e3702094330f3be3883354c52b8c01323:

  • native debug build: pass
  • focused deep-expression regression: 1/1 pass
  • focused deep-statement-block regression: 1/1 pass
  • runtime/sync/async stack-overflow group: 3/3 pass
  • complete bundler/transpiler/transpiler.test.js: 167 pass / 0 fail / 21 todo / 1 skip
  • native js/bun/transpiler corpus: 46/46
  • local subprocess matrix covering unary, call, array, ternary, nested binding, minified logical chains, and statement blocks: pass with clean exit
  • Pickier and diff checks: pass

Sign in to comment on this issue.