ReviewOS

zig-utils/zig-js

Test262: reject unknown or incomplete runner modes

#620
Closed chris opened this 8 days ago ·
8 days ago

Parents: #491, #493 Related: #617

Problem

The Test262 runner fell through to the full default corpus for every unrecognized first argument. A harmless-looking test262 --help therefore launched the complete parent driver and worker tree. Recognized modes with missing operands silently returned success. This could create long-lived hidden corpus work, contaminate evidence jobs, and turn CLI mistakes into expensive host load.

This was observed directly while validating #618/#619: an attempted --help probe spawned a class-expression subtree driver, remained alive after the calling tool returned, and invalidated an otherwise complete parent/candidate comparison.

Scope

  • Preserve no-argument execution as the intentional full-corpus entry point used by zig build test262.
  • Add explicit --help usage output that exits successfully without opening the corpus or spawning workers.
  • Reject unknown options/modes before runParent.
  • Reject missing required operands and malformed numeric worker ranges with a nonzero result instead of silently returning or coercing to zero.
  • Reject unexpected trailing operands for fixed-arity modes; retain the documented optional --diag filter and variadic --vm-witness paths.
  • Add deterministic argument-contract tests that do not execute Test262.

No-workaround rules

  • No argument allowlist in a shell wrapper, delayed validation after corpus discovery, silent success, worker spawning for usage output, or weakening the intentional no-argument full run.

Acceptance

  • No args retains the exact full-corpus dispatch to runParent.
  • --help is side-effect-free and documents every mode.
  • Unknown, incomplete, malformed, and over-complete fixed-arity invocations fail before corpus work.
  • --diag optional filtering and --vm-witness variadic paths remain exact in the command contract.
  • Focused CLI tests, valid-path Test262 smoke, and dependency/build gates pass; 0 Test262 semantic flips.
8 days ago

Fail-closed runner dispatch pushed in 34f5c8c6 (fix(test262): reject invalid runner modes (#620)).

Every named mode now passes through one pure exact-arity parser before corpus discovery. No arguments still dispatch directly to the existing runParent; --help writes complete usage and returns; worker numeric operands are parsed strictly; fixed-arity modes reject trailing operands; --diag retains its optional filter and --vm-witness retains one-or-more paths. Both test262 and test262-bin depend on the four standard-library CLI contract tests.

Validation at the commit:

  • CLI contracts: 4/4 passed;
  • production test262-bin build passed;
  • --help: exit 0, Test262 process count 0 → 0;
  • unknown, missing --diag operand, and malformed worker index: each exit 1, process count stays 0;
  • dependency audit: 15 classified edges, 0 migration targets, 60 tools.

A valid --diag corpus smoke and the final 0-flip statement remain pending because user-owned Home unit jobs are active; #620 stays open until that uncontended check completes.

8 days ago

Completed by 34f5c8c6 (fix(test262): reject invalid runner modes (#620)).

Final uncontaminated valid-path evidence:

  • two process snapshots five seconds apart were clear before launch;
  • the production candidate runner completed --diag test/language/statements/while with 0 failures / 38 passes;
  • the immediate post-run snapshot was clear, with no surviving Test262 driver or worker.

Together with the 4/4 pure CLI contract tests, production test262-bin build, side-effect-free help, nonzero invalid/missing/malformed invocations, and the 15-edge / 0-migration / 60-tool dependency audit, every acceptance item is complete. This changes dispatch validation only and flips 0 Test262 semantic cases.

Sign in to comment on this issue.