ReviewOS

zig-utils/zig-js

Conformance smoke: return failure status when cases fail

#1068
Closed chris opened this 9 days ago ·
9 days ago

Related: #1067

A local before run of zig build conformance scored 0/33 cases after its environment-only assertion globals stopped resolving, yet exited 0. Its runner logs failed cases but returns successfully regardless of the result. This allows automation to accept an entirely failed local smoke.

Acceptance:

  • Return a nonzero process status whenever any smoke case fails.
  • Preserve the normal 33-case success result and exit 0.
  • Verify an intentionally failing snapshot exits nonzero and retains the failed-case report.

This changes only the local smoke runner process contract; no engine or corpus semantics change.

9 days ago

Landed in 1be5308. The local smoke now returns ConformanceFailed after printing its report when any case fails.

Validation: the normal 33/33 run exits 0. A source snapshot with exactly one deliberately wrong arithmetic assertion reports 32/33, retains the failed-case message, and exits 1. Formatting and diff checks pass. No engine semantics changed; flips 0 test262 cases.

Sign in to comment on this issue.