ReviewOS

stacks/drone

Execution HTTP: diagnose child exit 137 in Mac suites

#93
Open chris opened this 10 days ago ·
10 days ago

Parent: #68. Related diagnostic discipline: #60. This is a retained actual full-suite failure, not an established root cause or a passing rerun.

At exact source 7797296102472b668f68a63778591e9c9c87e8b2, the native Mac ./buddy test run reports five execution HTTP failures with nativeRoutes=true:

CaseReported durationChild result
limits39,325.81 ms137
preflight37,474.95 ms137
middleware35,774.36 ms137
chunk35,869.38 ms137
admission35,097.07 ms137

All five nativeRoutes=false counterparts passed in the same original run. The parent HTTP fixture has an unchanged 35,000 ms SIGKILL guard and a 40,000 ms outer test guard. It does not record the fired flag or settled signal, so exit 137 alone does not prove which actor killed the child or which operation was outstanding. Child outputs include migration records in some cases but no completed oracle; the exact unfinished phase is unknown. The original parent removes each temporary fixture in finally, so those original databases are unavailable.

The parent runner reports Bun 1.4.2; the HTTP child runtime was not independently identified. The full suite is still running and must not be described as a full pass or restarted to erase these failures. An immutable observed log prefix is retained privately in campaign 560: 136950 bytes, SHA-256 cbe1866da4eeafb1f2fcda2bb9b24df5a1f5a44b11e51575e5ba2ecb648efeff, alongside exact source pins. The prefix has no ENOSPC or SQLITE_FULL marker establishing a storage cause. Low disk headroom was observed separately and is not a causal verdict.

Acceptance:

  • Retain the original full terminal result separately from this observed prefix, all five failure labels and exact runtime/source evidence that is actually available.
  • Reproduce a failing native-route case at its real fixture/HTTP boundary after the current sole local product run terminates; distinguish initial startup, route setup, native request/stream work and cleanup with bounded private-data-free phase records.
  • Observe parent/child runtime identity, monotonic elapsed time, watchdog-fired state, settled signal and cleanup action independently. Retain useful failure evidence before temporary fixture cleanup.
  • Establish and fix the actual cause. Do not infer it from exit 137, a later green rerun, disk state or another fixture family alone.
  • Qualify both native-route settings and the complete actual suite with the original guards, authority, response-release and source-admission assertions. Do not increase a deadline merely to obtain green.

No physical vehicle acceptance is implied.

10 days ago

The original Mac full run was deliberately ended after the five real native-route failures, to isolate the failing boundary instead of spending the remaining run on a revision that could not qualify. This is incomplete full-suite qualification, not a completed five-failure total or a green rerun.

The original log and immutable failure prefix remain retained. The actual process exit is SIGTERM (-15); the wrapper exits 241. All three captured owned processes were stopped after PID/parent/birth verification, with no SIGKILL escalation. No workflow or test was restarted because an observation timed out.

A fresh single-case baseline is now running at unchanged product/source HEAD 7797296, with the original 35-second parent / 40-second outer guards. Parent Bun 1.4.2 and the executable/source hashes are captured before execution and will be checked again afterward. The original HTTP child identity was not independently observed; that missing evidence remains part of this issue.

10 days ago

The unchanged 19-file executed prefix from the original Mac run reproduces a child exit 137, this time for admission nativeRoutes=false at 35,112.49 ms. The run ends with 207 pass, 24 skip and 1 fail. All five nativeRoutes=true cases pass in this prefix. The earlier observation that only true-mode cases failed therefore does not establish a true-mode-specific cause.

Controls on the same pinned source and local Bun binary: the single true-mode limits case passes; all ten HTTP cases in a file-only run pass; loading the original prefix while executing only the true-mode limits case passes. These later passes do not resolve or explain the original failures. The executed prefix retains the native Buddy 30,000 ms global default, the existing explicit 40,000 ms outer HTTP bounds and 35,000 ms child guard. All source/runtime pins remain unchanged.

Original canceled full-run logs, all control logs, per-process resource samples and exact terminal receipts are retained separately. Parent exit 137 still does not identify the actual kill actor or unfinished fixture phase. No timeout or admission/freshness checks have been relaxed; this issue remains open.

10 days ago

Added bounded HTTP fixture observations in #95, stacked on #92. Parent records now distinguish a settled child result, watchdog firing, a requested watchdog SIGKILL and a cleanup kill. Child records independently identify the runtime and the last entered/settled phase. Failed runs retain private stdout, stderr and lifecycle files, including when the success sentinel is missing. Evidence-write failures preserve the actual child result.

Actual process controls distinguish an externally delivered SIGKILL from the existing 35-second watchdog firing. Missing-sentinel privacy was verified before and after the assertion fix with a real native fixture that exited 0: the parent still failed and the raw marker stayed in private capture after the fix.

The original 19-file executed prefix, with its original file order and 30-second global default, passed under pinned Bun 1.4.2: 208 passed, 24 skipped, 0 failed, 232 tests. All ten HTTP children independently reported Bun 1.4.2 and completed without watchdog or cleanup kills. A separate ten-case file run passed under observed parent and child Bun 1.4.3. These are separate observations, not a runtime-cause claim.

Original 35-second child SIGKILL and 40-second case deadlines are unchanged. The earlier exit-137 failures remain unexplained; this issue remains open. Exact-head full CI for #95 is running before main publication.

10 days ago

The observation changes in #95 are now published on main at 4b4a854, as a normal fast-forward after independently reviewed source and the exact-head complete workflow qualified.

Original workflow: all five jobs and required steps passed, including 1,883 passed / 41 skipped / 0 failed across 1,924 tests and 19 verified VTOL CLI reports. The original archive is retained; this is a distinct qualification run, not a replacement for the earlier failed or canceled evidence.

This does not explain the earlier Mac child exits of 137. The original deadlines remain unchanged. Phase, runtime identity, watchdog and private failure-capture observations are available for the next reproduction, and this issue remains open.

Sign in to comment on this issue.