ReviewOS
Reload

also looking at this

stacks/drone

test: retain owned browser startup process evidence

#100
Merged chris wants to merge codex/browser-startup-process-evidence into main
21 files +1516 -155

Review threads live on the whole diff, not on one commit, so none are shown here - a thread's line means something in the branch's final form, and painting it into an intermediate step would put it on code it is not about.

.github/workflows/ci.ymlmodified+16-0
@@ -30,6 +30,22 @@ jobs:
3030 - run: bun run parity:check
3131 - run: python3 -m py_compile resources/functions/drone/camera/imx500_worker.py
3232 - run: python3 -m unittest discover -s tests/hardware
33 - name: Qualify startup observations with actual exited and live native processes
34 env:
35 BROWSER_LAUNCH_EVIDENCE: ${{ runner.temp }}/browser-launch-exited
36 BROWSER_LAUNCH_LIVE_EVIDENCE: ${{ runner.temp }}/browser-launch-live
37 run: bun test tests/unit/public-safety-browser-launch.test.ts tests/unit/public-safety-browser-launch-evidence.test.ts tests/unit/public-safety-browser-startup-process.test.ts
38 - uses: actions/upload-artifact@v4
39 if: always()
40 with:
41 name: browser-launch-native-evidence
42 if-no-files-found: ignore
43 retention-days: 7
44 path: |
45 ${{ runner.temp }}/browser-launch-exited/browser-launch.json
46 ${{ runner.temp }}/browser-launch-exited/chrome-startup-stderr.log
47 ${{ runner.temp }}/browser-launch-live/browser-launch.json
48 ${{ runner.temp }}/browser-launch-live/chrome-startup-stderr.log
3349 - name: Qualify cold build setup across the former hook deadline
3450 run: bun scripts/qualify-runtime-cold-build.ts "$RUNNER_TEMP/runtime-cold-build"
3551 - uses: actions/upload-artifact@v4
config/docs.tsmodified+1-0
@@ -51,6 +51,7 @@ const config: BunPressOptions = {
5151 { text: 'Raspberry Pi setup', link: '/raspberry-pi' },
5252 { text: 'Library API', link: '/api' },
5353 { text: 'Validation evidence', link: '/validation' },
54 { text: 'Browser qualification evidence', link: '/browser-qualification' },
5455 { text: 'Flight-development gates', link: '/flight-development' },
5556 ],
5657 },
docs/browser-qualification.mdadded+80-0
@@ -0,0 +1,80 @@
1# Browser qualification observations
2
3The rendered workspace and execution checks launch a fresh native Chrome process in the
4existing fixture-owned process group. They retain the original launch arguments and the
5400-read, 50 ms DevToolsActivePort discovery guard. No retry or browser flag bypass is added.
6The full page/route/Worker scenarios must still pass. A successful native launcher fixture
7does not qualify those scenarios or explain an intermittent Chrome startup failure.
8
9## Startup evidence
10
11`browser-launch.json` retains its exit/port counters and adds a `startup` observation. The
12observer requests samples near launch, 1, 5, 10 and 15 seconds, then at discovery end. Samples
13run asynchronously during discovery, with at most one actual task in flight and a 500 ms
14cooperative cancellation budget. Cancellation stops further process/resource reads after
15outstanding native calls settle. It never releases ownership of an unfinished task. A timeout
16disables later Linux samples and hashing, closes captured descriptors after settlement and
17emits a bounded pending-task notice. Finalization waits for that task; it cannot allow a page
18visit behind it. An uninterruptible filesystem call can hold finalization until the existing
19outer fixture guard fails and terminates its group. This is not a hard OS syscall deadline.
20Actual start/completion epochs are retained; labels are requested milestones, not
21promises that an event occurred at that exact time. A pre-page cleanup sample follows the
22existing root termination/2 second wait. No process or resource sample is requested after a
23page visit. Post-page stderr remains private.
24
25Each sample includes a native root PID existence probe. Linux adds selected `/proc` state:
26parent/group/session, start/user/system ticks, thread count, RSS and kernel wait channel.
27Names, argv, environment, stacks and memory contents are not exported. Main-task child
28traversal and previously observed descendants are capped at 64 processes per sample; root
29thread wait details are capped at 32. Start ticks flag observed PID reuse. Fork/exit/reparenting,
30permissions and truncation can make this incomplete. It is not proof of whole-group cleanup.
31The outer fixture owner separately terminates its isolated group before disposable removal.
32
33Expected discovery-file metadata, observer cgroup-v2 counters and system CPU/memory/IO pressure
34help distinguish missing files, a live process and resource contention. Cgroup counters may
35cover other runner work; system pressure covers other host workloads. Neither proves Chrome
36causality. Unsupported platforms, missing files, permission errors and observation timeouts
37are explicit outcomes. Numeric kernel fields are selected and bounded; raw status, cgroup
38membership, thread names and arbitrary text are not retained.
39
40Startup samples pin at most four distinct actual Linux `/proc/PID/exe` file descriptors
41without reading executable bytes. Only after discovery ends does it hash them under one
422 second cooperative hash budget, with a 512 MiB per-executable limit. Device/inode identity uses exact
43integer strings. Byte count and file timestamps are checked before retaining a digest; all
44pinned descriptors are then closed, with close errors reported. This identifies an observed
45backing executable rather than assuming the configured launcher is Chrome. It neither warms
46the binary before the attempt nor authenticates the code or its entire loaded graph.
47
48The browser version is retained only from a successful CDP version response and a closed
49Chrome version format. If discovery fails, it stays null. Do not infer the failed binary's
50version from another run without matching actual identity evidence. Opening descriptors,
51reading process files and hashing are observation work and can affect scheduling; the
52observed times and unavailable/timeout outcomes must accompany comparisons.
53
54## Native qualification
55
56```sh
57bun test tests/unit/public-safety-browser-launch.test.ts tests/unit/public-safety-browser-launch-evidence.test.ts tests/unit/public-safety-browser-startup-process.test.ts
58```
59
60One actual native fixture exits 23 before discovery. Another owns a live child until the
61original cleanup request. The checks preserve all 400 failed probes, inspect actual PIDs,
62verify parent/child shutdown and ensure argv/environment sentinels do not enter the report.
63On Linux the live fixture also checks actual descendant/start/RSS data and the retained
64executable hash against the independently read runtime binary. Optional `BROWSER_LAUNCH_EVIDENCE`
65and `BROWSER_LAUNCH_LIVE_EVIDENCE` retain reports in new directories. CI publishes only the
66allowlisted launch JSON and startup stderr, not scripts, credentials or private fixture files.
67A held OS-adapter promise separately verifies that cancellation cannot overlap another sample,
68finalize early or begin hashing while an operation remains unfinished. It does not qualify
69Linux field acquisition; the actual native Linux fixture provides that evidence.
70A Linux cancellation case also opens real executable descriptors and checks that they are
71closed without a digest once the held native sample settles.
72
73Linux-only observations require Linux evidence; a macOS pass does not establish them.
74[Issue #72](https://github.com/stacksjs/drone/issues/72) retains the original failed attempts and
75passing controls. Its root cause and correction remain open until reproduced and verified.
76These observations change evidence, not aircraft control, physical outputs or flight readiness.
77
78Field semantics follow the kernel's [proc reference](https://docs.kernel.org/filesystems/proc.html),
79[cgroup-v2 reference](https://docs.kernel.org/admin-guide/cgroup-v2.html) and
80[pressure reference](https://docs.kernel.org/accounting/psi.html).
docs/flight-development.mdmodified+3-0
@@ -23,3 +23,6 @@ The owned runtime follows the dependency-linked [roadmap](roadmap/README.md). Th
2323support comparison and development while native board controls are implemented and validated.
2424
2525Until those milestones are complete, the code here should be used for simulation and bench development.
26
27Rendered application checks have a separate [browser qualification contract](browser-qualification.md).
28A browser-startup failure does not establish a vehicle, estimator or physical flight failure.
tests/helpers/browser-launch-evidence.tsmodified+2-0
@@ -2,12 +2,14 @@ import { Buffer } from 'node:buffer'
22import { writeSync } from 'node:fs'
33import { mkdir, open, writeFile } from 'node:fs/promises'
44import { join } from 'node:path'
5import type { BrowserStartupEvidence } from './browser-startup-process'
56
67export interface BrowserLaunchObservation {
78 phase: 'startup-unavailable' | 'discovery-failed' | 'closed'
89 elapsedMs: number
910 launch: { pid: number, exitObserved: boolean, settledExitCode: number | null, signalCode: string | number | null, exitObservedBeforeCleanup: boolean, cleanupRequested: boolean, exitElapsedMs: number | null }
1011 portProbe: { attempts: number, successfulReads: number, failedReads: number, lastReadError: string | null, discovered: boolean }
12 startup?: BrowserStartupEvidence
1113}
1214export function browserFileErrorCode(error: unknown): string {
1315 const code = (error as NodeJS.ErrnoException | null)?.code
tests/helpers/browser-startup-linux.tsadded+167-0
@@ -0,0 +1,167 @@
1import { createHash } from 'node:crypto'
2import { lstat, open, readFile, readlink, readdir } from 'node:fs/promises'
3import type { FileHandle } from 'node:fs/promises'
4import { join } from 'node:path'
5import { browserFileErrorCode } from './browser-launch-evidence'
6
7const processLimit = 64, threadLimit = 32, executableLimit = 4, executableByteLimit = 512 * 1024 * 1024
8async function text(path: string, signal: AbortSignal) {
9 if (signal.aborted) return { value: null, code: 'ABORT_ERR' }
10 try {
11 const value = await readFile(path, { encoding: 'utf8', signal })
12 return value.length <= 32768 ? { value, code: null } : { value: null, code: 'BYTE_LIMIT' }
13 }
14 catch (error) { return { value: null, code: browserFileErrorCode(error) } }
15}
16const decimal = (value: string | undefined) => value && /^\d{1,20}$/.test(value) ? value : null
17const integer = (value: string | undefined) => { const n = decimal(value); return n !== null && Number.isSafeInteger(Number(n)) ? Number(n) : null }
18async function processState(pid: number, signal: AbortSignal) {
19 const [raw, status, wait, children] = await Promise.all([text(`/proc/${pid}/stat`, signal), text(`/proc/${pid}/status`, signal), text(`/proc/${pid}/wchan`, signal), text(`/proc/${pid}/task/${pid}/children`, signal)])
20 signal.throwIfAborted()
21 const end = raw.value?.lastIndexOf(') ') ?? -1
22 // tcomm may contain spaces/parentheses and is deliberately discarded.
23 const fields = end >= 0 ? raw.value!.slice(end + 2).trim().split(/\s+/) : []
24 const valid = fields.length >= 22 && /^[RSDZTtXxKWPI]$/.test(fields[0]!) && decimal(fields[19]) !== null
25 const rss = status.value?.match(/^VmRSS:\s+(\d+)\s+kB$/m)?.[1]
26 const ids = children.value?.trim().split(/\s+/).map(integer).filter((n): n is number => n !== null && n > 0) ?? []
27 return {
28 record: { pid, code: valid ? null : raw.code ?? 'FORMAT', state: valid ? fields[0]! : null,
29 parentPid: valid ? integer(fields[1]) : null, group: valid ? integer(fields[2]) : null, session: valid ? integer(fields[3]) : null,
30 startTicks: valid ? decimal(fields[19]) : null, userTicks: valid ? decimal(fields[11]) : null, systemTicks: valid ? decimal(fields[12]) : null,
31 threads: valid ? integer(fields[17]) : null, rssKiB: integer(rss), statusCode: status.code,
32 wait: wait.value !== null && /^[A-Za-z0-9_.:]{1,80}$/.test(wait.value.trim()) ? wait.value.trim() : null, waitCode: wait.code, childrenCode: children.code },
33 children: ids.slice(0, processLimit), childrenCapped: ids.length > processLimit,
34 }
35}
36async function counters(path: string, allowed: string[], signal: AbortSignal) {
37 const result = await text(path, signal), values: Record<string, string> = {}
38 signal.throwIfAborted()
39 for (const line of result.value?.split('\n') ?? []) {
40 const [key, value] = line.trim().split(/\s+/)
41 if (key && allowed.includes(key) && decimal(value) !== null) values[key] = value!
42 }
43 return { values, code: result.code }
44}
45async function scalar(path: string, signal: AbortSignal) {
46 const result = await text(path, signal), value = result.value?.trim()
47 signal.throwIfAborted()
48 return { value: value === 'max' || decimal(value) !== null ? value! : null, code: result.code ?? (value === 'max' || decimal(value) !== null ? null : 'FORMAT') }
49}
50async function pressure(resource: string, signal: AbortSignal) {
51 const result = await text(`/proc/pressure/${resource}`, signal), values: Record<string, { avg10: number, avg60: number, avg300: number, totalMicroseconds: string }> = {}
52 for (const line of result.value?.split('\n') ?? []) {
53 const match = /^(some|full) avg10=(\d{1,3}(?:\.\d{1,6})?) avg60=(\d{1,3}(?:\.\d{1,6})?) avg300=(\d{1,3}(?:\.\d{1,6})?) total=(\d{1,20})$/.exec(line)
54 if (match) values[match[1]!] = { avg10: Number(match[2]), avg60: Number(match[3]), avg300: Number(match[4]), totalMicroseconds: match[5]! }
55 }
56 return { values, code: result.code }
57}
58async function resources(signal: AbortSignal) {
59 signal.throwIfAborted()
60 const membership = await text('/proc/self/cgroup', signal)
61 signal.throwIfAborted()
62 const path = membership.value?.split('\n').find(line => line.startsWith('0::'))?.slice(3)
63 const base = path?.startsWith('/') && !path.split('/').includes('..') ? join('/sys/fs/cgroup', path) : null
64 const [cpuPressure, memoryPressure, ioPressure] = await Promise.all(['cpu', 'memory', 'io'].map(name => pressure(name, signal)))
65 signal.throwIfAborted()
66 const cgroup = base ? {
67 memoryCurrent: await scalar(join(base, 'memory.current'), signal), memoryMax: await scalar(join(base, 'memory.max'), signal),
68 memoryEvents: await counters(join(base, 'memory.events'), ['low', 'high', 'max', 'oom', 'oom_kill', 'oom_group_kill', 'sock_throttled'], signal),
69 cpu: await counters(join(base, 'cpu.stat'), ['usage_usec', 'user_usec', 'system_usec', 'nr_periods', 'nr_throttled', 'throttled_usec', 'nr_bursts', 'burst_usec'], signal),
70 pidsCurrent: await scalar(join(base, 'pids.current'), signal), pidsMax: await scalar(join(base, 'pids.max'), signal),
71 } : null
72 return { scope: 'observer cgroup, potentially shared with other runner processes', pressureScope: 'system pressure, shared with other host workloads', membershipCode: membership.code ?? (base ? null : 'NOT_V2'), cgroup, cpuPressure, memoryPressure, ioPressure }
73}
74
75type Executable = { path: string, device: string, inode: string, byteLength: number, observedPids: number[], openedAtMs: number, sha256: string | null, hashedAfterDiscovery: boolean, code: string | null, closeCode: string | null }
76/** No argv, environment, comm, stack or memory dump is exported. */
77export function createLinuxStartupObservation(pid: number, profile: string, began: number) {
78 const tracked = new Map<number, string>(), pinned = new Map<string, { handle: FileHandle, mtimeNs: bigint, ctimeNs: bigint, evidence: Executable }>()
79 const identityErrors: Array<{ pid: number, code: string }> = []
80 let handlesSealed = false, executableLimitReached = false
81 async function executable(id: number, signal: AbortSignal) {
82 if (handlesSealed || signal.aborted) return
83 let handle: FileHandle | undefined
84 try {
85 handle = await open(`/proc/${id}/exe`, 'r')
86 if (handlesSealed || signal.aborted) return
87 const stat = await handle.stat({ bigint: true }), key = `${stat.dev}:${stat.ino}`
88 if (handlesSealed || signal.aborted) return
89 const prior = pinned.get(key)
90 if (prior) { if (!prior.evidence.observedPids.includes(id)) prior.evidence.observedPids.push(id); return }
91 if (pinned.size >= executableLimit) { executableLimitReached = true; return }
92 const path = await readlink(`/proc/self/fd/${handle.fd}`)
93 if (handlesSealed || signal.aborted) return
94 if (!stat.isFile() || stat.size <= 0n || stat.size > BigInt(executableByteLimit) || path.length > 4096) { if (identityErrors.length < 8) identityErrors.push({ pid: id, code: 'IDENTITY_LIMIT' }); return }
95 pinned.set(key, { handle, mtimeNs: stat.mtimeNs, ctimeNs: stat.ctimeNs, evidence: { path, device: String(stat.dev), inode: String(stat.ino), byteLength: Number(stat.size), observedPids: [id], openedAtMs: performance.now() - began, sha256: null, hashedAfterDiscovery: false, code: null, closeCode: null } })
96 handle = undefined
97 }
98 catch (error) { if (identityErrors.length < 8) identityErrors.push({ pid: id, code: browserFileErrorCode(error) }) }
99 finally {
100 if (handle) {
101 try { await handle.close() }
102 catch (closeError) { if (identityErrors.length < 8) identityErrors.push({ pid: id, code: browserFileErrorCode(closeError) }) }
103 }
104 }
105 }
106 return {
107 async snapshot(signal: AbortSignal) {
108 signal.throwIfAborted()
109 const root = await processState(pid, signal), descendants: Array<typeof root.record & { reused: boolean }> = []
110 const check = (id: number, start: string | null) => {
111 if (!start) return false
112 const prior = tracked.get(id)
113 if (prior && prior !== start) return false
114 if (tracked.size < processLimit) tracked.set(id, start)
115 return true
116 }
117 const rootMatches = check(pid, root.record.startTicks)
118 if (rootMatches) await executable(pid, signal)
119 const queue = [...new Set([...(rootMatches ? root.children : []), ...tracked.keys()].filter(id => id !== pid))]
120 const seen = new Set<number>([pid]); let processLimitReached = root.childrenCapped
121 while (queue.length && seen.size < processLimit && !signal.aborted) {
122 const id = queue.shift()!
123 if (seen.has(id)) continue
124 seen.add(id)
125 const state = await processState(id, signal), matches = check(id, state.record.startTicks)
126 descendants.push({ ...state.record, reused: state.record.startTicks !== null && !matches })
127 if (matches) { await executable(id, signal); queue.push(...state.children.filter(n => !seen.has(n))); processLimitReached ||= state.childrenCapped }
128 }
129 processLimitReached ||= queue.length > 0 || tracked.size === processLimit
130 signal.throwIfAborted()
131 let threadCode: string | null = null
132 const allThreads = await readdir(`/proc/${pid}/task`).catch(error => { threadCode = browserFileErrorCode(error); return [] })
133 signal.throwIfAborted()
134 const threads = await Promise.all(allThreads.filter(id => /^\d+$/.test(id)).slice(0, threadLimit).map(async id => {
135 const wait = await text(`/proc/${pid}/task/${id}/wchan`, signal)
136 return { tid: Number(id), wait: wait.value !== null && /^[A-Za-z0-9_.:]{1,80}$/.test(wait.value.trim()) ? wait.value.trim() : null, code: wait.code }
137 }))
138 signal.throwIfAborted()
139 const discoveryFile = await lstat(join(profile, 'DevToolsActivePort')).then(value => ({ present: true, regular: value.isFile(), bytes: value.size, code: null })).catch(error => ({ present: false, regular: null, bytes: null, code: browserFileErrorCode(error) }))
140 signal.throwIfAborted()
141 return { status: 'observed' as const, root: { ...root.record, reused: root.record.startTicks !== null && !rootMatches }, descendants, processLimitReached,
142 childScope: 'main-task children and previously observed descendants; incomplete during fork/exit/reparenting', threads, threadCode, threadDetailsCapped: allThreads.length > threadLimit, discoveryFile, resources: await resources(signal) }
143 },
144 async settleExecutables(shouldHash: boolean) {
145 handlesSealed = true
146 const abort = new AbortController(), timer = setTimeout(() => abort.abort(), 2000)
147 try {
148 for (const { handle, evidence, mtimeNs, ctimeNs } of pinned.values()) {
149 try {
150 if (!shouldHash) { evidence.code = 'OBSERVATION_CANCELED'; continue }
151 if (abort.signal.aborted) { evidence.code = 'HASH_BUDGET'; continue }
152 const hash = createHash('sha256'); let bytes = 0
153 for await (const chunk of handle.createReadStream({ autoClose: false, start: 0, end: evidence.byteLength - 1, signal: abort.signal })) { hash.update(chunk); bytes += chunk.length }
154 const after = await handle.stat({ bigint: true })
155 if (abort.signal.aborted) { evidence.code = 'HASH_BUDGET'; continue }
156 if (bytes !== evidence.byteLength || after.size !== BigInt(bytes) || after.mtimeNs !== mtimeNs || after.ctimeNs !== ctimeNs) { evidence.code = 'EXECUTABLE_CHANGED'; continue }
157 evidence.sha256 = hash.digest('hex'); evidence.hashedAfterDiscovery = true
158 }
159 catch (error) { evidence.code = browserFileErrorCode(error) }
160 finally { await handle.close().catch(error => { evidence.closeCode = browserFileErrorCode(error) }) }
161 }
162 }
163 finally { clearTimeout(timer) }
164 },
165 evidence() { return { executables: [...pinned.values()].map(value => structuredClone(value.evidence)), identityErrors: structuredClone(identityErrors), executableLimitReached } },
166 }
167}
tests/helpers/browser-startup-process.tsadded+61-0
@@ -0,0 +1,61 @@
1import { browserFileErrorCode } from './browser-launch-evidence'
2import type { createLinuxStartupObservation } from './browser-startup-linux'
3
4type NativeSnapshot = Awaited<ReturnType<ReturnType<typeof createLinuxStartupObservation>['snapshot']>>
5export interface StartupProcessAdapter {
6 snapshot: (signal: AbortSignal) => Promise<NativeSnapshot | { status: 'unavailable', code: string | null }>
7 settleExecutables: (hash: boolean) => Promise<void>
8 evidence: ReturnType<typeof createLinuxStartupObservation>['evidence']
9}
10
11type Probe = 'present' | 'missing' | 'denied' | 'unavailable'
12function rootProbe(pid: number): Probe {
13 try { process.kill(pid, 0); return 'present' }
14 catch (error) {
15 const code = browserFileErrorCode(error)
16 return code === 'ESRCH' ? 'missing' : code === 'EPERM' ? 'denied' : 'unavailable'
17 }
18}
19
20/** Startup-only facts. A PID probe does not establish complete process-group ownership. */
21export function createBrowserStartupObserver(pid: number, began: number, linux: StartupProcessAdapter | null) {
22 type Linux = NativeSnapshot | { status: 'unsupported' | 'timeout' | 'unavailable', code: string | null }
23 const snapshots: Array<{ label: string, elapsedMs: number, completedAtMs: number, root: { pid: number, probe: Probe }, linux: Linux }> = []
24 const milestones = [0, 1000, 5000, 10000, 15000]
25 let next = 0, sealed = false, inFlight = false, canceled = false, browserVersion: string | null = null
26 const sample = async (label: string) => {
27 const elapsedMs = performance.now() - began, root = { pid, probe: rootProbe(pid) }
28 const abort = new AbortController()
29 let timer: ReturnType<typeof setTimeout> | undefined, expired = false
30 try {
31 let observation: Linux = { status: 'unsupported', code: null }
32 if (linux && canceled) observation = { status: 'unavailable', code: 'OBSERVATION_CANCELED' }
33 else if (linux) {
34 timer = setTimeout(() => {
35 expired = true; canceled = true; abort.abort()
36 console.error(JSON.stringify({ oracle: 'browser-startup-observation-pending', pid, elapsedMs: performance.now() - began }))
37 }, 500)
38 // Cancellation requests are not settlement. Own this actual task until it returns.
39 observation = await linux.snapshot(abort.signal).catch(error => ({ status: 'unavailable' as const, code: browserFileErrorCode(error) }))
40 if (expired) observation = { status: 'timeout', code: 'OBSERVATION_BUDGET' }
41 }
42 snapshots.push({ label, elapsedMs, completedAtMs: performance.now() - began, root, linux: observation })
43 }
44 finally { if (timer) clearTimeout(timer) }
45 }
46 let pending: Promise<void> = Promise.resolve()
47 return {
48 tick() {
49 if (!sealed && !canceled && !inFlight && next < milestones.length && performance.now() - began >= milestones[next]!) {
50 const at = milestones[next++]!
51 inFlight = true
52 pending = sample(at === 0 ? 'launch' : `elapsed-${at}ms`).finally(() => { inFlight = false })
53 }
54 },
55 async finishDiscovery() { sealed = true; await pending; await sample('discovery-ended'); await linux?.settleExecutables(!canceled) },
56 async cleanupEnded() { await sample('cleanup-ended') },
57 recordVersion(value: unknown) { if (typeof value === 'string' && /^(?:HeadlessChrome|Chrome)\/\d{1,6}(?:\.\d{1,8}){3}$/.test(value)) browserVersion = value },
58 evidence() { return { method: 'startup-only-process-observation-v1' as const, platform: process.platform, browserVersion, snapshots: structuredClone(snapshots), ...linux?.evidence() ?? { executables: [], identityErrors: [], executableLimitReached: false } } },
59 }
60}
61export type BrowserStartupEvidence = ReturnType<ReturnType<typeof createBrowserStartupObserver>['evidence']>
tests/helpers/workspace-browser.tsmodified+17-5
@@ -4,6 +4,8 @@ import { join } from 'node:path'
44import { observeBrowserOperation } from './browser-operation'
55import { browserFileErrorCode, retainBrowserLaunchEvidence } from './browser-launch-evidence'
66import type { BrowserLaunchObservation } from './browser-launch-evidence'
7import { createBrowserStartupObserver } from './browser-startup-process'
8import { createLinuxStartupObservation } from './browser-startup-linux'
79
810/** Dependency-free CDP, with a fresh profile. Credentials never enter command arguments. */
911async function openSocket(url: string): Promise<WebSocket> {
@@ -22,31 +24,41 @@ export async function startWorkspaceBrowser(root: string) {
2224 const child = Bun.spawn([process.env.BROWSE_BROWSER ?? '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', '--headless=new', '--disable-gpu', '--disable-background-timer-throttling', '--disable-renderer-backgrounding', '--disable-backgrounding-occluded-windows', '--disable-features=CalculateNativeWinOcclusion', '--disable-dev-shm-usage', '--disable-extensions', '--disable-component-update', '--no-first-run', '--no-default-browser-check', '--disable-background-networking', '--remote-debugging-port=0', `--user-data-dir=${profile}`, 'about:blank'], { stdout: 'ignore', stderr: Bun.file(join(root, 'chrome.log')) })
2325 const started = Date.now()
2426 const began = performance.now()
27 const startup = createBrowserStartupObserver(child.pid, began, process.platform === 'linux' ? createLinuxStartupObservation(child.pid, profile, began) : null)
2528 let closing = false
2629 const launch: BrowserLaunchObservation['launch'] = { pid: child.pid, exitObserved: false, settledExitCode: null, signalCode: null, exitObservedBeforeCleanup: false, cleanupRequested: false, exitElapsedMs: null }
2730 const portProbe: BrowserLaunchObservation['portProbe'] = { attempts: 0, successfulReads: 0, failedReads: 0, lastReadError: null, discovered: false }
2831 let pageVisited = false
29 const retain = (phase: BrowserLaunchObservation['phase']) => retainBrowserLaunchEvidence(root, { phase, elapsedMs: performance.now() - began, launch: { ...launch }, portProbe: { ...portProbe } }, { startupStderr: !pageVisited, exportDirectory: process.env.WORKSPACE_BROWSER_EVIDENCE ?? process.env.EXECUTION_BROWSER_EVIDENCE })
32 const retain = (phase: BrowserLaunchObservation['phase']) => retainBrowserLaunchEvidence(root, { phase, elapsedMs: performance.now() - began, launch: { ...launch }, portProbe: { ...portProbe }, startup: startup.evidence() }, { startupStderr: !pageVisited, exportDirectory: process.env.WORKSPACE_BROWSER_EVIDENCE ?? process.env.EXECUTION_BROWSER_EVIDENCE })
33 const cleanup = async () => {
34 closing = true; launch.cleanupRequested = true
35 if (child.exitCode === null) child.kill()
36 await Promise.race([child.exited, Bun.sleep(2000)])
37 if (!pageVisited) await startup.cleanupEnded()
38 }
3039 void child.exited.then(code => {
3140 launch.exitObserved = true; launch.settledExitCode = code; launch.signalCode = child.signalCode; launch.exitObservedBeforeCleanup = !closing; launch.exitElapsedMs = performance.now() - began
3241 if (!closing) console.log(`QA isolated Chrome exited unexpectedly: ${code}, elapsedMs:${Date.now() - started}`)
3342 })
3443 let port = ''
3544 for (let attempt = 0; attempt < 400; attempt++) {
45 startup.tick()
3646 portProbe.attempts++
3747 port = await readFile(join(profile, 'DevToolsActivePort'), 'utf8').then(value => { portProbe.successfulReads++; portProbe.lastReadError = null; return value.split('\n')[0]! }).catch(error => { portProbe.failedReads++; portProbe.lastReadError = browserFileErrorCode(error); return '' })
3848 if (port) { portProbe.discovered = true; break }
3949 await Bun.sleep(50)
4050 }
41 if (!port) { closing = true; launch.cleanupRequested = true; if (child.exitCode === null) child.kill(); await Promise.race([child.exited, Bun.sleep(2000)]); await retain('startup-unavailable'); throw new Error('Fresh browser must start') }
51 await startup.finishDiscovery()
52 if (!port) { await cleanup(); await retain('startup-unavailable'); throw new Error('Fresh browser must start') }
4253 // Keep a browser-level session, as well as page sessions, for the owned lifecycle.
4354 let browserSocket: WebSocket
4455 try {
4556 const response = await observeBrowserOperation('version-fetch', () => fetch(`http://127.0.0.1:${port}/json/version`, { signal: AbortSignal.timeout(5000) }))
46 const version = await observeBrowserOperation('version-json', () => response.json()) as { webSocketDebuggerUrl: string }
57 const version = await observeBrowserOperation('version-json', () => response.json()) as { webSocketDebuggerUrl: string, Browser?: unknown }
58 startup.recordVersion(version.Browser)
4759 browserSocket = await observeBrowserOperation('browser-socket', () => openSocket(version.webSocketDebuggerUrl))
4860 }
49 catch (error) { closing = true; launch.cleanupRequested = true; if (child.exitCode === null) child.kill(); await Promise.race([child.exited, Bun.sleep(2000)]); await retain('discovery-failed'); throw error }
61 catch (error) { await cleanup(); await retain('discovery-failed'); throw error }
5062 let browserSequence = 0
5163 browserSocket.onmessage = () => {}
5264 const keepAlive = setInterval(() => { if (browserSocket.readyState === WebSocket.OPEN) browserSocket.send(JSON.stringify({ id: ++browserSequence, method: 'Browser.getVersion' })) }, 5000)
@@ -96,6 +108,6 @@ export async function startWorkspaceBrowser(root: string) {
96108 await page.wait(javascript ? "document.querySelector('#incident-key') !== null && document.querySelector('[data-workspace-status]')?.textContent.trim() === 'disconnected'" : "document.querySelector('#incident-key') !== null", 'Compiled workspace must initialize')
97109 return page
98110 },
99 async close() { console.log(`QA browser.close requested, elapsedMs:${Date.now() - started}`); closing = true; launch.cleanupRequested = true; clearInterval(keepAlive); browserSocket.close(); for (const page of pages) page.close(); if (child.exitCode === null) child.kill(); await Promise.race([child.exited, Bun.sleep(2000)]); await retain('closed') },
111 async close() { console.log(`QA browser.close requested, elapsedMs:${Date.now() - started}`); clearInterval(keepAlive); browserSocket.close(); for (const page of pages) page.close(); await cleanup(); await retain('closed') },
100112 }
101113}
tests/unit/public-safety-browser-launch.test.tsmodified+60-1
@@ -1,5 +1,5 @@
11import { expect, test } from 'bun:test'
2import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
2import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
33import { tmpdir } from 'node:os'
44import { join } from 'node:path'
55import process from 'node:process'
@@ -25,6 +25,11 @@ test.skipIf(process.platform === 'win32')('failed native launch retains startup
2525 expect(report.launch.cleanupRequested).toBe(true)
2626 expect(report.portProbe.attempts).toBe(400)
2727 expect(report.portProbe.lastReadError).toBe('ENOENT')
28 expect(report.startup.platform).toBe(process.platform)
29 expect(report.startup.method).toBe('startup-only-process-observation-v1')
30 expect(report.startup.snapshots.at(-1).label).toBe('cleanup-ended')
31 expect(report.startup.snapshots.at(-1).root.pid).toBe(actualPid)
32 expect(report.startup.snapshots.at(-1).root.probe).toBe('missing')
2833 expect(report.stderr.status).toBe('retained')
2934 expect(report.stderr.retainedBytes).toBeLessThanOrEqual(65536)
3035 expect(await readFile(join(root, 'chrome-startup-stderr.log'), 'utf8')).toContain('controlled native browser startup refusal')
@@ -46,3 +51,57 @@ test.skipIf(process.platform === 'win32')('failed native launch retains startup
4651 else console.log(JSON.stringify({ evidence: 'controlled-native-launch', retainedRoot: root, passed }))
4752 }
4853}, 30000)
54
55// The existing native launch seam checks observed processes, not the cause of a Chrome failure.
56test.skipIf(process.platform === 'win32')('a live native launcher and child remain observable until cleanup without exporting argv or environment', async () => {
57 const supplied = process.env.BROWSER_LAUNCH_LIVE_EVIDENCE
58 const root = supplied ? (await mkdir(supplied), supplied) : await mkdtemp(join(tmpdir(), 'drone-browser-live-'))
59 const executable = join(root, 'launch.ts'), pidPath = join(root, 'pids.json')
60 const original = process.env.BROWSE_BROWSER
61 let passed = false
62 try {
63 await writeFile(executable, `#!${process.execPath}\nimport { writeFileSync } from 'node:fs'\nconst child=Bun.spawn([process.execPath,'-e','setInterval(()=>{},1000)','PRIVATE_CHILD_ARG_SENTINEL'],{stdout:'ignore',stderr:'ignore',env:{...process.env,PRIVATE_FIXTURE_TOKEN:'PRIVATE_CHILD_ENV_SENTINEL'}})\nwriteFileSync(${JSON.stringify(pidPath)},JSON.stringify([process.pid,child.pid]))\nprocess.on('SIGTERM',async()=>{child.kill();await child.exited;process.exit(0)})\nconsole.error('controlled live browser startup fixture')\nsetInterval(()=>{},1000)\n`, { mode: 0o700 })
64 process.env.BROWSE_BROWSER = executable
65 await expect(startWorkspaceBrowser(root)).rejects.toThrow('Fresh browser must start')
66 const [pid, childPid] = JSON.parse(await readFile(pidPath, 'utf8')) as [number, number]
67 const text = await readFile(join(root, 'browser-launch.json'), 'utf8'), report = JSON.parse(text)
68 expect(text).not.toContain('PRIVATE_CHILD_ARG_SENTINEL')
69 expect(text).not.toContain('PRIVATE_CHILD_ENV_SENTINEL')
70 expect(report.portProbe.attempts).toBe(400)
71 expect(report.launch.exitObservedBeforeCleanup).toBe(false)
72 expect(report.launch.exitObserved).toBe(true)
73 const before = report.startup.snapshots.find((s: any) => s.label === 'discovery-ended')
74 expect(before.root.pid).toBe(pid)
75 expect(before.root.probe).toBe('present')
76 expect(report.startup.snapshots.at(-1).root.probe).toBe('missing')
77 expect(report.startup.executables).toBeArray()
78 if (process.platform === 'linux') {
79 expect(before.linux.status).toBe('observed')
80 expect(before.linux.root.startTicks).toMatch(/^\d+$/)
81 expect(before.linux.root.rssKiB).toBeGreaterThan(0)
82 expect(before.linux.descendants.some((p: any) => p.pid === childPid && p.parentPid === pid)).toBe(true)
83 const identity = report.startup.executables.find((e: any) => e.observedPids.includes(pid) && e.sha256 !== null)
84 expect(identity).toBeDefined()
85 expect(identity.byteLength).toBe((await stat(process.execPath)).size)
86 expect(identity.sha256).toBe(new Bun.CryptoHasher('sha256').update(await Bun.file(process.execPath).arrayBuffer()).digest('hex'))
87 expect(identity.hashedAfterDiscovery).toBe(true)
88 }
89 else expect(before.linux.status).toBe('unsupported')
90 expect(() => process.kill(pid, 0)).toThrow()
91 expect(() => process.kill(childPid, 0)).toThrow()
92 passed = true
93 }
94 finally {
95 if (original === undefined) delete process.env.BROWSE_BROWSER
96 else process.env.BROWSE_BROWSER = original
97 const pids: number[] = JSON.parse(await readFile(pidPath, 'utf8').catch(() => '[]'))
98 for (const pid of pids) {
99 let gone = false
100 try { process.kill(pid, 0) }
101 catch (error) { gone = (error as NodeJS.ErrnoException).code === 'ESRCH' }
102 expect(gone).toBe(true)
103 }
104 if (passed && !supplied) await rm(root, { recursive: true, force: true })
105 else console.log(JSON.stringify({ evidence: 'controlled-live-native-launch', retainedRoot: root, passed }))
106 }
107}, 35000)
tests/unit/public-safety-browser-startup-process.test.tsadded+80-0
@@ -0,0 +1,80 @@
1import { expect, test } from 'bun:test'
2import { mkdtemp, readlink, readdir, rm } from 'node:fs/promises'
3import { tmpdir } from 'node:os'
4import { join } from 'node:path'
5import { createBrowserStartupObserver } from '../helpers/browser-startup-process'
6import type { StartupProcessAdapter } from '../helpers/browser-startup-process'
7import { createLinuxStartupObservation } from '../helpers/browser-startup-linux'
8
9// OS adapter seam: a held syscall promise checks ownership/cancellation, not Linux field acquisition.
10test('an expired observation remains owned through actual settlement and prevents later provider work', async () => {
11 let release!: () => void
12 const held = new Promise<void>(resolve => { release = resolve })
13 let calls = 0, active = 0, maximumActive = 0, finalized = false
14 const settlements: boolean[] = []
15 const adapter: StartupProcessAdapter = {
16 async snapshot() {
17 calls++; active++; maximumActive = Math.max(maximumActive, active)
18 try { if (calls === 1) await held; return { status: 'unavailable', code: 'CONTROLLED_OS' } }
19 finally { active-- }
20 },
21 async settleExecutables(hash) { expect(active).toBe(0); settlements.push(hash) },
22 evidence: () => ({ executables: [], identityErrors: [], executableLimitReached: false }),
23 }
24 const observer = createBrowserStartupObserver(process.pid, performance.now() - 1001, adapter)
25 let finishing: Promise<void> | undefined
26 try {
27 observer.tick()
28 await Bun.sleep(600)
29 observer.tick()
30 expect(calls).toBe(1)
31 finishing = observer.finishDiscovery().then(() => { finalized = true })
32 await Bun.sleep(10)
33 expect(finalized).toBe(false)
34 expect(settlements).toEqual([])
35 release(); await finishing
36 expect(maximumActive).toBe(1)
37 expect(calls).toBe(1)
38 expect(settlements).toEqual([false])
39 const final = observer.evidence()
40 observer.tick(); await Bun.sleep(10)
41 expect(observer.evidence()).toEqual(final)
42 expect(final.snapshots[0]!.linux.status).toBe('timeout')
43 expect(final.snapshots[0]!.completedAtMs).toBeGreaterThan(final.snapshots[0]!.elapsedMs + 500)
44 }
45 finally { release(); await finishing; await Bun.sleep(0) }
46})
47
48test.skipIf(process.platform !== 'linux')('cancellation closes actual pinned executable descriptors without hashing after the held native sample settles', async () => {
49 const root = await mkdtemp(join(tmpdir(), 'drone-observer-descriptors-'))
50 const executable = await readlink('/proc/self/exe')
51 const descriptors = async () => {
52 const ids = await readdir('/proc/self/fd')
53 const targets = await Promise.all(ids.map(id => readlink(`/proc/self/fd/${id}`).catch(() => null)))
54 return targets.filter(path => path === executable).length
55 }
56 const before = await descriptors(), began = performance.now()
57 const native = createLinuxStartupObservation(process.pid, root, began)
58 let release!: () => void, ready!: () => void
59 const held = new Promise<void>(resolve => { release = resolve }), captured = new Promise<void>(resolve => { ready = resolve })
60 const observer = createBrowserStartupObserver(process.pid, began, {
61 async snapshot(signal) {
62 const result = await native.snapshot(signal).catch(() => ({ status: 'unavailable' as const, code: 'NATIVE_UNAVAILABLE' }))
63 ready(); await held; return result
64 },
65 settleExecutables: hash => native.settleExecutables(hash), evidence: native.evidence,
66 })
67 let finishing: Promise<void> | undefined
68 try {
69 observer.tick(); await captured
70 expect(await descriptors()).toBeGreaterThan(before)
71 await Bun.sleep(600)
72 finishing = observer.finishDiscovery()
73 release(); await finishing
74 const report = observer.evidence()
75 expect(report.executables.length).toBeGreaterThan(0)
76 expect(report.executables.every(e => e.sha256 === null && e.code === 'OBSERVATION_CANCELED' && e.closeCode === null)).toBe(true)
77 expect(await descriptors()).toBe(before)
78 }
79 finally { release(); await (finishing ?? observer.finishDiscovery()); await rm(root, { recursive: true, force: true }) }
80})