also looking at this
feat: bind native runtime host boots
#50
22 files
+882
-81
Review threads live on the whole diff, not on one commit, so none are shown here - a thread's line means something in the branch's final form, and painting it into an intermediate step would put it on code it is not about.
| @@ -47,7 +47,8 @@ and board-clock echo limit constrain diagnostic leases rather than comparing unr | ||
| 47 | 47 | timestamp, lease duration and all diagnostic/status-only fields must be zero. |
| 48 | 48 | 2. The board immediately clears any existing diagnostic lease and binds the host token. Status |
| 49 | 49 | echoes it alongside the random per-boot token, exact version and capability flags. Boot tokens |
| 50 | are generated by the Pico SDK random API; the host test HAL uses a documented fixed test token. | |
| 50 | are generated by the Pico SDK random API on the board. The host test HAL uses a fresh | |
| 51 | supervisor-owned launch nonce, checked on every host status response. | |
| 51 | 52 | 3. The Pi verifies the echoed token, nonzero boot token, known capabilities, valid sample, health, |
| 52 | 53 | state-size bound and monotonic status sequence. Invalid status loses the session. |
| 53 | 54 | 4. Lease/stop commands must match both tokens and have a strictly larger command sequence. |
| @@ -77,11 +78,20 @@ capability link, not a `FlightAdapter`, and cannot cause non-neutral output. | ||
| 77 | 78 | |
| 78 | 79 | ## Host HAL test channel |
| 79 | 80 | |
| 80 | `drone_runtime_host --stdio` accepts only `tick <uint64 microseconds>` and `frame <128 lowercase hex | |
| 81 | Start the host with `drone_runtime_host --stdio --boot-id <16 lowercase hex characters>`. | |
| 82 | The supervisor allocates a fresh nonzero 64-bit cryptographic random nonce for each launch; | |
| 83 | missing, zero, malformed or additional arguments fail before processing input. This startup | |
| 84 | argument binds the host process to its supervisor and does not attest its executable or hardware. | |
| 85 | Older supervisors and host binaries must be rebuilt together. The 64-byte wire format stays v1. | |
| 86 | ||
| 87 | The host accepts only `tick <uint64 microseconds>` and `frame <128 lowercase hex | |
| 81 | 88 | characters>`, each newline-terminated and bounded by a 160-byte input buffer. Each returns one |
| 82 | 89 | 128-character status-frame hex line. `tick` advances virtual time; it does not sleep in real time. |
| 83 | 90 | This channel is not a firmware transport or public network service. The Stacks harness serializes |
| 84 | requests, bounds response lines, times out unresponsive processes and closes them after each run. | |
| 91 | requests, bounds response lines and times out unresponsive processes. Every returned status must | |
| 92 | match the supervisor's launch nonce. Invalid responses kill the process and permanently fail | |
| 93 | that transport. Returned status frames are immutable. Closing waits for actual process exit and | |
| 94 | returns its exit code and signal; repeated closes retain the same result. | |
| 85 | 95 | |
| 86 | 96 | The optional [DS sensor stream](sensors.md) shares UART TX as complete 128-byte frames. |
| 87 | 97 | DR remains 64 bytes. Awaited DR replies take priority when the current frame finishes; partial |
| @@ -330,6 +330,11 @@ and known-good slot roles, identity-bound runtime health, current-policy selecti | ||
| 330 | 330 | and rollback, authenticated key maintenance and the full fault campaign remain |
| 331 | 331 | required by #42. |
| 332 | 332 | |
| 333 | The [host bench](runtime.md) now binds each process to a fresh supervisor-owned boot nonce | |
| 334 | and requires a clean exit. This prerequisite provides no installed image identity, bounded | |
| 335 | health transcript or known-good confirmation. Slot health must still bind the verified image, | |
| 336 | owned boot/session challenge and actual runtime evidence before changing known-good state. | |
| 337 | ||
| 333 | 338 | ## Repository and signed image contract |
| 334 | 339 | |
| 335 | 340 | Use a standard TUF repository with `metadata/` and `targets/` directories, |
| @@ -41,6 +41,12 @@ boundaries, clock rollback, identity/version rejection, reset state and ten thou | ||
| 41 | 41 | The same fixed-buffer UART state machine used by Pico is also exercised through a host I/O harness |
| 42 | 42 | for fragmentation, resynchronization, corruption, RX/TX budgets and pending transmit buffers. |
| 43 | 43 | |
| 44 | Each host launch receives a fresh supervisor-owned boot nonce. The report includes its decimal | |
| 45 | `bootId`, and the harness checks that identity on every immutable response. The bench reports | |
| 46 | success only after its actual process exits cleanly. This is host session separation, without | |
| 47 | image attestation or update health confirmation. It does not measure board boot entropy or | |
| 48 | physical timing. See the [host startup contract](runtime-protocol.md#host-hal-test-channel). | |
| 49 | ||
| 44 | 50 | Build a standalone CLI for the current platform: |
| 45 | 51 | |
| 46 | 52 | ```sh |
| @@ -10,8 +10,16 @@ static int nibble(char c) { | ||
| 10 | 10 | return -1; |
| 11 | 11 | } |
| 12 | 12 | int main(int argc, char** argv) { |
| 13 | if (argc != 2 || std::strcmp(argv[1], "--stdio") != 0) return 2; | |
| 14 | drone::Runtime runtime(0x1122334455667788ull); | |
| 13 | if (argc != 4 || std::strcmp(argv[1], "--stdio") != 0 | |
| 14 | || std::strcmp(argv[2], "--boot-id") != 0 || std::strlen(argv[3]) != 16) return 2; | |
| 15 | uint64_t boot = 0; | |
| 16 | for (std::size_t i = 0; i < 16; ++i) { | |
| 17 | const int value = nibble(argv[3][i]); | |
| 18 | if (value < 0) return 2; | |
| 19 | boot = (boot << 4) | uint64_t(value); | |
| 20 | } | |
| 21 | if (boot == 0) return 2; | |
| 22 | drone::Runtime runtime(boot); | |
| 15 | 23 | uint64_t now = 0; |
| 16 | 24 | char line[160]; |
| 17 | 25 | while (std::fgets(line, sizeof(line), stdin)) { |
| @@ -6,7 +6,7 @@ import { decodeRuntimeFrame, encodeRuntimeFrame, runtimeCrc32, runtimeFlags } fr | ||
| 6 | 6 | |
| 7 | 7 | export const runtimeBenchFaults = ['none', 'deadline', 'stale', 'expiry', 'replay', 'version'] as const |
| 8 | 8 | export type RuntimeBenchFault = typeof runtimeBenchFaults[number] |
| 9 | export async function runRuntimeBench(options: { project?: string, ticks?: number, fault?: RuntimeBenchFault, onBuild?: (artifact: string) => Promise<void>, onRequest?: (line: string, atMs: number) => void | Promise<void>, onStatus?: (status: RuntimeFrame, atMs: number) => void | Promise<void> } = {}) { | |
| 9 | export async function runRuntimeBench(options: { project?: string, ticks?: number, fault?: RuntimeBenchFault, onBuild?: (artifact: string) => Promise<void>, onRequest?: (line: string, atMs: number) => void | Promise<void>, onStatus?: (status: Readonly<RuntimeFrame>, atMs: number) => void | Promise<void> } = {}) { | |
| 10 | 10 | const ticks = options.ticks ?? 1000 |
| 11 | 11 | const fault = options.fault ?? 'none' |
| 12 | 12 | if (!Number.isSafeInteger(ticks) || ticks < 1 || ticks > 100000 || !runtimeBenchFaults.includes(fault)) throw new RangeError('Use 1..100000 ticks and a supported bench fault') |
| @@ -16,6 +16,7 @@ export async function runRuntimeBench(options: { project?: string, ticks?: numbe | ||
| 16 | 16 | const session = createRuntimeSession() |
| 17 | 17 | const started = performance.now() |
| 18 | 18 | let time = 1000 |
| 19 | let operationFailed = false, operationError: unknown | |
| 19 | 20 | async function exchange(line: string) { |
| 20 | 21 | await options.onRequest?.(line, time / 1000) |
| 21 | 22 | const status = await harness.exchange(line) |
| @@ -62,11 +63,21 @@ export async function runRuntimeBench(options: { project?: string, ticks?: numbe | ||
| 62 | 63 | || (fault === 'deadline' && !(status.flags & runtimeFlags.deadlineFault))) throw new Error('Native bench failure did not return the required neutral-only, unleased state') |
| 63 | 64 | return { |
| 64 | 65 | evidence: 'host-harness', boardCandidate: 'pico2', protocolVersion: 1, fault, |
| 66 | bootId: String(harness.bootId), | |
| 65 | 67 | ticks: status.ticks, virtualElapsedUs: time, hostElapsedMs: Math.round((performance.now() - started) * 1000) / 1000, |
| 66 | 68 | stateBytes: status.stateBytes, maxExecutionUs: status.maxExecutionUs, deadlineMisses: status.deadlineMisses, |
| 67 | 69 | neutralOnly: true, leaseActive: false, physicalFlightEnabled: false, |
| 68 | 70 | note: 'Host virtual-clock evidence. GPIO timing, stack high-water and hardware watchdog require a Pico 2 bench measurement.', |
| 69 | 71 | } |
| 70 | 72 | } |
| 71 | finally { await harness.close() } | |
| 73 | catch (error) { operationFailed = true; operationError = error; throw error } | |
| 74 | finally { | |
| 75 | let exit | |
| 76 | try { exit = await harness.close() } | |
| 77 | catch (error) { | |
| 78 | if (operationFailed) throw new AggregateError([operationError, error], 'Native bench operation and process cleanup failed') | |
| 79 | throw error | |
| 80 | } | |
| 81 | if (!operationFailed && (exit.exitCode !== 0 || exit.signal !== null)) throw new Error('Native bench process did not exit cleanly') | |
| 82 | } | |
| 72 | 83 | } |
| @@ -1,14 +1,25 @@ | ||
| 1 | 1 | import { decodeRuntimeFrame } from './protocol' |
| 2 | import { randomBytes } from 'node:crypto' | |
| 3 | ||
| 4 | function freshBootId() { | |
| 5 | let value = 0n | |
| 6 | while (value === 0n) value = randomBytes(8).readBigUInt64BE() | |
| 7 | return value | |
| 8 | } | |
| 9 | export interface RuntimeHarnessExit { exitCode: number, signal: string | number | null } | |
| 2 | 10 | |
| 3 | 11 | /** ASCII stepping exists only in the host HAL. Board UART carries raw 64-byte frames. */ |
| 4 | 12 | export function openRuntimeHarness(artifact: string) { |
| 5 | const child = Bun.spawn([artifact, '--stdio'], { stdin: 'pipe', stdout: 'pipe', stderr: 'inherit' }) | |
| 13 | const bootId = freshBootId() | |
| 14 | const child = Bun.spawn([artifact, '--stdio', '--boot-id', bootId.toString(16).padStart(16, '0')], { stdin: 'pipe', stdout: 'pipe', stderr: 'inherit' }) | |
| 6 | 15 | const reader = child.stdout.getReader() |
| 7 | 16 | let pending = '' |
| 8 | 17 | let active = false |
| 9 | 18 | let closed = false |
| 19 | let failed = false | |
| 20 | let closing: Promise<Readonly<RuntimeHarnessExit>> | undefined | |
| 10 | 21 | async function exchange(line: string) { |
| 11 | if (closed || active || line.length > 150 || line.includes('\n')) throw new Error('Invalid or concurrent harness request') | |
| 22 | if (closed || failed || active || line.length > 150 || line.includes('\n')) throw new Error('Invalid, failed or concurrent harness request') | |
| 12 | 23 | active = true |
| 13 | 24 | const timeout = setTimeout(() => child.kill('SIGKILL'), 2000) |
| 14 | 25 | try { |
| @@ -24,19 +35,37 @@ export function openRuntimeHarness(artifact: string) { | ||
| 24 | 35 | const hex = pending.slice(0, end) |
| 25 | 36 | pending = pending.slice(end + 1) |
| 26 | 37 | if (!/^[a-f\d]{128}$/.test(hex)) throw new Error('Invalid native harness output') |
| 27 | return decodeRuntimeFrame(Uint8Array.fromHex(hex)) | |
| 38 | const frame = decodeRuntimeFrame(Uint8Array.fromHex(hex)) | |
| 39 | if (frame.kind !== 'status' || frame.bootId !== bootId) throw new Error('Native harness boot identity rejected') | |
| 40 | return Object.freeze(frame) | |
| 28 | 41 | } |
| 42 | catch (error) { failed = true; child.kill('SIGKILL'); throw error } | |
| 29 | 43 | finally { active = false; clearTimeout(timeout) } |
| 30 | 44 | } |
| 31 | 45 | return { |
| 46 | get bootId() { return bootId }, | |
| 32 | 47 | exchange, |
| 33 | async close() { | |
| 34 | if (closed) return | |
| 48 | async close(): Promise<Readonly<RuntimeHarnessExit>> { | |
| 49 | if (closing) return closing | |
| 35 | 50 | closed = true |
| 36 | child.stdin.end() | |
| 37 | const timeout = setTimeout(() => child.kill('SIGKILL'), 1000) | |
| 38 | try { await child.exited } | |
| 39 | finally { clearTimeout(timeout); await reader.cancel().catch(() => {}); reader.releaseLock() } | |
| 51 | if (active) child.kill('SIGKILL') | |
| 52 | closing = (async () => { | |
| 53 | let inputFailed = false, inputError: unknown | |
| 54 | try { child.stdin.end() } | |
| 55 | catch (error) { inputFailed = true; inputError = error; child.kill('SIGKILL') } | |
| 56 | const timeout = setTimeout(() => child.kill('SIGKILL'), 1000) | |
| 57 | try { | |
| 58 | const exitCode = await child.exited | |
| 59 | if (inputFailed) throw inputError | |
| 60 | return Object.freeze({ exitCode, signal: child.signalCode ?? null }) | |
| 61 | } | |
| 62 | finally { | |
| 63 | clearTimeout(timeout) | |
| 64 | try { await reader.cancel() } | |
| 65 | finally { reader.releaseLock() } | |
| 66 | } | |
| 67 | })() | |
| 68 | return closing | |
| 40 | 69 | }, |
| 41 | 70 | } |
| 42 | 71 | } |
| @@ -1,4 +1,7 @@ | ||
| 1 | 1 | import { beforeAll, describe, expect, test } from 'bun:test' |
| 2 | import { mkdtemp, rm, writeFile } from 'node:fs/promises' | |
| 3 | import { tmpdir } from 'node:os' | |
| 4 | import { join } from 'node:path' | |
| 2 | 5 | import fixtures from '../fixtures/runtime/frames.json' |
| 3 | 6 | import { commandFrame, decodeRuntimeFrame, encodeRuntimeFrame, runtimeCrc32, type RuntimeFrame } from '../../resources/functions/drone/runtime/protocol' |
| 4 | 7 | import { createRuntimeSession } from '../../resources/functions/drone/runtime/session' |
| @@ -86,13 +89,60 @@ async function withNative(run: (harness: ReturnType<typeof openRuntimeHarness>) | ||
| 86 | 89 | } |
| 87 | 90 | |
| 88 | 91 | describe('actual native kernel process', () => { |
| 92 | test('separate supervised boots have fresh owned identities and a clean idempotent process exit', async () => { | |
| 93 | const first = openRuntimeHarness(artifact), second = openRuntimeHarness(artifact) | |
| 94 | try { | |
| 95 | expect(typeof first.bootId).toBe('bigint') | |
| 96 | expect(first.bootId).not.toBe(0n) | |
| 97 | expect(second.bootId).not.toBe(first.bootId) | |
| 98 | expect((await first.exchange('tick 1000')).bootId).toBe(first.bootId) | |
| 99 | expect((await second.exchange('tick 1000')).bootId).toBe(second.bootId) | |
| 100 | const exit = await first.close() | |
| 101 | expect(exit).toEqual({ exitCode: 0, signal: null }) | |
| 102 | expect(await first.close()).toEqual(exit) | |
| 103 | } | |
| 104 | finally { await first.close(); await second.close() } | |
| 105 | }) | |
| 106 | test('native host startup refuses missing, zero, malformed or extra boot identity arguments', async () => { | |
| 107 | const invalid = [ | |
| 108 | ['--stdio'], ['--stdio', '--boot-id', '0'.repeat(16)], | |
| 109 | ['--stdio', '--boot-id', 'f'.repeat(15)], ['--stdio', '--boot-id', 'F'.repeat(16)], | |
| 110 | ['--stdio', '--boot-id', 'z'.repeat(16)], ['--stdio', '--boot-id', '1'.repeat(16), 'extra'], | |
| 111 | ] | |
| 112 | for (const args of invalid) { | |
| 113 | const child = Bun.spawn([artifact, ...args], { stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' }) | |
| 114 | const [exit, stdout, stderr] = await Promise.all([child.exited, new Response(child.stdout).text(), new Response(child.stderr).text()]) | |
| 115 | expect(exit).toBe(2) | |
| 116 | expect(stdout).toBe('') | |
| 117 | expect(stderr).toBe('') | |
| 118 | } | |
| 119 | }) | |
| 120 | test('a valid-CRC response from a different boot fails the transport and its actual process is reaped', async () => { | |
| 121 | const root = await mkdtemp(join(tmpdir(), 'drone-wrong-host-boot-')), script = join(root, 'wrong-boot.ts'), launcher = join(root, 'launch') | |
| 122 | const protocol = join(import.meta.dir, '../../resources/functions/drone/runtime/protocol.ts') | |
| 123 | await writeFile(script, `#!${process.execPath}\nimport { decodeRuntimeFrame, encodeRuntimeFrame } from ${JSON.stringify(protocol)}\nconst frame = decodeRuntimeFrame(Uint8Array.fromHex(${JSON.stringify(fixtures.frames.status)}))\nframe.bootId = BigInt('0x' + Bun.argv.at(-1)) ^ 1n\nawait Bun.stdin.stream().getReader().read()\nconsole.log(Buffer.from(encodeRuntimeFrame(frame)).toString('hex'))\nawait Bun.sleep(10000)\n`, { mode: 0o700 }) | |
| 124 | // A standalone fixture must not load the application's Bun preloads. | |
| 125 | const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'` | |
| 126 | await writeFile(launcher, `#!/bin/sh\ncd ${quote(root)} || exit 1\nexec ${quote(process.execPath)} ${quote(script)} "$@"\n`, { mode: 0o700 }) | |
| 127 | const harness = openRuntimeHarness(launcher) | |
| 128 | try { | |
| 129 | await expect(harness.exchange('tick 1000')).rejects.toThrow('boot identity') | |
| 130 | await expect(harness.exchange('tick 2000')).rejects.toThrow('failed') | |
| 131 | const exit = await harness.close() | |
| 132 | expect(exit.exitCode).not.toBe(0) | |
| 133 | expect(exit.signal).not.toBeNull() | |
| 134 | expect(await harness.close()).toEqual(exit) | |
| 135 | } | |
| 136 | finally { await harness.close(); await rm(root, { recursive: true, force: true }) } | |
| 137 | }) | |
| 89 | 138 | test('reports timestamped samples and cross-language handshake/lease/stop', async () => withNative(async harness => { |
| 90 | 139 | const first = await harness.exchange('tick 1000') |
| 91 | 140 | expect(first).toMatchObject({ ticks: 1, sample: -999, sampleTimeUs: 1000n, flags: 67 }) |
| 92 | 141 | const link = createRuntimeSession({ nonce: () => 45n }) |
| 93 | 142 | const send = (bytes: Uint8Array) => harness.exchange(`frame ${Buffer.from(bytes).toString('hex')}`) |
| 94 | 143 | const hello = await send(link.hello(1)); link.accept(encodeRuntimeFrame(hello), 1) |
| 95 | expect(hello.bootId).toBe(0x1122334455667788n) | |
| 144 | expect(hello.bootId).toBe(harness.bootId) | |
| 145 | expect(Object.isFrozen(hello)).toBe(true) | |
| 96 | 146 | const lease = await send(link.lease(1)); link.accept(encodeRuntimeFrame(lease), 1) |
| 97 | 147 | expect(lease.leaseUs).toBe(100000) |
| 98 | 148 | expect((await send(link.stop(1))).leaseUs).toBe(0) |
| @@ -126,6 +176,7 @@ describe('actual native kernel process', () => { | ||
| 126 | 176 | expect(report).toMatchObject({ evidence: 'host-harness', fault, neutralOnly: true, leaseActive: false, physicalFlightEnabled: false }) |
| 127 | 177 | expect(report.stateBytes).toBeLessThanOrEqual(128) |
| 128 | 178 | expect(report.deadlineMisses).toBe(fault === 'deadline' ? 1 : 0) |
| 179 | expect(report.bootId).toMatch(/^[1-9][0-9]{0,19}$/) | |
| 129 | 180 | } |
| 130 | 181 | }, 20000) |
| 131 | 182 | test('invalid CLI parameters cannot silently execute a default bench run', async () => { |
| @@ -139,6 +190,8 @@ describe('actual native kernel process', () => { | ||
| 139 | 190 | await expect(harness.exchange('tick 2000')).rejects.toThrow('concurrent') |
| 140 | 191 | expect((await pending).ticks).toBe(1) |
| 141 | 192 | await expect(harness.exchange('not-a-host-command')).rejects.toThrow('exited') |
| 193 | expect(await harness.close()).toEqual({ exitCode: 2, signal: null }) | |
| 194 | await expect(harness.exchange('tick 3000')).rejects.toThrow('failed') | |
| 142 | 195 | }) |
| 143 | 196 | }) |
| 144 | 197 | }) |