ReviewOS

also looking at this

stacks/drone

feat: expose verified metadata persistence and failure phases

#45
Merged chris wants to merge codex/runtime-trust-sdk into main
30 files +1358 -133

Review threads live on the whole diff, not on one commit, so none are shown here - a thread's line means something in the branch's final form, and painting it into an intermediate step would put it on code it is not about.

bun.lockmodified+3-0
Changes to bun.lock
@@ -38,6 +38,9 @@
3838 "version": "0.75.221",
3939 },
4040 },
41 "patchedDependencies": {
42 "tuf-js@6.0.0": "patches/tuf-js@6.0.0.patch",
43 },
4144 "overrides": {
4245 "@stacksjs/bunpress": "0.2.22",
4346 },
docs/runtime-updates.mdmodified+29-0
Changes to docs/runtime-updates.md
@@ -98,6 +98,35 @@ Metadata versions must be positive safe integers. Expiration dates must be real
9898UTC calendar dates in the TUF `YYYY-MM-DDTHH:MM:SSZ` format before the SDK makes
9999expiry comparisons; malformed or automatically normalized dates are rejected.
100100
101### Maintained SDK persistence seam
102
103This repository applies `patches/tuf-js@6.0.0.patch` through Bun's root
104`patchedDependencies`. It extends the pinned MIT-licensed client; these are
105repository-maintained options, not options supplied by unmodified upstream 6.0.0.
106The patch retains upstream signature, role and target verification.
107
108`onVerifiedMetadata(role, bytes)` receives an owned byte copy after the SDK
109accepts remote metadata. The SDK awaits it before writing that cache file or
110advancing to the next role. A rejected observer aborts refresh, including when
111its exception is an HTTP 404. Cache-preferred refresh also propagates persistence
112errors instead of retrying past the rejected observer. Initial root loading and
113cached metadata do not
114emit new observations. An authenticated intermediate root may be observed
115before a later final-root expiry check rejects the refresh. Discard an updater
116after any refresh failure: its upstream in-memory trust set may already have
117advanced. Reusing that instance is not a recovery or persistence retry protocol.
118
119`referenceTime` optionally supplies a finite, copied fixed start time for future
120recovery replay. It does not establish a trusted clock. Historical replay must
121never authorize firmware use: a coordinator must perform fresh verification at
122a trusted current wall time before using a candidate. Existing staging and
123inspection pass neither extension and continue to use the current wall clock.
124
125The seam alone does not persist a protected prefix or make SDK cache writes
126transactional. Its implementation and tests are a prerequisite for the protected
127store and recovery coordinator in issue #42. They do not complete that issue or
128permit flashing, boot selection or physical deployment.
129
101130The selected target must have SHA-256, a nonzero length of at most 16 MiB and
102131exactly these fields in its signed `custom.drone` object:
103132
package.jsonmodified+3-0
Changes to package.json
@@ -82,5 +82,8 @@
8282 },
8383 "overrides": {
8484 "@stacksjs/bunpress": "0.2.22"
85 },
86 "patchedDependencies": {
87 "tuf-js@6.0.0": "patches/tuf-js@6.0.0.patch"
8588 }
8689}
patches/README.mdadded+27-0
Changes to patches/README.md
@@ -0,0 +1,27 @@
1# Maintained dependency patches
2
3## tuf-js 6.0.0
4
5`tuf-js@6.0.0.patch` applies to the npm 6.0.0 release from
6[the TUF JavaScript client](https://github.com/theupdateframework/tuf-js), upstream
7commit `c30653d07f0c41fbe5a2030d25eca6fd04f8547f`. The original MIT copyright and permission
8notice is retained in `tuf-js.LICENSE` and in the installed package.
9
10The patch adds an awaited, copy-owning `onVerifiedMetadata` observer before the
11SDK writes each newly verified remote metadata file, plus an owned finite
12`referenceTime` for recovery replay. Default callers keep upstream verification rules and the current fixed start
13time. Persistence failure aborts rather than being interpreted as a missing
14remote root or a cache miss under `forceCache`. Discard failed updater instances;
15their in-memory trust set is not rolled back. It does not make cache writes
16transactional or implement a protected trust store.
17
18Bun applies the patch through root `patchedDependencies` and the lockfile.
19Always prepare with `bun patch tuf-js@6.0.0` before editing an installed package,
20then generate changes with `bun patch --commit tuf-js@6.0.0`. Do not edit shared
21package caches. For upgrades, review the upstream change against every patch
22hunk and rerun the SDK observer cases, update/enrollment regressions, clean
23frozen installation and copied standalone CLI qualification. Never silently
24carry a patch onto another version.
25
26See `docs/runtime-updates.md` for the trust and clock limits. The extensions are
27maintained here and are not claimed to be unmodified upstream options.
patches/tuf-js.LICENSEadded+21-0
Changes to patches/tuf-js.LICENSE
@@ -0,0 +1,21 @@
1MIT License
2
3Copyright (c) 2022 GitHub and the TUF Contributors
4
5Permission is hereby granted, free of charge, to any person obtaining a copy
6of this software and associated documentation files (the "Software"), to deal
7in the Software without restriction, including without limitation the rights
8to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9copies of the Software, and to permit persons to whom the Software is
10furnished to do so, subject to the following conditions:
11
12The above copyright notice and this permission notice shall be included in all
13copies or substantial portions of the Software.
14
15THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21SOFTWARE.
patches/tuf-js@6.0.0.patchadded+147-0
Changes to patches/tuf-js@6.0.0.patch
@@ -0,0 +1,147 @@
1diff --git a/node_modules/tuf-js/.bun-tag-c3b7c9e27e9980d7 b/.bun-tag-c3b7c9e27e9980d7
2new file mode 100644
3index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
4diff --git a/dist/store.d.ts b/dist/store.d.ts
5index 20f835c29dee82ea3e61c775c83d8a34fad787dc..3b9c2f88cbe9082b45927ccabccb44f1d56d8739 100644
6--- a/dist/store.d.ts
7+++ b/dist/store.d.ts
8@@ -2,7 +2,7 @@ import { Metadata, Root, Snapshot, Targets, Timestamp } from '@tufjs/models';
9 export declare class TrustedMetadataStore {
10 private trustedSet;
11 private referenceTime;
12- constructor(rootData: Buffer);
13+ constructor(rootData: Buffer, referenceTime?: Date);
14 get root(): Metadata<Root>;
15 get timestamp(): Metadata<Timestamp> | undefined;
16 get snapshot(): Metadata<Snapshot> | undefined;
17diff --git a/dist/store.js b/dist/store.js
18index 1b1669029a8db00b7cc594ef202984f3d243a0e4..0ef50b3d8b599b7d49392d7d8c5f63d3c6f0ecce 100644
19--- a/dist/store.js
20+++ b/dist/store.js
21@@ -6,9 +6,9 @@ const error_1 = require("./error");
22 class TrustedMetadataStore {
23 trustedSet = {};
24 referenceTime;
25- constructor(rootData) {
26+ constructor(rootData, referenceTime = new Date()) {
27 // Client workflow 5.1: record fixed update start time
28- this.referenceTime = new Date();
29+ this.referenceTime = new Date(referenceTime.getTime());
30 // Client workflow 5.2: load trusted root metadata
31 this.loadTrustedRoot(rootData);
32 }
33diff --git a/dist/updater.d.ts b/dist/updater.d.ts
34index 883df0fe6a6628ef5d16bdb0de17c0850d7d19ec..5ea5691230b1c59bcec7021c92fb9a13c24c3314 100644
35--- a/dist/updater.d.ts
36+++ b/dist/updater.d.ts
37@@ -9,6 +9,10 @@ export interface UpdaterOptions {
38 fetcher?: Fetcher;
39 forceCache?: boolean;
40 config?: Partial<Config>;
41+ /** Repository-maintained extension: called after verification, awaited before cache persistence. */
42+ onVerifiedMetadata?: (role: string, bytes: Buffer) => void | Promise<void>;
43+ /** Repository-maintained extension: copied fixed start time for recovery replay, not a trusted clock source. */
44+ referenceTime?: Date;
45 }
46 export declare class Updater {
47 private dir;
48@@ -19,6 +23,7 @@ export declare class Updater {
49 private trustedSet;
50 private config;
51 private fetcher;
52+ private onVerifiedMetadata;
53 constructor(options: UpdaterOptions);
54 refresh(): Promise<void>;
55 getTargetInfo(targetPath: string): Promise<TargetFile | undefined>;
56diff --git a/dist/updater.js b/dist/updater.js
57index 94a7658155740fee3f3b7b32937905cc7f791e90..d62b74cd9c6907af7cdd80a80b8160a9452f9db8 100644
58--- a/dist/updater.js
59+++ b/dist/updater.js
60@@ -57,6 +57,7 @@ class Updater {
61 trustedSet;
62 config;
63 fetcher;
64+ onVerifiedMetadata;
65 constructor(options) {
66 const { metadataDir, metadataBaseUrl, targetDir, targetBaseUrl, fetcher, config, } = options;
67 this.dir = metadataDir;
68@@ -64,8 +65,16 @@ class Updater {
69 this.targetDir = targetDir;
70 this.targetBaseUrl = targetBaseUrl;
71 this.forceCache = options.forceCache ?? false;
72+ const referenceTime = options.referenceTime ?? new Date();
73+ if (!(referenceTime instanceof Date) || !Number.isFinite(referenceTime.getTime())) {
74+ throw new error_1.ValueError('Invalid fixed update reference time');
75+ }
76+ if (options.onVerifiedMetadata !== undefined && typeof options.onVerifiedMetadata !== 'function') {
77+ throw new error_1.ValueError('Invalid verified metadata observer');
78+ }
79+ this.onVerifiedMetadata = options.onVerifiedMetadata;
80 const data = this.loadLocalMetadata(models_1.MetadataKind.Root);
81- this.trustedSet = new store_1.TrustedMetadataStore(data);
82+ this.trustedSet = new store_1.TrustedMetadataStore(data, new Date(referenceTime.getTime()));
83 this.config = { ...config_1.defaultConfig, ...config };
84 const userAgent = config?.userAgent
85 ? `${config.userAgent} tuf-js/${package_json_1.version}`
86@@ -92,6 +101,10 @@ class Updater {
87 await this.loadTimestamp({ checkRemote: false });
88 }
89 catch (error) {
90+ // A failed persistence observer/cache write is not a cache miss.
91+ if (error instanceof error_1.PersistError) {
92+ throw error;
93+ }
94 await this.loadRoot();
95 await this.loadTimestamp();
96 }
97@@ -175,7 +188,7 @@ class Updater {
98 // Client workflow 5.3.4 - 5.4.7
99 this.trustedSet.updateRoot(bytesData);
100 // Client workflow 5.3.8: persist root metadata file
101- this.persistMetadata(models_1.MetadataKind.Root, bytesData);
102+ await this.persistMetadata(models_1.MetadataKind.Root, bytesData);
103 }
104 catch (error) {
105 if (error instanceof error_1.DownloadHTTPError) {
106@@ -222,7 +235,7 @@ class Updater {
107 throw error;
108 }
109 // Client workflow 5.4.5: persist timestamp metadata
110- this.persistMetadata(models_1.MetadataKind.Timestamp, bytesData);
111+ await this.persistMetadata(models_1.MetadataKind.Timestamp, bytesData);
112 }
113 // Load local and remote snapshot metadata.
114 // Client workflow 5.5: update snapshot role
115@@ -248,7 +261,7 @@ class Updater {
116 // Client workflow 5.5.2 - 5.5.6
117 this.trustedSet.updateSnapshot(bytesData);
118 // Client workflow 5.5.7: persist snapshot metadata file
119- this.persistMetadata(models_1.MetadataKind.Snapshot, bytesData);
120+ await this.persistMetadata(models_1.MetadataKind.Snapshot, bytesData);
121 }
122 catch (error) {
123 throw new error_1.RuntimeError(`Unable to load snapshot metadata error ${error}`);
124@@ -284,7 +297,7 @@ class Updater {
125 // Client workflow 5.6.2 - 5.6.6
126 this.trustedSet.updateDelegatedTargets(bytesData, role, parentRole);
127 // Client workflow 5.6.7: persist targets metadata file
128- this.persistMetadata(role, bytesData);
129+ await this.persistMetadata(role, bytesData);
130 }
131 catch (error) {
132 throw new error_1.RuntimeError(`Unable to load targets error ${error}`);
133@@ -358,9 +371,13 @@ class Updater {
134 const filePath = encodeURIComponent(targetInfo.path);
135 return path.join(this.targetDir, filePath);
136 }
137- persistMetadata(metaDataName, bytesData) {
138+ async persistMetadata(metaDataName, bytesData) {
139 const encodedName = encodeURIComponent(metaDataName);
140 try {
141+ // The observer owns its copy and must finish before this cache or the next role advances.
142+ if (this.onVerifiedMetadata) {
143+ await this.onVerifiedMetadata(metaDataName, Buffer.from(bytesData));
144+ }
145 const filePath = path.join(this.dir, `${encodedName}.json`);
146 log('WRITE %s', filePath);
147 fs.writeFileSync(filePath, bytesData.toString('utf8'));
tests/unit/updates-sdk.test.tsadded+160-0
Changes to tests/unit/updates-sdk.test.ts
@@ -0,0 +1,160 @@
1import { afterEach, expect, test } from 'bun:test'
2import { copyFile, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises'
3import { tmpdir } from 'node:os'
4import { basename, join } from 'node:path'
5import { Updater, type UpdaterOptions } from 'tuf-js'
6import { repository } from '../fixtures/updates/repository'
7
8const roots: string[] = []
9const servers: ReturnType<typeof Bun.serve>[] = []
10afterEach(async () => {
11 for (const server of servers.splice(0)) await server.stop(true)
12 for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true })
13})
14
15async function setup(options: Parameters<typeof repository>[1] = { rotate: true }, missing?: string) {
16 const root = await mkdtemp(join(tmpdir(), 'drone-sdk-test-')); roots.push(root)
17 const repo = join(root, 'repo'), cache = join(root, 'cache'), requests: string[] = []
18 const fixture = await repository(repo, options)
19 await mkdir(cache, { mode: 0o700 })
20 await copyFile(fixture.rootPath, join(cache, 'root.json'))
21 const server = Bun.serve({ hostname: '127.0.0.1', port: 0, async fetch(request) {
22 const path = new URL(request.url).pathname
23 requests.push(path)
24 if (!/^\/metadata\/(?:[0-9]+\.)?(?:root|timestamp|snapshot|targets)\.json$/.test(path) || basename(path) === missing) return new Response(null, { status: 404 })
25 const file = Bun.file(join(repo, 'metadata', basename(path)))
26 return await file.exists() ? new Response(file) : new Response(null, { status: 404 })
27 } }); servers.push(server)
28 const input = { metadataDir: cache, metadataBaseUrl: `http://127.0.0.1:${server.port}/metadata`, config: { fetchRetries: 0, fetchTimeout: 1000 } } satisfies UpdaterOptions
29 return { root, repo, cache, fixture, requests, input }
30}
31
32const roles = ['root', 'timestamp', 'snapshot', 'targets'] as const
33
34test('the maintained SDK observer receives exact verified bytes before each cache write', async () => {
35 const item = await setup(), seen: string[] = []
36 const updater = new Updater({ ...item.input, async onVerifiedMetadata(role, bytes) {
37 seen.push(role)
38 if (role === 'root') expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(1)
39 else expect(await Bun.file(join(item.cache, `${role}.json`)).exists()).toBe(false)
40 const file = role === 'root' ? '2.root.json' : role === 'timestamp' ? 'timestamp.json' : `1.${role}.json`
41 expect(bytes).toEqual(await readFile(join(item.repo, 'metadata', file)))
42 } })
43 await updater.refresh()
44 expect(seen).toEqual([...roles])
45 expect((await readdir(item.cache)).sort()).toEqual(['root.json', 'snapshot.json', 'targets.json', 'timestamp.json'])
46})
47
48test('a pending observer prevents the cache write and the next metadata request', async () => {
49 const item = await setup()
50 let release!: () => void, entered!: () => void
51 const gate = new Promise<void>(resolve => { release = resolve })
52 const seen = new Promise<void>(resolve => { entered = resolve })
53 const pending = new Updater({ ...item.input, async onVerifiedMetadata(role) {
54 if (role === 'root') { entered(); await gate }
55 } }).refresh()
56 try {
57 await seen
58 expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(1)
59 expect(item.requests).toEqual(['/metadata/2.root.json'])
60 }
61 finally { release(); await pending }
62})
63
64for (const failedRole of roles) {
65 test(`an observer rejection for ${failedRole} aborts before that cache or a later role advances`, async () => {
66 const item = await setup(), seen: string[] = []
67 const updater = new Updater({ ...item.input, async onVerifiedMetadata(role) {
68 seen.push(role)
69 if (role === failedRole) throw new Error('protected persistence rejected')
70 } })
71 await expect(updater.refresh()).rejects.toThrow('Failed to persist metadata')
72 expect(seen).toEqual(roles.slice(0, roles.indexOf(failedRole) + 1))
73 if (failedRole === 'root') expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(1)
74 else expect(await Bun.file(join(item.cache, `${failedRole}.json`)).exists()).toBe(false)
75 })
76}
77
78test('a genuine SDK HTTP404 thrown by the observer cannot be mistaken for the end of root rotation', async () => {
79 const source = await setup({ rotate: false }, 'timestamp.json')
80 const downloadFailure = await new Updater(source.input).refresh().then(() => { throw new Error('Missing metadata unexpectedly succeeded') }, error => error)
81 expect(downloadFailure.statusCode).toBe(404)
82 const item = await setup(), seen: string[] = []
83 await expect(new Updater({ ...item.input, onVerifiedMetadata(role) { seen.push(role); throw downloadFailure } }).refresh()).rejects.toThrow('Failed to persist metadata root')
84 expect(seen).toEqual(['root'])
85 expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(1)
86})
87
88test('the observer owns a byte copy and cannot mutate SDK verification or cache bytes', async () => {
89 const item = await setup(), observed: Record<string, Buffer> = {}
90 await new Updater({ ...item.input, onVerifiedMetadata(role, bytes) { observed[role] = Buffer.from(bytes); bytes.fill(0) } }).refresh()
91 for (const role of roles) expect((await readFile(join(item.cache, `${role}.json`))).equals(observed[role]!)).toBe(true)
92})
93
94test('an unauthorized root never reaches the observer and leaves the initial cache intact', async () => {
95 const item = await setup({ rotate: true, badRotation: true }), seen: string[] = []
96 await expect(new Updater({ ...item.input, onVerifiedMetadata(role) { seen.push(role) } }).refresh()).rejects.toThrow()
97 expect(seen).toEqual([])
98 expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(1)
99})
100
101test('a later invalid target signature preserves only the earlier verified metadata observations', async () => {
102 const item = await setup({ rotate: true, badTargetsSignature: true }), seen: string[] = []
103 await expect(new Updater({ ...item.input, onVerifiedMetadata(role) { seen.push(role) } }).refresh()).rejects.toThrow()
104 expect(seen).toEqual(['root', 'timestamp', 'snapshot'])
105 expect(await Bun.file(join(item.cache, 'targets.json')).exists()).toBe(false)
106})
107
108test('snapshot-bound tampering is rejected before it reaches the observer', async () => {
109 const item = await setup(), seen: string[] = []
110 await writeFile(join(item.repo, 'metadata/1.targets.json'), '{}')
111 await expect(new Updater({ ...item.input, onVerifiedMetadata(role) { seen.push(role) } }).refresh()).rejects.toThrow()
112 expect(seen).toEqual(['root', 'timestamp', 'snapshot'])
113})
114
115test('the supplied fixed start time is owned before callers can mutate it', async () => {
116 const item = await setup(), time = new Date('2030-01-01T00:00:00Z')
117 const updater = new Updater({ ...item.input, referenceTime: time })
118 time.setUTCFullYear(2040)
119 await updater.refresh()
120 expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(2)
121})
122
123test('fixed-time recovery replay is distinct from current-time authorization', async () => {
124 const past = await setup({ expires: '2025-01-01T00:00:00Z' })
125 await new Updater({ ...past.input, referenceTime: new Date('2024-01-01T00:00:00Z') }).refresh()
126 const current = await setup({ expires: '2025-01-01T00:00:00Z' })
127 await expect(new Updater(current.input).refresh()).rejects.toThrow('expired')
128})
129
130test('invalid fixed time is rejected before loading even a missing bootstrap root', () => {
131 expect(() => new Updater({ metadataDir: '/missing-drone-sdk-bootstrap', metadataBaseUrl: 'http://127.0.0.1:1', referenceTime: new Date(Number.NaN) })).toThrow('Invalid fixed update reference time')
132})
133
134
135test('a fully cached trusted refresh does not emit observations for cache bytes', async () => {
136 const item = await setup()
137 await new Updater(item.input).refresh()
138 const seen: string[] = []
139 await new Updater({ ...item.input, onVerifiedMetadata(role) { seen.push(role) } }).refresh()
140 expect(seen).toEqual([])
141})
142
143test('an authenticated intermediate root is observed before final-root expiry rejects the refresh', async () => {
144 const item = await setup({ rotate: true, metadataFields: { rotatedRoot: { expires: '2020-01-01T00:00:00Z' } } }), seen: string[] = []
145 await expect(new Updater({ ...item.input, onVerifiedMetadata(role) { seen.push(role) } }).refresh()).rejects.toThrow('expired')
146 expect(seen).toEqual(['root'])
147 expect(JSON.parse(await readFile(join(item.cache, 'root.json'), 'utf8')).signed.version).toBe(2)
148})
149
150
151test('forceCache cannot swallow a rejected remote timestamp observer when its local cache is missing', async () => {
152 const item = await setup({ rotate: false }), seen: string[] = []
153 await expect(new Updater({ ...item.input, forceCache: true, onVerifiedMetadata(role) {
154 seen.push(role)
155 if (role === 'timestamp') throw new Error('protected timestamp persistence rejected')
156 } }).refresh()).rejects.toThrow('Failed to persist metadata timestamp')
157 expect(seen).toEqual(['timestamp'])
158 expect(item.requests).toEqual(['/metadata/timestamp.json'])
159 expect((await readdir(item.cache)).sort()).toEqual(['root.json'])
160})