ReviewOS

also looking at this

craft-native/craft

Decide dependency advisories instead of counting them

#320
Merged glennmichael123 wants to merge fix/advisory-retention-policy into main
6 files +324 -1
9 days ago

Closes #276.

docs/VULNERABILITY_SCANNING.md ended with "No baseline or vulnerability exemption is added by this workflow". That was accurate, and it was the problem: the scan reported severity counts and stayed green, so six moderate findings looked identical on the day a seventh arrived.

The gate

scripts/advisory-policy.ts reads bun audit --json and fails four ways:

ConditionWhy
An advisory nobody reviewedA new finding is a regression, not a bigger number
A review past its expiryReasoning must not outlive the facts
A retention matching no current advisoryThe list cannot accumulate notes about packages that moved on
Any High or CriticalNever retainable, whatever the entry says

It runs on every pull request and again in validate-release. A failed or unparseable scan throws rather than reading as an empty report — the failure mode that let the old scan pass on no data.

Each entry in scripts/retained-advisories.ts answers one question: by what path would an installed Craft package reach this code? If the answer is "it would", the advisory gets fixed instead.

The two judgement calls

The Svelte peer range stays ^3.0.0 || ^4.0.0. The six moderate advisories reach up to 5.55.6, so the first unaffected version is 5.55.7 — outside the declared range entirely, and bun says so itself: svelte 4.2.20 -> 5.55.7 / blocked by a dependent's range. There is no patched 3.x or 4.x to move to. Closing them means widening the range to Svelte 5, which breaks every consumer on 3.x or 4.x.

Svelte is a peer and dev dependency whose published files are dist and src; the framework is never bundled, so an installed Craft package carries no Svelte code and the version a consumer runs is the one they chose. Every advisory is in Svelte's own SSR renderer, which the binding does not call. A breaking API change to dodge an advisory in code Craft does not ship is the worse trade at 0.0.92 — so it is retained with that reasoning recorded and an expiry. Happy to flip this if you'd rather take Svelte 5 now; it is a product call, not a security one.

bun audit fix is wired into nothing. It "fixes" the esbuild advisory by downgrading to 0.27.2, falling below the window >=0.27.3 <0.28.1 rather than rising above it, and reports Fixed 1 vulnerability. Moving the build toolchain backwards to satisfy a scanner is worse than a recorded retention. This directly contradicts one of #276's own acceptance criteria, which asked for bun audit fix; the issue's alternative — a reviewed severity/allowlist gate — is what landed.

Evidence

  • 9 unit tests cover each failure mode, including that a retention is scoped to its package rather than the id alone.
  • Against the live audit: Advisory policy satisfied: 7 reviewed and unreachable, 0 unreviewed, 0 expired.
  • Mutation: deleting the esbuild retention exits 1 with "nobody has reviewed; add them to scripts/retained-advisories.ts with a reason, or fix them". Restored, exit 0. Verified without a pipe, since the exit code is what CI depends on.
  • bunx --bun pickier . 30 warnings / 0 errors (unchanged), bun run typecheck clean, bun run verify:docs valid, bun test scripts/ 155 pass / 0 fail.

What this does not close

#279 stays open. It is about the SBOM's Grype findings — 30 High across benchmark, fixture and CI-tool paths — and this gate reads bun audit, a different scanner with different output. #316 closed three of its five asks (no swallowed exit codes, scanning scoped to shipped archives, pinned installers). The remaining two want a Grype-side equivalent of what this adds for the lockfile.

6 changed files on the files tab, with 0 review threads.

Sign in to comment on this pull request.