ReviewOS

also looking at this

stacks/bun-query-builder

test: raise instead of skipping when a provided live service is missing

#1187
Merged glennmichael123 wants to merge test/loud-service-gates into main
4 files +194 -9
packages/bun-query-builder/test/gate.tsadded+62-0
Changes to packages/bun-query-builder/test/gate.ts
@@ -0,0 +1,62 @@
1/**
2 * Whether a precondition a test needs is allowed to be missing.
3 *
4 * A gated test — one that needs a live server, or a built `dist/` — has to do
5 * something when its precondition is absent. Skipping is right on a developer
6 * machine: nobody should have to run Postgres to change a string helper. In CI
7 * the same absence means the opposite, because the workflow PROVIDES the
8 * precondition: it starts Postgres 17 as a service, waits for `pg_isready`,
9 * exports `DATABASE_URL`, and builds before testing. So in CI a missing
10 * precondition is a broken job, and skipping turns it into a green tick over
11 * tests that never ran.
12 *
13 * This repository has been bitten by that twice, and both are written up where
14 * they happened:
15 *
16 * - every live-Postgres gate built its URL from `process.env.USER`, which on a
17 * runner is `runner`, so every probe failed auth and all 14 files skipped —
18 * "CI stayed green while the guards for the two most severe bugs in the
19 * release never ran at all" (see the docblock in `pg.ts`)
20 * - the CI `test` job did not build, so the published-types guard returned
21 * early on every run and checked nothing, which is how an `any` reached
22 * `client.d.ts` (see `public-surface-types.test.ts`)
23 *
24 * Both were fixed by repairing the cause. Neither closed the mechanism, so the
25 * next cause — a rotated service password, a service that fails to start, a
26 * build step someone reorders — lands in the same silence. That is what this
27 * closes: `required()` is true in CI, so the gate raises instead of skipping.
28 *
29 * `QB_REQUIRE_LIVE` overrides it in both directions: `0` for a CI-like
30 * environment that genuinely has no services (a fork running the suite without
31 * them), `1` to get CI's strictness locally.
32 */
33
34import process from 'node:process'
35
36export function requiredInCi(): boolean {
37 const override = process.env.QB_REQUIRE_LIVE
38 if (override !== undefined)
39 return override !== '0' && override !== 'false' && override !== ''
40 return Boolean(process.env.CI)
41}
42
43/**
44 * Report an absent precondition: raise where it must be present, warn and
45 * return `false` where skipping is legitimate.
46 *
47 * Returning the boolean rather than calling `describe.skipIf` here keeps the
48 * decision at the call site, where bun:test needs it at module scope.
49 */
50export function gateUnavailable(what: string, reason: string, remedy: string): false {
51 if (requiredInCi()) {
52 throw new Error(
53 `[gate] ${what} is required here but unavailable: ${reason}. `
54 + `CI provides it, so this is a broken job rather than an absent optional dependency — `
55 + `skipping would report a green run over tests that never executed. ${remedy} `
56 + `Set QB_REQUIRE_LIVE=0 if this environment genuinely has none.`,
57 )
58 }
59
60 console.warn(`[gate] skipping ${what} tests: ${reason}. ${remedy}`)
61 return false
62}