also looking at this
test: raise instead of skipping when a provided live service is missing
#1187
4 files
+194
-9
| @@ -0,0 +1,62 @@ | ||
| 1 | /** | |
| 2 | * Whether a precondition a test needs is allowed to be missing. | |
| 3 | * | |
| 4 | * A gated test — one that needs a live server, or a built `dist/` — has to do | |
| 5 | * something when its precondition is absent. Skipping is right on a developer | |
| 6 | * machine: nobody should have to run Postgres to change a string helper. In CI | |
| 7 | * the same absence means the opposite, because the workflow PROVIDES the | |
| 8 | * precondition: it starts Postgres 17 as a service, waits for `pg_isready`, | |
| 9 | * exports `DATABASE_URL`, and builds before testing. So in CI a missing | |
| 10 | * precondition is a broken job, and skipping turns it into a green tick over | |
| 11 | * tests that never ran. | |
| 12 | * | |
| 13 | * This repository has been bitten by that twice, and both are written up where | |
| 14 | * they happened: | |
| 15 | * | |
| 16 | * - every live-Postgres gate built its URL from `process.env.USER`, which on a | |
| 17 | * runner is `runner`, so every probe failed auth and all 14 files skipped — | |
| 18 | * "CI stayed green while the guards for the two most severe bugs in the | |
| 19 | * release never ran at all" (see the docblock in `pg.ts`) | |
| 20 | * - the CI `test` job did not build, so the published-types guard returned | |
| 21 | * early on every run and checked nothing, which is how an `any` reached | |
| 22 | * `client.d.ts` (see `public-surface-types.test.ts`) | |
| 23 | * | |
| 24 | * Both were fixed by repairing the cause. Neither closed the mechanism, so the | |
| 25 | * next cause — a rotated service password, a service that fails to start, a | |
| 26 | * build step someone reorders — lands in the same silence. That is what this | |
| 27 | * closes: `required()` is true in CI, so the gate raises instead of skipping. | |
| 28 | * | |
| 29 | * `QB_REQUIRE_LIVE` overrides it in both directions: `0` for a CI-like | |
| 30 | * environment that genuinely has no services (a fork running the suite without | |
| 31 | * them), `1` to get CI's strictness locally. | |
| 32 | */ | |
| 33 | ||
| 34 | import process from 'node:process' | |
| 35 | ||
| 36 | export function requiredInCi(): boolean { | |
| 37 | const override = process.env.QB_REQUIRE_LIVE | |
| 38 | if (override !== undefined) | |
| 39 | return override !== '0' && override !== 'false' && override !== '' | |
| 40 | return Boolean(process.env.CI) | |
| 41 | } | |
| 42 | ||
| 43 | /** | |
| 44 | * Report an absent precondition: raise where it must be present, warn and | |
| 45 | * return `false` where skipping is legitimate. | |
| 46 | * | |
| 47 | * Returning the boolean rather than calling `describe.skipIf` here keeps the | |
| 48 | * decision at the call site, where bun:test needs it at module scope. | |
| 49 | */ | |
| 50 | export function gateUnavailable(what: string, reason: string, remedy: string): false { | |
| 51 | if (requiredInCi()) { | |
| 52 | throw new Error( | |
| 53 | `[gate] ${what} is required here but unavailable: ${reason}. ` | |
| 54 | + `CI provides it, so this is a broken job rather than an absent optional dependency — ` | |
| 55 | + `skipping would report a green run over tests that never executed. ${remedy} ` | |
| 56 | + `Set QB_REQUIRE_LIVE=0 if this environment genuinely has none.`, | |
| 57 | ) | |
| 58 | } | |
| 59 | ||
| 60 | console.warn(`[gate] skipping ${what} tests: ${reason}. ${remedy}`) | |
| 61 | return false | |
| 62 | } | |