ReviewOS

stacks/reportshq

QA hardening and launch checklist

#22
Open chris opened this 8 days ago ·
8 days ago

Context

The pre-launch gate. Everything shipped in #2-#21 gets adversarial verification here - this issue is done when someone could charge money for the product without flinching.

Scope

  • Security pass (stacks-security-audit skill): authz matrix re-verified across every surface (ingest key vs bearer vs share token vs anonymous), cross-project isolation fuzzing, share-link leakage (#16), webhook signature verification (#17), rate limits under concurrency (#6), secrets hygiene (no keys in logs or client payloads).
  • Correctness pass: query engine (#10) fixture audit - hand-computed expected values for every measure/grain/dimension combination on a frozen dataset; rollup-vs-raw equivalence re-run at production scale factor; timezone/DST edge cases (a schedule at 02:30 on a DST-change day, a month-grain query across a year boundary).
  • Billing pass: full Stripe test-mode lifecycle with simulated clock (trial-free subscribe, upgrade, payment failure, dunning, cancel, resubscribe) asserting tier resolution (#17) and limit enforcement (#18) at every step; month-rollover metering.
  • Performance pass: ingest sustained-load test (documented target, e.g. 1k events/s on the prod box shape without 429s below quota), builder preview latency budget (#12), public share under burst (#16 cache holding). Record numbers in the repo.
  • Design pass: stacks-design-taste Section 14 pre-flight re-run across marketing (#19), app shell (#13), builder (#12), chart gallery (#11), emails (#15) - both themes, mobile/tablet/desktop via stacks-browse; accessibility sweep (keyboard-only builder session, focus order, contrast from the #3 token docs, reduced-motion honored); em-dash grep across all user-visible strings must return zero.
  • Docs pass: quickstart (#20) executed verbatim on a clean machine profile; every doc code sample runs.
  • Ops pass: rollback drill (#21), backup restore drill (restore the Postgres backup to a scratch DB and boot the app against it), on-call notes for the known failure modes.
  • Launch checklist committed to the repo with sign-offs per pass.

Acceptance

  • All passes green with evidence linked in the issue (test output, recorded numbers, screenshots where visual); open findings either fixed or explicitly accepted in the checklist.

Depends on all of #2-#21. Part of #1 (M7 Launch). Ground rules: see #1.

Sign in to comment on this issue.