CI/CD and production deploy: Hetzner, Porkbun DNS

Context

Production for reportshq.org: Hetzner via ts-cloud, push-to-deploy from GitHub Actions, DNS at Porkbun. loghq is the working reference (config/cloud.ts with provider: 'hetzner', mode: 'server'; .github/workflows/deploy.yml with branch-to-env mapping; note loghq's DEPLOY.md still describes an older AWS path and is stale - do not copy it).

CI is the only clean release path: buddy deploy tars the working directory, not the commit, so a laptop deploy ships whatever is checked out. Deploys run from CI, gated on green lint/typecheck/test, with concurrency queuing (cancel-in-progress: false).

Scope

• config/cloud.ts: provider: 'hetzner', mode: 'server', project.slug: 'reportshq' (already set in #2 - verify), sites for reportshq.org + www redirect; decide dedicated box vs attachTo an existing one and document the choice (a dedicated box avoids the shared-box port/slug hazards; if attaching instead, pick ports from ss -lntp on the box, never from config files of other tenants).
• Production database: Postgres on the box (loghq's prod posture) with backups (ts-backups from ~/Code/Libraries if it fits; otherwise a documented pg_dump timer) - an analytics product must not lose the event stream.
• .github/workflows/deploy.yml: branch-to-env (main -> production), needs green CI, secrets: HCLOUD_TOKEN (needed even when attaching - server resolution goes through the Hetzner API), dedicated DEPLOY_SSH_KEY (public half derived in-job with ssh-keygen -y; never reuse another tenant's key), DOTENV_PRIVATE_KEY_PRODUCTION, APP_KEY, Porkbun keys. Post-deploy check: fail the job unless every site answers 200 - deploy exiting 0 does not mean the site works.
• Encrypted .env.production via buddy env:encrypt; verify the release actually loads production env (a known framework foot-gun had a release loading .env.development and baking ciphertext into URLs - test for a sane APP_URL in rendered output post-deploy).
• DNS at Porkbun for reportshq.org: A (+ AAAA only if the box serves IPv6; never AAAA a mail host), www, and mail records if transactional mail runs through the family mail server - schedule/invite emails (#15, #4) need working outbound mail + SPF/DKIM before launch; verify with buddy email:* / buddy dns tooling.
• TLS via the rpx path ts-cloud emits (per-site cert renew units exist and are enabled - audit rpx-cert-renew-reportshq.* after first deploy).
• Staging: stage branch -> a staging env or subdomain, same pipeline.
• Runbook docs/deploy.md: secrets inventory, how to roll back (ts-cloud atomic releases + health gate keep the previous release on failure), how to verify.

Acceptance

• Merge to main deploys automatically; https://reportshq.org serves with valid TLS and correct APP_URL; a deliberately red test blocks the deploy; rollback procedure exercised once.

Depends on #2. Part of #1 (M7 Launch). Ground rules: see #1.

https://reportshq.org is live, deploying on every push to main.

/                200   Let's Encrypt, expires 13 Nov
www              301 → apex
/pricing /features /compare/* /docs   200
/sitemap.xml /robots.txt              200
POST /ingest     401 unauthenticated, 200 with a key

Verified end to end against production rather than trusting status codes: register, log in, create a project, send two events (stored), send a malformed one (dropped, with a stated reason, not silently). The test account, project and events have been deleted.

APP_URL renders as a real origin in canonical/og:url, so the ciphertext foot-gun this issue warns about is not present. rpx-cert-renew-reportshq.timer exists and is scheduled.

Pipeline: the deploy job lives in ci.yml with needs: [lint, typecheck, test] rather than in its own workflow. My first attempt was a separate deploy.yml that re-ran the gates itself and failed on its first push because it had not copied CI's key:generate + migrate steps. One definition of green. A red test blocking the deploy has now been demonstrated twice, both times accidentally and both times correctly.

Attach vs dedicated box: attached to the stacks box. Slug is unique, ports 3150/3158 were taken from ss -lntp, and config/cloud.ts documents why for each.

Secrets are split by blast radius. .env.production is committed encrypted and holds application secrets only, including an APP_KEY that was missing entirely. Hetzner, Porkbun and the SSH key are repo secrets and deliberately not in the shipped env file. The deploy key is dedicated to this tenant.

DNS at Porkbun: apex and www now A + AAAA to the box, replacing the parking ALIAS. mail.reportshq.org stays IPv4-only on purpose.

Runbook: docs/deploy.md.

Two things worth knowing

The blocker was upstream: buddy deploy splices a db:backup --before-migrations in front of a site's migrate step, and stacks 0.70.371 hardcoded the legacy vendored CLI path in it, so it could never run on an app with the current layout. Fixed in stacksjs/stacks eca2844682, released in 0.70.376; upgrading was the whole fix.

A mistake of mine nearly shipped something worse. I declared database as a shared path, which replaced the release's database/migrations/*.sql, so migrate found nothing, said "your production database is already up to date", and created only the auth tables. Every page served correctly with no users/projects/events tables; the first real signup would have been a 500. The shared entry is now the sqlite file. Separately, the default database path sat inside the atomic release directory, so every push would have wiped all customer data with no symptom but its absence.

GET /ingest/verify also 404'd in production (stacksjs/stacks#2326: a root-level GET is answered by the view handler and never reaches the router). Four docs pages told people to curl it. It now answers POST as well, and the docs use POST.

Not done

• Postgres. Running SQLite. The pre-migration dump buddy deploy takes is the only backup; there is no scheduled backup timer yet. Worth deciding before real event volume.
• Staging (stage → staging env) is not set up.
• Rollback is documented but has not been exercised once, which this issue asks for.