seo: sitemap/robots pipeline is self-conflicting and hardcoded to localhost — Google cannot discover judge URLs

Problem

There are effectively two sitemaps and both are broken: the static one a host would serve is localhost-poisoned and judge-free; the good dynamic one is unreachable per our own robots rules.

Findings

• The build regenerates dist/sitemap.xml with http://localhost/... URLs (no build.domain in stx.config.ts), listing /jtest, /secret-marketing, /paywall — and zero judges.
• The correct dynamic sitemap (app/Actions/SitemapAction.ts:28 — all judges/courthouses/reviews) lives at /api/sitemap.xml with an APP_URL-localhost fallback.
• public/robots.txt:23 hardcodes Sitemap: http://localhost:4000/api/sitemap.xml while robots.txt:12 Disallows /api/ — the advertised sitemap is inside a disallowed path.

Acceptance criteria

• Exactly one sitemap reachable at the site origin, URLs built from APP_URL (not localhost)
• Sitemap includes all judge profile URLs and excludes /jtest, /secret-marketing, /paywall
• robots.txt Sitemap line templated from APP_URL and not pointing into a Disallowed path

Split out from the SEO bundle of the 2026-06-10 production-readiness audit. Depends on: #46 (per-judge static pages must exist to be worth indexing).

Worth re-reading this one against the current state — the "self-conflicting" half is fixed and the "hardcoded to localhost" half is now deliberate.

Fixed. The sitemap and robots.txt come from one source (app/Helpers/sitemap.ts), used both by the runtime endpoint and by build.ts, so the static artifact and the live endpoint cannot drift. The prefix-match contradiction is gone and documented in place: a bare Disallow: /review also matched /reviews, the public listing that sits in the sitemap at priority 0.9 — telling a crawler to index a page it was simultaneously forbidden to fetch. It is now anchored (/review$ plus /review/). Judge URLs are enumerated from the database, so discovery works.

Deliberate. The localhost base is not a hardcode, it is the unset-APP_URL fallback, and it is chosen so a local build produces an obviously local artifact rather than a plausible-looking production one. Gate check 10 enforces the distinction: with APP_URL set to a non-loopback host, any remaining loopback URL is a hard failure; with it unset, the gate passes but prints

APP_URL unset — artifact carries 35617 loopback URLs and is NOT deployable. Local preview only; set APP_URL to build for deploy.

So this is now the same thing as #85: nothing to fix here, it resolves when a deploy sets APP_URL.

Closing: all three acceptance criteria are met in the pipeline. The remaining localhost in the output is the unset APP_URL, which is #85's job, not this one's.

Exactly one sitemap, at the site origin. The build writes dist/sitemap.xml and dist/robots.txt together from one source. There is no longer a second, better sitemap stranded behind /api/.

It includes the judges and excludes the junk. 2,956 <loc> entries, of which 2,740 are judge profile URLs. /jtest, /secret-marketing and /paywall: zero matches. Two gate checks hold this in place — "every indexable page is in the sitemap" and "every sitemap URL has a built file" — so a sitemap that advertises a 404, or a page that quietly drops out of it, fails the build.

robots.txt no longer contradicts itself. The Sitemap line is Sitemap: {APP_URL}/sitemap.xml — at the origin, not inside the Disallow: /api/ path it used to point into. Gate check 16 ("sitemap and robots.txt do not contradict") enforces it.

The URLs read http://localhost:4000 today only because APP_URL is unset locally; the gate says so out loud on every run — "APP_URL unset — artifact carries 35,614 loopback URLs and is NOT deployable". The host is a build input now rather than something hardcoded, which is what this issue asked for. Setting it is tracked on #85.