ReviewOS

stacks/bench-review

judges: directory supply + identity accuracy — 8 seeded judges, no importer, no admin CRUD, thin schema, dead correction channel

#50
Closed glennmichael123 opened this 10 days ago ·
10 days ago

Problem

A judge-review platform lives or dies on its judge directory. Today the entire inventory is 8 hand-seeded judges across 6 courthouses, with no way to grow it (no importer, no admin create path, broken public submit form) and no way to keep it accurate (no disambiguation fields, no unique identity, no lifecycle, no correction channel). Misattributed reviews on a same-name judge are first-party platform data — CDA 230 does not shield profile records we author — so identity correctness is a legal exposure, not just a product gap.

Findings

  • Supply is 8 judges / 6 courthouses with no bulk-import pipeline. database/seeders/JudgeSeeder.ts:42-135 (8 judges) and database/seeders/CourtHouseSeeder.ts:29-79 (6 courts) are the only working create paths. No scripts/ dir; CourtListener appears only in opinions-related comments (app/Actions/Admin/Opinions/CreateJudgeOpinionAction.ts:11, routes/api.ts:143).
  • The two nominal create paths are broken — one dangerously. app/Models/Judge.ts:38-41 declares useApi.routes: ['store'], so the framework auto-CRUD generator registers POST /api/judges with no auth (storage/framework/orm/routes.ts:708-709, mounted via storage/framework/core/actions/src/dev/api.ts:142) — contradicting routes/api.ts:109-111's "keep those guarded by auth" note. It's also broken: fillable keys are camelCase (imageUrl, practiceArea) with no snake_case mapping, and court_house_id isn't fillable. A framework bug is filed upstream; bench should not rely on it. Meanwhile the public "Submit a Judge" form (resources/views/judges/submit.stx:37) POSTs to /api/judges/submit, which has no route or action — every submission 404s.
  • Judges schema is too thin for identity or roster data. database/migrations/0000000007-create-judges-table.sql holds only name, image_url, practice_area, court_house_id — no jurisdiction, status (active/retired/deceased), external_id, bio, appointed_year, or slug, even though submit.stx already collects those fields.
  • No unique identity or merge tooling. The name index is non-UNIQUE (0000000082-create-judges_name_index-index-in-judges.sql); app/Actions/Admin/ has no Judges directory. Same-name judges at the same (or null) courthouse are indistinguishable.
  • Correction channel is advertised but dead. resources/views/faq.stx:35 promises a "Suggest an edit" link that exists on no profile component; resources/views/contact.stx:111 offers a "Judge data correction" topic but the form posts to /api/contact, which has no route.
  • Opinion provenance is optional, not enforced. judge_opinions does have citation/source_url/source_provider (0000000032-create-judge_opinions-table.sql:4-10), but source_url is nullable, so admin-entered "Recent Rulings" can publish with zero source.

Acceptance criteria

  • One-shot importer (seeder or CLI) against the CourtListener judges API populating court_houses + judges; at minimum one full jurisdiction imported beyond the 8 seeds
  • judges schema widened: jurisdiction/state, status (active|retired|deceased), external_id, bio, appointed_year, slug
  • Auto-CRUD POST /api/judges removed (drop store from useApi) or replaced by an auth-gated admin action; verified no unauthenticated judge write path remains
  • Admin judge create/edit (and merge-duplicates) actions under app/Actions/Admin/Judges/ with an admin view
  • /api/judges/submit backend implemented (submissions land in a reviewable queue) or the form and FAQ references removed
  • Courthouse + jurisdiction rendered next to the judge name everywhere reviews appear, for same-name disambiguation
  • Working "Report incorrect information" / "Suggest an edit" link on judge profiles, routed to a live intake with judge_id attached
  • source_url (or citation) required before a judge_opinions row is published

From the 2026-06-10 production-readiness audit.

Upstream (framework root causes):

9 days ago

The supply half of this is done; the identity-accuracy half is not.

Was: 8 hand-seeded judges across 6 courthouses, no importer, no bulk path.

Now: 2,740 real sitting federal judges and 202 courthouses, ingested from CourtListener's bulk export by scripts/ingest-courtlistener.ts, with 76 courthouses carrying building-level coordinates from Wikidata and 1,324 judges carrying real education records.

The importer is the part that closes "no way to grow it": it reads the quarterly bulk CSVs (no auth, no API quota), filters to federal jurisdictions, excludes clerks/retired/terminated/deceased/aliases, and supports --dry-run / --snapshot= / --refresh. Its output was validated against an independent Python reference (3361/3361 courts, 16191/16191 positions) after a hand-rolled CSV parser bug was found and fixed — a bare " in an unquoted field was flipping quoted mode and silently merging rows, which had produced 102 courts instead of 3,361.

Spot-checked for correctness rather than just volume: Alito → A.B. Princeton 1972, J.D. Yale 1975.

Still open from this issue: no admin CRUD, no disambiguation fields for same-name judges, no lifecycle (the import is sitting judges only), and the correction channel is still dead. The legal exposure argument in the original description stands — those are the parts that address it, and they are untouched.

Worth noting the fabricated-data cleanup that went with the import: the per-judge pages had been rendering invented biography ("J.D., Stanford Law School", "Bar Admission: California, 2005") and an invented case history with asserted outcomes ("State v. Johnson — Guilty") identically across all 2,740 real named judges. All removed; education is now real-or-empty, and bar admission / firm history / case outcomes are not coming back without a source.

9 days ago

Triage — the premise is stale, the issue is not.

"8 seeded judges" is no longer true: the database holds 2,740 judges, and the build pre-renders 2,743 judge pages plus 202 courthouse pages. Supply is not the emergency it was when this was filed.

What this issue asked for that still does not exist:

  • No importer. Nothing in scripts/ imports judges; the 2,740 arrived as a one-off. There is no repeatable path for the next batch.
  • No admin judge CRUD. routes/api.ts has exactly one admin/judges route, and it creates opinions. An admin cannot add, edit or correct a judge through the app.
  • Thin schema and the dead correction channel are unchanged.

Worth connecting: the search-demand capture added for #57 now feeds this directly. judge_search_queries records every zero-result directory search, aggregated by query with a hit count, and the admin dashboard ranks them as "Judges people couldn't find". That is the import backlog this issue needs, ordered by actual demand rather than guesswork — it just needs ./buddy migrate and some traffic before it has anything in it.

9 days ago

All eight criteria are met. Closing summary of this pass:

Already done, never ticked — the CourtListener importer (scripts/ingest-courtlistener.ts, 2,740 judges / 202 courthouses), the auto-CRUD POST /api/judges removal, and the /judges/submit backend from #48.

Identity columns (64cdc189) — jurisdiction, state, status, external_id, bio, appointed_year, slug, with partial unique indexes. The backfill sets only what is derivable: state from the courthouse, status='active' (true — the importer skips terminated positions and deceased people), and deduped slugs. jurisdiction stays NULL deliberately: every ingested court is federal, so a blanket 'federal' would have put a false jurisdiction on the 8 hand-seeded California state judges. The ingest script now sets it, plus external_id (cl:<person id>), where it is a fact about the source.

Admin CRUD + merge (dec7f877, aa5a4ce3) at /admin/judges. Merge exists because the data needs it: the import keys on (name, courthouse) and a judge can hold two seats, so 22 rows across 10 names are duplicates. Two unique constraints are checked before anything moves — a half-finished merge leaves reviews pointing at a deleted row. A reviewer with a live review of both profiles, or two verified claimants, gets a 409 naming the rows, which the UI lists.

Opinion provenance (f0df001b) — no source_url and no citation means no publish. These are claims we author about how a named judge decided a real case.

Correction channel (5fd9e76a) — POST /api/judges/{id}/correction into the existing submissions queue with the judge attached. The FAQ had promised this link since launch and it existed nowhere.

Courthouse beside the name (6131de85) — composed server-side as one context_line and rendered on the feed, article, my-reviews and reviewer profiles. Absent parts are dropped rather than printed as placeholders.

Follow-ups, not blockers:

  • resources/components/Bench/Feed.stx is included by nothing — dead code with the same markup as ReviewsFeed, worth deleting
  • bio is in the schema and the admin editor but no public surface renders it
  • merge is id-based in the UI; a picker would be friendlier once duplicates are worked through
9 days ago

Closing — all eight acceptance criteria met and verified.

  • CourtListener importer: 2,740 judges across 202 courthouses
  • judges widened with jurisdiction, state, status, external_id, bio, appointed_year and slug, backfilled only where the data supports it
  • no unauthenticated judge write path remains (routes: ['index', 'show'])
  • admin create/edit and duplicate merge under app/Actions/Admin/Judges/, with a page at /admin/judges
  • /api/judges/submit lands in a reviewable queue
  • courthouse + jurisdiction next to the judge name on every surface that lists reviews
  • "Report incorrect information" on each profile, routed to that same queue with judge_id attached
  • a ruling cannot publish without a source URL or a citation

Follow-ups recorded in the comment above rather than held open here: Bench/Feed.stx is dead code, bio has no public surface yet, and the merge control takes an id where a picker would be friendlier.

Sign in to comment on this issue.