ReviewOS

stacks/ts-analytics

Epic: Architecture & Scaling

#111
Closed glennmichael123 opened this 9 days ago ·
9 days ago

Fix the scaling cliff (dormant aggregation pipeline), remove the dead legacy API, enable caching, and settle the ingest path.

Tracks (5):

  • #94 — Scale: activate pre-aggregation (hourly/daily rollups) — handlers full-scan today
  • #95 — Tech debt: remove/reconcile the dead legacy API surface
  • #96 — Perf: enable read-through cache by default + extend coverage
  • #97 — Ingest: finish or remove the SQS buffering path
  • #98 — Scale: load test + DynamoDB capacity plan + documented limits

Part of the Fathom + Sentry product backlog. See #44 for the original roadmap.

9 days ago

All five tracks shipped — closing the epic.

  • #94 — daily pre-aggregation activated: ROLLUP#DAY# items written by an hourly idempotent job; /stats + /timeseries read rollups for complete past days, raw only for the live window
  • #95 — legacy API surface quarantined with LEGACY headers + a README source-of-truth table (live path: src/router.ts + src/handlers/*)
  • #96 — read-through cache on by default, extended to 20 more endpoints (measured 852ms → 15ms on repeat polls)
  • #97 — the half-wired SQS ingest branch removed (it silently broke sessions + non-pageview events when enabled); ingest is cleanly direct, the consumer Lambda stays as standalone opt-in infra
  • #98 — load-test harness (scripts/load-test.ts) + written capacity plan (docs/deploy/scaling.md): raw reads grow linearly with events (sequential 1MB pages), rollups are O(days); per-site partition ceiling ~300–400 pageviews/sec; cost ≈ $3.75/M pageviews

The scaling cliff from the audit is gone: dashboards no longer full-scan raw events, and the breaking points are measured and documented.

Sign in to comment on this issue.