ReviewOS

stacks/ts-analytics

Epic: Fathom-parity audit — gaps & correctness tracking

#163
Closed glennmichael123 opened this 9 days ago ·
9 days ago

Tracking issue for the adversarial Fathom-parity audit (98 agents; 81/87 findings confirmed against the code; 6 refuted).

Verdict

Not pound-for-pound with Fathom yet. Architecture is genuinely competitive (cookieless daily-salted hashing, single-table DynamoDB, uptime monitoring, full UTM/campaigns, link clicks, funnels, ingest-time bot/spam drop, OAuth+roles). Features are ~70–80% (several headline ones minted-but-inert). Correctness is the gap — confirmed critical data-accuracy and security bugs, and a privacy posture that undersells (live path doesn't honor DNT, no TTL on raw events, full IPs to US third parties).

Feature-parity scorecard

AreaStatus
Tracking — core (pageviews/SPA/events/UTM/clicks/bot-drop)✅ on par
Tracking — revenue/e-commerce❌ broken (#141, #132)
Tracking — GA import / migration❌ missing (#155)
Tracking — IP/path exclusion🟡 not enforced (#159)
Dashboard — breakdowns / realtime / funnels✅ on par
Dashboard — read API / shared / embeds❌ inert (#154, #152)
Dashboard — period comparison❌ broken (#153)
Dashboard — date range / segmentation / export🟡 partial (#139)
Privacy / compliance🟡 primitives solid, posture gaps (#140, #142, #143, #144)
Account / multi-tenant / billing🟡 real auth, billing inert (#156, #157)

Tier 0 — ship-blockers (security + silent data corruption)

  • #160 — CRITICAL: XSS in served minimal tracker
  • #161 — CRITICAL: lost session updates (concurrency)
  • #162 — CRITICAL: late-event undercount across all stats
  • #151 — SESSION# queries lack pagination (1MB truncation)
  • #147 — unvalidated client sessionId
  • #148 — visitor-hash collapse on unknown IP+UA
  • #149 — heatmap points unbounded · #150 — scroll depths unbounded · #134 — event props unbounded
  • #158 — no rate limiting on /collect, /t, key validation
  • #130 — weak RNG (CSPRNG for keys/tokens)

Tier 1 — privacy posture

  • #142 — honor DNT (+ GPC)
  • #143 — TTL on raw events
  • #144 — third-party geo IP / plaintext HTTP / ipAnonymization
  • #140 — GDPR export/delete wrong index (erasure non-functional)

Tier 2 — parity-blocking features

  • #141 — revenue end-to-end (+ #132 events/goals value)
  • #152 — real shared dashboards
  • #154 — read API
  • #153 — period comparison
  • #155 — GA4 importer
  • #159 — enforce excludedIps/excludedPaths

Tier 3 — correctness polish & completeness

  • #145 — bounce/entry race · #146 — unique double-count boundary
  • #133 — CLS x1000 · #136 — timeseries UTC bucketing · #137 — comparison N+1 · #138 — sendBeacon/double-include
  • #135 — server-side session timeout · #131 — multi-tenant leak when auth off · #157 — admin removes owner · #156 — billing
  • #139 — reporting partials · #129 — low-severity backlog

Bottom line: Tier 0 + Tier 1 are the difference between "looks like Fathom" and "trustworthy like Fathom" — mostly small, surgical fixes. Tier 2 is what removes the asterisk on "switch from Fathom."

9 days ago

Closing: the audit is done

Every tier of this epic has shipped and been verified. Updated scorecard:

AreaWasNow
Tracking — core✅✅ + guard/version/idempotency/bot-filter hardening (#138 #179 #169 #166)
Tracking — revenue/e-commerce❌✅ #141 #132
GA import / migration❌✅ #155 (CSV + Data API, both phases)
IP/path exclusion🟡✅ #159 (+ ingest-drop telemetry #175)
Read API / shared / embeds❌✅ #154 #152
Period comparison❌✅ #153 (chart overlay + card deltas) + #137
Date range / segmentation / export🟡✅ #139 (custom/all-time ranges, screen sizes, report CSV; drill-down split → #180)
Privacy / compliance🟡✅ #140 #142 #143 #144 #176 #178 (honest docs, retention at ingest, complete GDPR)
Accuracy & scale (Tier 0 + prod-readiness)❌✅ all of #134–#151, #158–#162, #165–#179 — cluster-safe salts, session GSI, dimensional rollups (All Time works), 675-test harness
Split-domain production—✅ #118
Account / multi-tenant🟡✅ #157 #131; billing remains (#156, needs product decisions)

Still open (tracked separately): #156 billing (needs plan/pricing decisions), #129 polish nits, #180 filter drill-down.

Error tracking was extracted to a standalone product (28a3a02, pre-xray-split tag) — ts-analytics is a pure Fathom/GA competitor, and at this point it is pound-for-pound on the things this audit measured.

Sign in to comment on this issue.