ReviewOS

stacks/analyticshq

[feature] Core Web Vitals — advertised in the nav, not implemented

#41
Closed glennmichael123 opened this 9 days ago ·
9 days ago

Gap

The site sells Core Web Vitals. The product does not have them.

/features/web-vitals is live and says, present tense, no hedge:

Real-user LCP, INP, and CLS collected from the same script, shown next to your traffic.

It is also in the nav mega-menu ("Real-user LCP, INP, and CLS beside traffic"), the footer, the homepage, the features index, and use-cases/publishers.

There is no implementation. Grepping LCP|CLS|INP|TTFB|FCP across every .ts/.stx/.sql outside node_modules, storage/, dist/ and config/pantry returns five files, all of them marketing. No column, no endpoint, no collection in public/script.js, no dashboard panel.

Of the eight features the nav advertises, seven are built. This is the eighth, and it is the only one with neither code nor an issue.

Why it matters

This is the one place where the marketing site describes a capability the product lacks, which makes it a trust problem before it is a feature problem. It is also cheap: the reference implementation exists in ts-analytics (src/Analytics.ts:3140-3205 for collection, src/handlers/performance.ts for the read side).

And it is privacy-clean, which is not true of most things we could add. PerformanceObserver reports timings the browser has already computed for its own rendering; no identifier is involved and nothing new is learned about a person. Guardrail #28 does not touch this.

Proposed

Port it, with four deliberate departures from the ts-analytics version rather than a straight copy:

1. Drop FID. ts-analytics collects it. Google retired FID in favour of INP in March 2024, and our own marketing only ever claims LCP/INP/CLS. Collect LCP, CLS, INP, FCP, TTFB.

2. One beacon per page, not five. ts-analytics fires a separate request per metric as each finalizes. Five metrics on every pageview is a 5x increase in ingest volume for a product whose pitch includes being light. LCP, CLS and INP are only final at visibilitychange → hidden anyway, so buffer and send one combined beacon there (sendBeacon, falling back to fetch(keepalive)), with pagehide as the Safari fallback.

3. Store CLS as a real number. ts-analytics multiplies CLS by 1000 on the wire because its beacon rounds every value to an integer, then divides back at read time in two separate places — which was already the subject of a bug (ts-analytics#133). A numeric column and an unrounded value removes the class entirely.

4. Report p75 only above a minimum sample count. The p75 of one sample is not a percentile, it is that one person's phone. This is the same reasoning as the k-anonymity floor in #40, and it should reuse privacy.minSegmentSize rather than invent a second threshold.

Shape

  • migration: web_vitals (site_id, session_id, path, metric, value, timestamp), FK to sites with ON DELETE CASCADE, indexed (site_id, metric, timestamp)
  • app/Models/WebVital.ts
  • public/script.js: collection, opt-out via data-vitals="false" alongside the existing data-respect-dnt="false"
  • /collect: accept the vitals beacon
  • app/Analytics/vitals.ts: thresholds, good/needs-improvement/poor rating, p75
  • GET /api/sites/{siteId}/vitals and .../vitals-trends
  • a dashboard panel beside the pages report, since that adjacency is precisely what the marketing page promises
  • retention purge, per-visitor erasure (#5) and full-site data deletion must all cover the new table

Priority: P1 — we are already advertising it.

9 days ago

Shipped in ee8ff07.

The nav item now has code behind it. Collection is in public/script.js, storage in migration 47, the read side in app/Analytics/vitals.ts + two endpoints, and the panel sits directly under Top pages — which is the specific adjacency /features/web-vitals promises ("one click from the pages report").

All four departures from the ts-analytics implementation landed as proposed: no FID, one beacon per page instead of five, CLS stored as a real number, and a p75 withheld below the same k as segments.

One thing the issue did not anticipate. Adding a WebVital model made buddy migrate generate its own migration for the table — with value as integer, because the generator maps schema.number() that way. That is precisely the bug the float column exists to prevent: an integer column floors CLS (0.05–0.25) to 0 and every site reports perfect layout stability. It was a no-op only because the hand-written migration ran first, so a database built from the generated one would have had it silently. Deleted the model and the generated migration; web_vitals is now hand-written SQL like segments, funnels and alerts, and there is a test asserting no model exists to regenerate it from.

Verified rather than asserted. 11 checks against a real Postgres — the float survives the round trip, percentile_disc agrees with the JS percentile() on the same rows, the floor withholds a 2-sample metric while reporting an 8-sample one, and the FK cascade fires. Then 16 against dashboard.stx rendered through stx with a real share token, so the panel is filled by the actual query rather than a fixture: LCP reports 2.60s / 8 samples, CLS is withheld as 2 of 5, INP reads No data yet, and the "not affected by the active filter" note appears only when a filter is on.

I also mutated each guarded behaviour and confirmed the tests catch it. Nine of ten did; the tenth was vacuous — it checked that the vitals branch contained a 204 return, and the branch has three, so deleting the last one (letting control fall through into goal matching) kept it green. Replaced with a structural check that the block's own last statement is the return.

Not done, and worth its own issue when someone wants it: a device breakdown. "Is my site slow on mobile" is the question people actually have, and it is purely additive — a column plus a GROUP BY. Vitals are unfilterable until then, and the API response and the panel both say so instead of returning unfiltered numbers that would read as filtered.

390 tests pass, tsc clean, lint clean.

Sign in to comment on this issue.