ReviewOS

stacks/analyticshq

[privacy] Segments + funnels can narrow an aggregate report to one person — decide the k-anonymity floor

#40
Closed glennmichael123 opened this 9 days ago ·
9 days ago

Gap

Segments (#23) can narrow a population before a count is taken, and funnels (#21) report per-step counts. Neither applies a minimum-population floor, so a sufficiently narrow segment reduces an "aggregate" report to the behaviour of one identifiable person.

Every filter in the chain is individually innocuous and genuinely aggregate. The disclosure is created by composing them.

Reproduction

Seeded a modest site — 61 visitors, mostly US/desktop, plus exactly one from Iceland on a tablet — and narrowed with the live filter engine:

-- the site as a whole --
   visitors matching no filter: 61
   funnel: Viewed pricing=31  ->  Started checkout=11

-- narrowing, one innocuous filter at a time --
   country = IS           visitors = 1
     + device = tablet    visitors = 1

-- and the funnel for that segment --
   segment population: 1 visitor(s)
   Viewed pricing       1
   Started checkout     1

Two filters. The result is a complete behavioural timeline for one person, served from endpoints whose entire promise is that they are aggregate.

Note the funnel results endpoint does not accept filters today, so the last block models what applying a saved segment to it would produce. That is the obvious next feature request, which is precisely why the floor should be decided before it ships rather than after.

Why this is ours to care about

This is not the usual k-anonymity setting, where the concern is published data. The dashboard is authenticated and scoped to sites you own or are a member of — you are looking at your own site's visitors.

It matters anyway, because the product's claim is that nobody can identify an individual visitor, including the site owner. That is the whole basis for being cookieless and consent-free: there is no individual to consent about. A report that reconstructs one person's journey contradicts the claim the tracking script's existence rests on.

It also has teeth in the multi-tenant case shipped in #19: a viewer on a shared site is not necessarily someone the owner intended to hand individual-level behaviour to.

Related: the public widget surface (#26) side-steps this entirely by exposing no dimensions and no filters. That was the right answer there — remove the capability rather than guard it — but it is not available here, because segmenting is the feature.

Options

A. k-anonymity floor on segmented results. Suppress any cell whose population is below k (≈5), applied only when a filter or segment is active. Unsegmented totals are never suppressed — "you had 3 visitors yesterday" identifies nobody, and suppressing it would make the product look broken on day one.

  • honest, standard, and the strictest option
  • needs a decision about what a suppressed cell renders as, and that is a UX question as much as a privacy one

B. Minimum segment population. Refuse to apply a segment matching fewer than k identities at all, with a clear message. Fails earlier and is far easier to explain than a scatter of suppressed cells.

  • one check, one message, one place
  • coarser: a segment just over the line still yields near-unique cells downstream

C. Decide the risk is acceptable and write down why. Defensible for a single-owner install; much weaker once #19 memberships and shared dashboards are in play.

Recommendation

B for now, A later if the reports get richer. B is a single check at the point a segment is resolved, it explains itself, and it does not require designing a suppressed-cell state across every chart. A is the more complete answer but pays for that with UX surface in a dozen places.

Either way, k belongs in config/privacy.ts next to retentionDays and respectDnt, so an operator can raise it, and the default should be stated in the privacy documentation rather than buried.

Why this is not already implemented

Turning any of this on changes what every low-traffic filtered view returns. A new site would find most segmented views empty on the first day, which reads as a broken product rather than a working privacy guarantee. That is a product decision, not something to slip in as part of a feature commit — which is why #23 and #21 shipped without it and this exists instead.

Not blocking

Nothing shipped is unsafe today: the funnel endpoint takes no filters, so the composition above cannot currently be performed through the API in one call. This should be settled before funnels accept segments.

9 days ago

Implemented in 1fc500a — option B, the minimum segment population.

A filtered report describing fewer than privacy.minSegmentSize distinct visitors is refused. One check where the segment resolves, rather than suppressing individual cells everywhere: one message, one place, no designed empty state needed in every chart.

The default is on. k = 5, overridable via ANALYTICSHQ_MIN_SEGMENT_SIZE, and only an explicit 0 disables it. A privacy guard that ships disabled is decoration.

Note this parses differently from retentionDays(), where unset means disabled — here a mistyped env var falls back to the default rather than to off, because a typo that silently disabled this would be a privacy regression nothing reports.

Unfiltered reports are never suppressed, however small. "You had 3 visitors yesterday" identifies nobody, and hiding a new install's own totals would make the product look broken rather than careful. count === 0 short-circuits before the population is counted, which also keeps the extra COUNT(DISTINCT) off the most common request there is.

It fails closed. segmentPopulation returns null when the count can't be established, and null suppresses. A guard that fails open isn't a guard — a database blip would otherwise serve exactly the report this exists to withhold.

422, not a 200 carrying zeroes. A suppressed report returned as a success is indistinguishable from a real report of zero, and any client that forgot the flag would render "0 visitors" as though measured. The refusal names the threshold and not the population — "suppressed because 1" would disclose the number being withheld.

The cost, measured rather than argued

The stated objection was that low-traffic sites would find filtered views empty. Against the sites in the development database, counting single-dimension segments withheld at k=5:

sitevisitorssegmentswithheld
test-local177 (100%)
Acme Docs104181 (6%)
Acme Blog150330 (0%)

A real site loses nothing. The one-visitor site loses everything — which is correct, because on that site every segment genuinely is a disclosure.

Verification

14 checks against real Postgres: the exact #40 disclosure suppressed, large segments untouched, the boundary at exactly k serving (< not <=), and both fail-closed paths. Five sabotages fail by name — failing open on an unknown population, an off-by-one at the floor, one endpoint skipping the guard, leaking the population in the refusal, and an env typo disabling it. 316 tests pass.

One thing left open

The funnel results endpoint takes no filters, so it has no segment to check and is untouched. When it accepts one, it must call the same guard — which is exactly why this issue asked for the floor to be settled before funnels accept segments rather than after. Worth a line in whatever issue adds that.

Sign in to comment on this issue.