Close the catalog data gap with AllTrails and Strava

Problem

Measured against AllTrails and Strava, our gap is not the feature list — it is that the catalog holds 596,556 trails and almost none of them carry the data someone opens a trail page to read.

What was measured

400 trails sampled live from production on 25 Sep 2026, at offsets 0, 60k, 200k and 400k of the catalog — a straight cut through the middle, not a worst case.

Difficulty across the sample: 310 easy, 21 moderate, 69 hard — graded from length alone, because ascent is missing.

A second sample of the first 200 in the default ordering was barely better: 8/200 with elevation, 5/200 with a real photo.

What that looks like on a page

/trail/105902 (Theodore Solomons Trail) today: "Illustrative photo", elevation gain "Not recorded", a generated description ending "Mapped by the OpenStreetMap community", 0 reviews, 0 records. /trails opens sorted longest-first, so the first screen is the North Country Trail at 1,161 miles, the Grand Enchantment at 1,036 and the Iditarod at 996 — every one of them graded HARD.

Sources for the comparison

• Strava's own feature and subscription page.
• AllTrails' help centre (Plus and Peak benefit articles, feature index) — alltrails.com refuses automated browsers, so their support site stands in for the product tour.
• Wildloop from this repository and from the live API.

The work

Four fixes, in order. The first two are the same problem, and three of the four run against machinery that already exists here.

• #1003 — Backfill elevation gain for the whole catalog
• #1004 — Grade difficulty on ascent, not distance alone
• #1005 — Open the catalog on what is near and well rated, not the longest trails
• #1006 — Put real photos on the trails people actually open

Backlog behind that

Not filed separately yet — say if any of these should be.

What AllTrails has and we do not: conditions forecast (temperature, precipitation, snow depth, air quality) beside our community reports; offline download of a park or a drawn area rather than one trail; waypoints and off-route alerts while recording; park pages; points of interest; custom and shareable lists; sending routes out to Garmin and COROS; printed maps. Peak adds 3D maps, a multi-day route assistant, smart re-routing, plant/insect identification and community heatmaps.

What Strava has and we do not: goals of any kind — the concept does not exist in the codebase; progress and fitness over time; AI activity summaries; finish-time predictions; routes generated from recorded activity volume; Local Legend.

What neither of them has: the territory game, and a trail catalog sitting next to clubs, events and challenges. That is the reason to exist — and it only works on top of a catalog people trust.

The catalog is mostly not trails

Measured while working #1005. This reframes the epic: enriching 596,556 rows is the wrong target, because most of them are not things anybody would set out to walk.

1,200 trails, six offsets spanning the catalog (0 → 500,000), default ordering, 30 Sep 2026.

Median trail length: 0.54 miles.

The catalog is imported from OpenStreetMap, where a way is whatever a mapper drew between two junctions. So two thirds of the rows are path segments, not trails — and one real trail is often several rows. In the first four rows of the default listing today, Gwt Francis Peak To Morgan Co. appears twice (13.88 mi and 1.35 mi) and Canjilon Mountain appears twice (6.72 mi and 0.45 mi).

23.2% of rows share a name with another row in the sample — 1,029 distinct names across 1,200 rows. Continental Divide NST, Link Trail and Red Trail each appear 6 times.

Why this changes the plan

The four fixes here assume the catalog is a list of trails that needs better data. It is closer to a list of path geometry that needs assembling into trails first.

• #1006 (photos) — sourcing a photo for a 0.3-mile unnamed segment is wasted work, and picking which of six Red Trail rows gets it is unanswerable. Merging first makes the target perhaps a quarter the size and each row worth a photo.
• #1004 (difficulty on ascent) — ascent over a fragment is meaningless. Grading needs whole trails.
• #1002 (this issue) — "0% rated" partly reflects that most rows are not things anybody would review.

Same for the third-party comparison: AllTrails does not list six Red Trail entries. Their catalog is curated routes, ours is raw OSM ways. That is the difference behind "they look higher quality than us", more than any single missing field.

Suggested addition to the work list

• Merge or suppress way fragments — collapse rows sharing a name and touching geometry into one trail; keep fragments out of the catalog surface. Probably worth more than any single enrichment, and it shrinks the cost of all three others.

Partial mitigation already landed

d5ec7c1 (#1005) adds a browse_band generated column so the catalog opens on day hikes rather than on thru-hikes or fragments. That fixes the first screen without fixing the data — fragments are demoted, not merged.

Separate bug found while sampling

sort=name returns HTTP 520 past offset ~55,000. Cloudflare 520 with the origin timing out at ~12s.

sort=name  offset=55,000  ->  200 OK   (5.8s)
sort=name  offset=60,000  ->  520      (11.9s)

Not reproducible on the default sort, sort=distance or sort=longest at the same offset — so ordering 596k rows by name has no usable index. The Name chip in the catalog breaks for anyone paging deep. Worth its own issue; say the word and I will file it.

Method

Live production API, limit=200 at offsets 0 / 70,000 / 150,000 / 260,000 / 380,000 / 500,000, de-duplicated by id. Default ordering, which at the time of sampling was still national_trail DESC and therefore length-neutral — d5ec7c1 had not deployed. Elevation 0.1% and rating 0.0% in the same sample, consistent with the 25 Sep numbers above.

Merge-or-suppress fragments is in (48a2caf, 263cca7, a6fbd8c). Pieces are recorded in a new trail_parts table and hidden from the catalog, search, suggestions, the sitemap and coverage counts. Their pages 301 to the trail they belong to, and nothing that points at a trail id changes.

A row folds when it shares an exact name and country with a row it physically joins (the existing clusterRuns geometry check) and starts within 3 miles of it. The kept row is the NPS/USFS record first, then the longest. Rows with reviews or photos, and hand-entered rows, are never folded.

On 8,368 production rows around LA, Boulder and Garmisch, 35% fold: LA 30.5%, Boulder 36.0%, Garmisch 37.6%. Rows listed under 0.4 mi drop from 3,922 to 1,981. Temescal Ridge Trail goes from 10 rows to 1, Mesa Trail from 17 to 3.

trails:fold-fragments runs nightly from 10:40 UTC, and the ingest folds each shard as it writes. Kept trails still carry their own distance and line, so merging the pieces' lines into the kept trail is the natural follow-up.

Measured again, 6 Oct 2026 — the first run comparable to the baseline

400 trails, sampled evenly across the id space rather than by offset, read-only through the public API. Ids come from the sitemaps, so every sampled id is a trail that exists. Same method as buddy trails:catalog-coverage (#1009), which samples by id for the reason the top of this issue could not be re-run: the default ordering changed under #1005, so an offset no longer selects the same trails.

Sample spans ids 21 → 1,430,869. Catalog total reported by the API: 427,023.

Difficulty across the sample: 326 easy, 60 moderate, 14 hard — against 310 / 21 / 69 in September. Neither the difficulty split nor the surface figure is a trend: the two samples are different trails, which is the whole reason the measurement moved to ids.

The elevation backfill is still running

elevation_checked_at on the sampled rows carries timestamps from 25 Sep through today, at roughly 11,000–17,000 trails a day with some idle days. That is about 79,000 of 427,023 checked so far, so on this rate the catalog is covered around early November.

Of the trails it has reached, 66% came back with a real value (49 of 74). The rest were checked and had nothing to give. So the honest projection for #1003 when it completes is elevation on about two thirds of the catalog, from zero in September — and 82% of the catalog has not been looked at yet, which is why the sampled figure is 12% rather than 66%.

What has not moved, and why

• Ratings and reviews are still 0/400 and have no mechanism behind them yet. Nothing in the four closed children addressed this; it is the first thing in the backlog above that a trail page actually reads.
• Photos are 0/400 as expected. #1006 curated the trails people open, which a random sample across 427,000 ids will not land on. Measuring that one needs the engaged set, not this one.

Two things found while measuring

• The catalog was 596,556 trails in September and reports 427,023 now, with the id space reaching 1.43M. Something removed roughly 170,000 rows. Worth knowing what, before this number is read as progress or as loss.
• A trail's own page was still rendering its grade raw, so the 351 of 400 trails with no measured ascent showed a flat "Easy" a few rows above "Elevation gain — Not recorded". #1004 marked the catalog cards and missed the trail page, its facts table, its nearby list and the meta description. Fixed in f0b1842.

Correction to the comment above: nothing was removed. The 596,556 → 427,023 drop is fragment folding (263cca7), not deletion.

trail_parts records each row that is a piece of another trail. A piece is hidden rather than deleted — its row stays, so saved trails, activities, reviews, photos, efforts, segments, page views and plans keep pointing at it, and deleting its trail_parts row lists it again. The catalog filters with id NOT IN (SELECT trail_id FROM trail_parts), and the API's meta.total is literally COUNT(trails) − COUNT(trail_parts), so the ~170,000 "missing" rows are OSM way fragments now presented as parts of the trail they belong to. buddy trails:fold-fragments --dry-run reports the real figure.

That makes the September baseline worse to compare against than the comment above allows for, in a way that matters for reading it: that sample was taken by offset over the unfolded catalog, so some of its 400 trails were way fragments — short, flat, unnamed pieces. They are the rows most likely to be graded easy and least likely to carry an ascent, which flatters the 310-easy split and part of the 0/400 elevation figure. The 6 Oct sample is drawn from the sitemaps, which exclude pieces, so it measures what a visitor can actually browse.

The elevation reading stands on its own either way: it comes from elevation_checked_at on the sampled rows, not from the comparison.