ReviewOS

home-lang/home

home-lang.org: per-page OG and meta, docs accuracy, analytics verification, and benchmark provenance

#836
Open glennmichael123 opened this 9 days ago ·
9 days ago

From Chris, in chat:

and then lets make sure we get the marketing page improved some more home-lang.org that it has all the proper og images, meta info, etc. for all pages make sure benchmarks are accurate please as well make sure analyticshq is properly tracking that site too make sure the docs are accurate too improve the docs too (remove stuff like these paths /Users/chrisbreuer/Code/bun/... etc)

Five separate asks that happen to share a surface. Splitting them out, because the benchmark one is not like the others.

Marketing page and metadata

  • OG images for every page, not just the home page. A link to any docs page currently shares with whatever the default is
  • Per-page title and description, distinct rather than templated off the site name
  • Twitter/X card tags, since the accounts being set up now will be posting these links
  • Canonical URLs
  • Check what a real share looks like, in a validator rather than by reading the markup

Docs

  • Accuracy pass against current behaviour
  • Remove local absolute paths such as /Users/chrisbreuer/Code/bun/.... These leak a contributor's directory layout into published docs and are never reproducible for a reader
  • A guard so they cannot come back. A test that greps the docs corpus for /Users/, /home/, and absolute paths under a home directory is a few lines and holds forever. Placeholders like /Users/username/ are fine and should stay allowed, which is what the Stacks docs use

Analytics

  • Confirm AnalyticsHQ is actually receiving pageviews from home-lang.org, by looking at delivered events and not just at the presence of the script tag
  • Decide whether the docs pages are tracked as well as the marketing pages

Benchmarks: the one with a methodology problem

"Make sure benchmarks are accurate" is the hardest item here, and accuracy is not a proofreading pass. From the measurement work on stacksjs/stacks#2779:

Absolute numbers are not comparable across runs on hosted machines. The same stock workload measured 14.90 us/req on one runner and 28.57 us/req on another, for identical source. One run was a 4-core Xeon 8573C, another an EPYC 7763. Any published figure that compares a number from one run against a number from another is measuring the runner, not the thing.

What that means for a published benchmark:

  • Every comparison is within a single run, on one machine, against the alternatives measured in that same run
  • State the repeat count, and publish a median rather than a best-of
  • State the machine, the core count, and the toolchain versions next to the numbers
  • Keep the raw output of the measuring tool as an artifact, so a number can be traced back
  • A ratio against a baseline measured in the same run survives a hardware change; a bare microsecond figure does not
  • Re-measure before publishing. Numbers taken from an older run on different hardware cannot be refreshed by editing the page

If any current figure on the site cannot be traced to a run that measured its comparison points together, it should come down rather than be adjusted.

Acceptance

  • Sharing any page produces a correct, page-specific preview
  • No published doc contains a path from a contributor's machine, and a test fails if one returns
  • AnalyticsHQ shows delivered events from the site
  • Every published benchmark figure names its machine and repeat count, and its comparisons come from one run

Part of stacksjs/stacks#2874, which indexes the five workstreams from the same conversation.

9 days ago

First docs-hygiene batch landed directly on main as 78ad18e. It removes 97 contributor-specific absolute paths from four high-density parity documents, replacing Home-repo references with <repo>, the vendored tsgo checkout with _submodules/typescript-go, and external Bun/Pantry checkouts with user-independent ~/Code paths.

Verification: exactly four files changed; git diff --check passes; BUN_ZIG_SOURCE_AUDIT_2026-05-26.md and PARITY-BUN.md are Pickier-clean; BUN_PARITY_PLAN.md retains its existing 21 warnings; TS_PARITY_PLAN.md retains its exact existing 227 findings. No lint delta and no generic /Users/username placeholder was changed. Eleven contributor paths remain across four lower-density documents and are the next cleanup batch.

9 days ago

Second docs-hygiene batch landed directly on main as e5dd996. It removes the final eleven contributor-specific absolute paths from four documents, using <repo> for in-repository references and ~/Code paths for explicitly external checkouts.

The Markdown inventory now contains zero /Users/chrisbreuer, /Users/chris, or /Users/glennmichaeltorregosa paths. Generic examples remain untouched. Exactly four files changed; git diff --check passes; two files are Pickier-clean, while PANTRY_INTEGRATION.md and the 2026-06-01 audit retain their pre-existing one-warning and three-warning Markdown baselines. Next is a tested guard so these paths cannot return.

9 days ago

Permanent documentation-path regression guard landed in 14054ec9b02deaa97b564dbebbe8e18910011a43 (test(docs): guard contributor home paths (#836)).

  • Scans README.md and all published docs/**/*.md files (97 files currently).
  • Rejects contributor-specific macOS, Linux, and Windows home-directory paths.
  • Allows documented placeholders such as /Users/username and /home/user, while excluding URL, relative-path, and repository-path false positives.
  • Wired into the CI lint job through bun run check:docs-paths.
  • Verified with 4 focused tests (including a black-box failing CLI case), Pickier, dependency-manifest validation, and git diff --check.
9 days ago

Page-title metadata fix landed in be8911aec58cbc0b1c7e8d382c43236590cf8865 (fix(docs): restore page-specific titles (#836)).

The root cause was not Home frontmatter: the locally resolved BunPress 0.1.9 rendered the global site title and description into every page head. Home now requires published BunPress 0.2.17, whose renderer carries page frontmatter/H1 metadata through to <title>, Open Graph, Twitter, canonical, and structured-data output. The two Home pages without an H1 were corrected and given explicit metadata.

Production-build audit after the change: 97/97 HTML pages have complete canonical/OG/Twitter tags, 97 distinct document titles, and 97 distinct canonical URLs. bun install --frozen-lockfile, Pickier, dependency-manifest validation, and git diff --check pass. Descriptions are still generic on most pages; I am keeping that checkbox open and handling those as reviewed content batches rather than claiming completion.

9 days ago

Two reviewed description batches are now on main:

  • 328097c823199cef262f9f9c216c5bfb16103df4 — core capability, TypeScript, type-system, pattern-matching, and getting-started pages.
  • da565657e02e1d9d4e47d93d7bfea70f04ed81e9 — the five introductory guide pages.

Each commit changes exactly five source documents. After each batch I ran Pickier, git diff --check, a guarded production build, and a generated-head audit. The site remains 97/97 pages with matching canonical/OG/Twitter fields and 97 unique titles/canonicals; distinct reviewed descriptions have increased from 6 to 16. The remaining page descriptions are still open.

9 days ago

Two more five-file accuracy/metadata batches landed:

  • 165fd562542b5d0fd1123823990c43f744cff11e and 63e7edab0ede8dfc2cdb0930b85b9915e8aeb41a qualify advanced and feature-reference pages against the capability matrix instead of describing in-progress async, comptime, Result, ownership, FFI, generics, macros, and native optimization as finished.
  • 90c3694aacb6a8be0afc51d46b660b7620588285 moves use-case limitations into the first screen and removes the untraceable “start-up you can measure in microseconds” claim. CLI, game, OS, systems, and web-service pages now distinguish current interpreter/single-entrypoint support from intended native/module/runtime behavior.

Each batch is Pickier-clean, passes git diff --check, and was rendered under the 3,840 MB guard. The generated audit remains 97/97 structurally complete with 97 unique titles/canonicals; reviewed distinct descriptions are now 31.

9 days ago

Three source-backed documentation batches are now on main:

  • ae10d86fb synchronizes parity metadata with the measured runtime/JSC/LSP state and removes stale or nonportable provenance.
  • 8557de4b6 replaces unverified standard-library API claims with an explicit language-facing support boundary, an implementation-source inventory, and an accurate explanation of the Zig-only basics module. It also synchronizes the capability matrix with the live parity page.
  • 5af035fa9 replaces obsolete ion.*, corrupted prose, conflicting lockfile claims, and speculative package-manager promises across the configuration, package, roadmap, and Pantry guides. The pages now document the two actual loader orders, current JSONC/TOML limitations, native home.lock, eight-worker downloader, and native-versus-Pantry routing.

Verification:

  • scripts/measure-parity.sh --diff: runtime 1,449; node 28; JSC 157; compat 16; LSP 76, all in sync.
  • Pickier: zero warnings on every changed page.
  • Both production documentation builds passed under HOME_RUN_MAX_MB=3840, peaking at 237 MB and 246 MB.
  • Generated metadata audit: 97/97 pages structurally complete, 97 unique titles, 97 unique canonical URLs, and 43 distinct descriptions (up from 31 before these batches).
  • Both new commits contain exactly five files and were pushed directly to origin/main under glennmichael123 <gtorregosa@gmail.com> without coauthors.

The issue remains open for the remaining per-page descriptions/OG coverage, live share-validator verification, delivered AnalyticsHQ event verification, and benchmark provenance work.

9 days ago

Progress landed directly on main:

  • 5223de08e (docs: align architecture and tooling guides (#836)): corrected five architecture/tooling pages against the live CLI and repository layout. Removed stale ion.toml, nonexistent src/ion.zig, the old 22-package count, and unsupported completeness claims. Pickier and git diff --check passed; the guarded production docs build peaked at 263 MB. Generated output retained 97 unique page titles and canonical URLs, with distinct descriptions increasing from 43 to 48.
  • b44a54136 (docs: scope core language references (#836)): rewrote five language-reference pages around the behavior evidenced by current fixtures and implementation: closures, default parameters, named arguments, traits, and type inference. Unsupported backend and advanced semantic claims are now explicitly scoped instead of presented as complete. Pickier and git diff --check passed; the guarded production docs build peaked at 209 MB. The generated audit found 97 unique titles, 97 unique canonicals, and 53 distinct descriptions. It also caught a duplicate trait-page title before commit, which was corrected.

I did not count the Phase 12 home test adapter response as a passing feature test; this batch was verified through source/fixture audit plus the production docs build and generated metadata audit. Issue remains open for the remaining page descriptions, a permanent metadata guard, delivered AnalyticsHQ event verification, and benchmark provenance.

9 days ago

Two more verified batches landed directly on main:

  • 9a47a08b9 (docs: correct advanced language feature status (#836)): replaced five overclaiming pages with evidence-scoped references for comprehensions, struct literals, tuple spread, operator overloading, and multiple dispatch. Direct checks establish the current boundary: 30_struct_literal_dot_field.home passes; anonymous struct types are still reported unknown; comprehension results are typed as void; tuple-spread results are non-indexable; operator-overload and multimethod modules remain unintegrated scaffolding. Pickier and the docs-path guard passed. The guarded production build completed at 227 MB peak with 97 unique titles/canonicals and 58 distinct descriptions.
  • bddaf31a0 (ci: gate generated docs metadata (#836)): added a generated-HTML metadata checker, four focused tests, a package command, and a CI gate. It requires exactly one complete description/canonical/Open Graph/Twitter set per page, checks title/description/image consistency, enforces HTTPS home-lang.org URLs, and rejects duplicate titles or canonicals. Its first repository run caught a double-escaped title plus fallback description on UPSTREAM_PARITY_BASELINES; that page was corrected in the same five-file commit. Focused docs tests pass 8/8. The guarded production build peaked at 203 MB, and the checked-in command now reports 97 pages, 97 unique titles, 97 unique canonicals, and 59 distinct descriptions.

The issue remains open: page-specific descriptions are still incomplete (the guard reports diversity but does not misrepresent shared fallback copy as finished), delivered AnalyticsHQ events still need verification, and published benchmark claims still need traceable same-run provenance.

9 days ago

Pushed 6c2abccbf (fix(bench): publish median benchmark reports (#836)) directly to main.

This closes the benchmark-reporting methodology gap in the harness itself:

  • reports now publish medians from every retained fresh-process sample and compute ratios from same-run medians;
  • schema-2 results must record CPU model, machine model where available, logical core count, OS/release, and architecture;
  • reports print pinned compiler versions plus Node, Hyperfine, and Python versions from the verified provenance snapshot;
  • invalid/missing timing samples and incomplete host metadata are rejected rather than silently summarized;
  • the harness guide now distinguishes historical mean-based checkpoints from new median reports, so old figures cannot be relabeled without raw samples.

Verification: 102/102 benchmark-harness regression tests passed under the 3,840 MB supervisor; Pickier passed on all five changed files; an existing verified 30-round artifact rendered successfully as a median report. A live host probe recorded Apple M2 Pro / Mac14,9, 10 logical cores, Darwin 25.3.0 arm64.

Remaining benchmark acceptance work: the published full-suite snapshot 20260905T225537Z is not present locally, benchmark result directories are gitignored, and the repository's benchmark workflow has no retrievable run/artifact in the current GitHub Actions inventory. I am not treating those historical numbers as traceable. Next is durable artifact retention plus a fresh admitted full-suite run before replacing any public table.

9 days ago

Benchmark provenance is now materially complete on main:

  • 6c2abccbf — median-based reports, enforced machine/core metadata, compiler and measurement-tool versions, and invalid-sample rejection.
  • 77c8ec090 — deterministic evidence packaging that preserves every Hyperfine round byte-for-byte, normalizes only repository-local metadata paths to $REPO, adds checksums, and refuses incomplete/changed provenance.
  • 870b4e802 — replaces the stale 20/20 marketing claim with the fresh current-source result, updates README + parity + performance docs, and commits the durable raw-evidence archive.
  • c9eea785b — makes the manual benchmark workflow package both raw and deterministic evidence, retain artifacts for 90 days, and tests that it cannot silently fall back to stale evidence.

Fresh result 20261007T141452Z: source 6c2abccbf, Apple M2 Pro / Mac14,9, 10 logical cores, macOS 26.3.1 arm64, TS 6.0.3, native TS 7.0.2, Node 24.4.1, Hyperfine 1.20.0, Python 3.12.4. It retained 20 workloads × 30 rounds × 3 compilers = 1,800 samples after three warmups. Home has the lowest median on 8/20, not 20/20; the regressions are split into #837 instead of hidden.

Evidence: https://github.com/home-lang/home/raw/main/bench/vs_tsgo/evidence/20261007T141452Z.tar.gz Archive SHA-256: 7ff6a7a0a7d9ee3cb19269274abe70a79566e5a32c45c0c0ed981488257f4a15 Measured Home executable SHA-256: bbb407f640151df4d15f856d3a78d1e3f42693c3543f5328d985c8975df5564f

Verification: 104/104 benchmark-harness tests, 8/8 docs policy tests, Pickier clean, production docs build successful, generated metadata complete for 97 pages, all 40 full-table rows plus 10 README rows machine-checked against raw medians, 602/602 archive checksums verified. Guarded peaks: 3,044 MB build, 465 MB benchmark, 244 MB docs build.

This satisfies the current benchmark methodology/provenance portion of #836. The performance result itself is deliberately not called complete: #837 tracks restoring the twelve losses without weakening admission or semantics. Analytics delivered-event verification and the remaining page-specific metadata/docs work stay open here.

9 days ago

The post-fix benchmark snapshot and durable evidence are now published directly on main.

  • ebffe93d0fff8d8b1da1af5145ea0a2a85c380e0 fixes a general near-quadratic checker regression by indexing and caching the exact @noImplicitThis source fact instead of rescanning the full source for every identifier expression.
  • ff15995f00220d13ceb065a6b781d79b53b75f6a updates the README, parity page, and complete performance document from the former 8/20 current-source result to the new 19/20 result and commits the verified raw-evidence archive.
  • The unchanged 2,048-family workload improved from 44,012.1 ms in the prior full run to 593.0 ms. It is now the sole remaining loss, at 1.44× behind native TS 7; optimization remains tracked in #837 rather than being hidden behind a marketing claim.

Fresh result 20261007T161511Z: Apple M2 Pro / Mac14,9, 10 logical cores, macOS 26.3.1 arm64; TypeScript 6.0.3, native TypeScript 7.0.2, Home 0.1.0; 20 admitted workloads × 30 rounds × 3 compilers = 1,800 retained samples after three warmups.

Evidence: https://github.com/home-lang/home/raw/main/bench/vs_tsgo/evidence/20261007T161511Z.tar.gz Archive SHA-256: 3ce9703591f967f081aa35bf1e704b88f83dae412e7ad51699a8bd99e3e79c8e Measured Home executable SHA-256: 8e13b2e0f2e41e8e3fe1e5d01e1f090a85b85f6ac582555599fd4409e7128e63

Verification: 104/104 benchmark-harness tests, 602/602 archive checksums, byte-identical deterministic regeneration, all published current-snapshot rows checked from raw samples, targeted Pickier clean, 96-page production docs build, and both docs policy gates clean. Guarded peaks remained below the 3,840 MB ceiling.

The requested broader comparison against the Rust and C++ TypeScript ports plus Bun's bun check is recorded in #837 with the same-run fairness requirements. This issue remains open for the remaining page-specific descriptions/OG coverage and delivered AnalyticsHQ event verification.

9 days ago

The final current-source TypeScript snapshot is now published directly on main.

  • 5f90572c77c2c8a21d8ef31337eaf17f2fc4664e fixes the remaining large-predicate scaling regression with a lazy source-root overload index; all 4,472 checker tests and 104 benchmark-harness tests pass.
  • 68ab901de16a783739782723f261806143f35571 updates the README, parity page, and full performance document and commits the deterministic raw-evidence archive.
  • Result 20261007T183321Z retains 20 workloads × 30 rounds × 3 compilers = 1,800 successful samples after three warmups. Home now has the lowest median on 20/20 admitted workloads; the repaired large row is 308.3 ms versus native TS 419.7 ms and tsc 1,229.8 ms.

Evidence: https://github.com/home-lang/home/raw/main/bench/vs_tsgo/evidence/20261007T183321Z.tar.gz Archive SHA-256: de0068831ee0d037b9c103232d7b3eeb51e140c984f8d6be54b866dd355e1cfe Measured Home executable SHA-256: 2423c443af320c1334a0ad19264ef7d6a9ad7d0a323846baedaf3bb6dc4ae77c

The harness report was regenerated from the retained result directory and matched every published median and ratio. Pickier passes, the production docs build generated 96 pages, and the evidence archive verifies all 602 checksums with byte-identical deterministic regeneration.

The requested comparison with the Rust and C++ TypeScript ports plus Bun check is now tracked separately in #838. That issue requires identical admitted projects, equivalent check-only work, pinned revisions/hashes, same-machine rotating runs, and retained raw evidence; screenshot or cross-machine figures will not be reused. #837 is closed as completed. This issue remains open for its non-benchmark metadata, sharing-validator, and delivered AnalyticsHQ-event work.

9 days ago

Two follow-up documentation-path commits are now on main:

  • cdb4023924238ddd9a4e78fa9dbec031e162b8ca normalizes contributor-specific paths in four published JSON evidence documents to <repo>, ~/Code/bun, and ~/.local/share/pantry. Recorded hashes, measurements, and compressed raw artifacts are unchanged.
  • b098534148bc9242ab317734b93410a90278a55a extends the permanent guard from Markdown to every published Markdown and JSON document, adds a black-box JSON failure control with exact location reporting, and documents the placeholder contract.

Verification: all four edited JSON files parse; Pickier and git diff --check pass; the focused policy suite passes 4/4; bun run check:docs-paths now checks 190 files and reports the full Markdown/JSON corpus portable. Both commits were pushed directly under glennmichael123 <gtorregosa@gmail.com> with no coauthors.

Sign in to comment on this issue.