ReviewOS

zig-utils/zig-js

Strings: fuse managed WTF-8 classification and canonicalization

#600
Closed chris opened this 9 days ago ·
9 days ago

Parents: #473, #479 Follow-up to: #599 Related: #461, #493

Evidence

After #599 removes serial FNV, the staged current-main five-second cold profile still records 242 top-of-stack samples in strcell.canonicalizeSurrogates and 115 in strcell.contentHash. Managed string construction currently scans the same input independently to detect the 0xED surrogate lead and to derive ASCII/Latin-1 flags before hashing. The common no-surrogate path then performs an unchanged byte copy.

Surrogate-pair folding cannot change either classification: any input containing a WTF-8 surrogate is already non-ASCII and non-Latin-1, and its folded astral UTF-8 output remains non-ASCII and non-Latin-1. That lets one exact pre-copy classification serve canonicalization and cached hash flags without weakening validation.

Scope

  • Produce surrogate presence plus ASCII/Latin-1 classification in one vectorized pass.
  • Thread that immutable result through managed borrowed/owned canonicalization and cached hashing.
  • Preserve the public standalone canonicalization and content-hash helpers through wrappers that compute their own classification once.
  • Keep the no-surrogate owned-allocation transfer, allocator mismatch copy, paired-surrogate fold, lone-surrogate preservation, and all OOM ownership rules exact.

Constraints

  • No skipped canonicalization or validation, borrowed lifetime extension, input-specific recognition, hidden interning, or unbounded cache.
  • Exact bytes remain authoritative on cached-hash collisions.
  • Classification reuse must be proven valid for ASCII, Latin-1, BMP, astral, paired-surrogate, and lone-surrogate inputs.
  • Do not touch stacks, queues, jobs, Promise, microtasks, Worker lifecycle, or scheduler implementation.

Acceptance

  • Exact-parent production cold medians and collapsed leaves improve with identical checksum and attribution inventories.
  • Tests cover classification equivalence before/after every canonicalization category and source/target allocator ownership.
  • String/RegExp/JSON/Temporal-adjacent test262, focused normal/TSan/OOM/GC/C-API tests, and full units pass without regressions or leaks.
9 days ago

Rejected after exact-parent measurement; no code landed.

The candidate fused the WTF-8 surrogate-presence scan with ASCII/Latin-1 classification and carried the result through canonicalization and hashing. Against exact parent 043f9788:

  • 101 alternating cold start pairs: 10.464 ms -> 10.380 ms median (-0.802%); candidate won 68/101.
  • Dedicated representative_strings workload (1,000 iterations), 9 alternating pairs: 1.000349 s -> 1.000249 s median (-0.010%); candidate won 5/9.
  • Sampling did not reduce total relevant leaf work: the fused classifier migrated work into canonicalizeSurrogatesClassified while XXH3 leaf cost remained effectively present.
  • Focused ownership, surrogate-canonicalization, managed-GC finalization, allocator-mismatch, and classification tests passed, so the rejection is based on value rather than correctness.

The ~0.8% cold-start signal does not justify an extra classification type/API or proof obligation when the isolated strings workload is neutral. The candidate was fully reverted; tracked HEAD remains clean at 043f9788.

Sign in to comment on this issue.