ReviewOS

zig-utils/zig-js

Strings: compare relational operands by representation-aware UTF-16 units

#667
Closed chris opened this 9 days ago ·
9 days ago

Parents: #473, #493 Related: #480

Problem

Abstract Relational Comparison must order String operands lexicographically by UTF-16 code unit. The current comparator accepts raw asStr() slices and always decodes them as canonical WTF-8.

That is correct while every StringCell stores WTF-8, but wrong for the prepared flat-Latin-1 representation. A flat byte sequence such as [0xC3, 0xA9] represents two code units U+00C3/U+00A9; the current iterator decodes it as one canonical UTF-8 U+00E9 scalar. Distinct strings can therefore compare as equal for <, <=, >, and >= once flat storage is enabled.

Scope

  • Compare String Values through the existing representation-aware UTF-16 code-unit iterator.
  • Preserve allocation-free streaming, lone-surrogate ordering, astral high/low surrogate ordering, and numeric/Symbol behavior.
  • Remove the duplicate WTF-8-only relational iterator rather than add a second representation workaround.
  • Exercise tree-walker and bytecode-eligible comparison shapes with ASCII, Latin-1 ambiguity, BMP, astral, and lone-surrogate cases.
  • Compile and run the focused witness with both canonical and physical flat-Latin-1 storage.

No-workaround rules

  • No representation forcing, eager transcode, replacement character, byte-order shortcut, invalid-cell tolerance, Test262 exclusion, or comparator allocation.
  • No stacks, queues, jobs, Promise, Worker, or Map/Set iterator changes.

Acceptance

  • All four relational operators follow exact UTF-16 code-unit order for every supported String representation.
  • Comparison remains streaming and allocation-free.
  • Focused units, relevant nonzero Test262 accounting, and the full unit gate pass with zero regressions.
  • The focused witness passes with flat_storage_active both disabled and enabled.

Implementation and evidence

  • 75e56136 routes relational String Values through the representation-aware iterator and consolidates canonical search matching on that iterator's explicit WTF-8 entry point.
  • The fail-first allocation witness executes <, <=, >, and >= over the ambiguous Latin-1 pair with 0 allocations.
  • Focused tree-walker/VM witnesses: 2/2 passed with canonical storage and 2/2 with physical flat-Latin-1 storage enabled; the switch was restored and was not landed.
  • Test262 relational-operator subtrees: 184 passed, 0 failed.
  • Full unit gate: 1,758 passed, 1 skipped, 0 failed, 0 leaked.
9 days ago

Completed and pushed to main in fix(string): compare relational values by UTF-16 (#667).

  • All four relational operators stream exact UTF-16 units from the StringCell's physical representation.
  • The ambiguous physical flat bytes C3 A9 remain two Latin-1 units rather than collapsing to U+00E9.
  • Tree-walker and forced bytecode VM: 2/2 under canonical storage and 2/2 with physical flat storage enabled.
  • Fail-first allocator: 0 allocations across <, <=, >, and >=.
  • Test262: 184/184 across the four relational subtrees.
  • Full units: 1,758 passed, 1 skipped, 0 failed, 0 leaked.

No user-owned stacks, queues, jobs, Promise, Worker, or Map/Set iterator surface changed.

Sign in to comment on this issue.