ReviewOS

zig-utils/zig-js

Array: compare default-sort strings by UTF-16 code units

#685
Closed chris opened this 9 days ago ·
9 days ago

Parent: #473

Problem

CompareArrayElements for Array.prototype.sort and Array.prototype.toSorted applies ToString to both values, then performs lexicographic ordering over UTF-16 code units. The runtime currently compares the resulting WTF-8 byte slices with std.mem.order. UTF-8 byte order diverges from UTF-16 code-unit order across the supplementary/BMP boundary; for example U+10000 must sort before U+FFFF because its leading UTF-16 surrogate is 0xD800.

The engine already has a representation-aware, allocation-free UTF-16 string comparator for relational operators. Default array sorting should reuse that owned primitive after preserving ToString order.

Scope

  • Preserve undefined ordering and user comparator behavior.
  • Preserve left-then-right ToString coercion and abrupt completion.
  • Compare default-sort strings as UTF-16 code-unit sequences without materializing UTF-16 buffers.
  • Cover Array.prototype.sort and toSorted through their shared comparator.
  • Add an astral/BMP regression with exact coercion order and a physical StringData witness.

This issue does not change promises, jobs, queues, workers/threads, or Map/Set iterators.

Acceptance

  • U+10000 sorts before U+FFFF under the default comparator.
  • Both operands are coerced once, left before right, in the two-element witness.
  • Canonical and forced-flat physical string storage produce the same result.
  • Focused Debug, ReleaseSafe, and TSan tests pass.
  • Full sharded units report zero failures and zero leaks.
  • The 54-file Test262 Array.prototype.sort subtree is measured before and after; any flip is reported exactly.
9 days ago

Audit update: the sibling Array.prototype.toSorted subtree is also 21/21 before the fix. Its implementation bypassed the shared undefined-last wrapper and fed undefined to the default string comparator; the patch routes both sort and toSorted through CompareArrayElements semantics. The focused witness covers that ordering explicitly. Baselines are therefore sort 54/54 and toSorted 21/21, with no existing corpus flip expected.

9 days ago

Implemented and pushed in https://github.com/zig-utils/zig-js/commit/5c3bcc8a403eda5613cb88ac0c1d9d750de8f169.

Default Array sort comparison now retains ToString results as string Values and streams their UTF-16 code units through the engine representation-aware comparator. Array.prototype.toSorted now also uses the undefined-last CompareArrayElements wrapper instead of coercing undefined through the string comparator.

Validation: focused Debug, forced-flat storage, ReleaseSafe, and TSan 1/1 each; full units 1778 passed, 1 skipped, 0 failed, 0 leaked. Test262 remained sort 54/54 before to 54/54 after and toSorted 21/21 before to 21/21 after; flips 0.

Sign in to comment on this issue.