ReviewOS

zig-utils/zig-js

Threads: synchronize iterator helper records

#18
Closed chris opened this 9 days ago ·
9 days ago

Parent: #12, #1. Related: #17.

Goal

Give lazy Iterator Helper records (Iterator.prototype.map/filter/take/drop/flatMap, Iterator.concat, Iterator.zip, and Iterator.zipKeyed) a no-GIL synchronization story for their mutable helper state.

Why this is separate from #17

Issue #17 is tracking direct Object.elements side doors. Iterator helpers do contain internal arrays (src, captured methods/keys, opened iterators, flags, padding), but those arrays are only part of the problem. The helper record itself also mutates shared fields during .next() / .return():

  • running
  • done
  • started
  • counter
  • inner
  • inner_next

A same-helper cross-thread call can race those fields even if every backing array read is moved behind Object.elements_lock. So this needs a helper-record lock or an equivalent documented single-owner restriction, not a drive-by element-storage patch.

Acceptance criteria

  • Decide and document whether Iterator Helper objects are single-consumer/thread-affine or shared-realm callable from multiple JS threads.
  • If shared, add a lock or equivalent synchronization around helper record state transitions.
  • Do not hold the helper lock or Object.elements_lock across user JS callbacks, iterator next/return, or other re-entrant calls; snapshot whatever is needed first.
  • Migrate the internal helper arrays away from raw borrowed elements.items slices where they can be observed across calls or used while user code runs.
  • Add focused regression/TSan coverage for re-entrant and cross-thread helper .next() / .return() behavior.
  • Update docs/threads/bindings.md / threading docs with the final ruling.
9 days ago

Landed the first #18 implementation slice.

Commit: bd715f9 (fix(iterator): lock helper entry state)

What changed:

  • Added IterHelper.lock.
  • .next() now checks/publishes running and done under the helper lock, then releases the lock before calling source next or user callbacks.
  • .return() now checks/publishes helper close state under the helper lock, then releases the lock before calling underlying iterator return paths.
  • Added a focused no-GIL witness: a worker enters helper.next() and blocks in the source iterator; a concurrent main-thread helper.next() sees the running helper and throws TypeError instead of racing helper state.
  • Updated docs/threads/bindings.md with the partial ruling.

Validation:

  • zig build test -Dtest-filter="Iterator" --summary none
  • zig build test -Dtest-filter="parallel_js: Iterator helper rejects concurrent next while running" --summary none
  • git diff --check

Remaining before closing #18:

  • Internal helper arrays (concat captured source/method arrays, zip iterator/flag/padding/key arrays) still need snapshot/helper cleanup so raw elements.items slices are not borrowed while user code can run.
9 days ago

Landed the second #18 slice and I think the sync lazy Iterator Helper criteria are now covered.

Commit: b15eea2 (fix(iterator): snapshot helper internals)

What changed in this slice:

  • Iterator.concat now reads the live source/method arrays through locked element helpers instead of raw elements.items access while stepping sources.
  • Iterator.zip / zipKeyed snapshot internal source/key arrays before iterator callouts and use locked element helpers for flags, padding, and result reads/writes.
  • return() close paths snapshot the internal iterator list before invoking underlying return() methods.
  • Added Object.internalElementsSnapshot() for engine-owned element-backed tuples/lists where Array iteration semantics would be the wrong abstraction.
  • Updated docs/threads/bindings.md from the partial state to the final sync-helper ruling.

Validation reused for this exact dirty chunk before committing:

  • zig build test -Dtest-filter='Iterator' --summary none
  • zig build test -Dtest-filter='parallel_js: Iterator helper rejects concurrent next while running' --summary none
  • git diff --check

Notes:

  • Construction-time raw elements fills remain intentionally private-before-publication and are documented that way.
  • I found async iterator helpers still share the IterHelper struct but currently have their own unsynchronized .next() / .return() paths; I’m tracking that separately because this issue was scoped to sync lazy Iterator.* helpers.
9 days ago

Closing: sync lazy Iterator Helper record state and runtime internal-array access are now synchronized/snapshotted in bd715f9 and b15eea2, with focused no-GIL coverage and docs updated. AsyncIterator helper state is separate follow-up work.

Sign in to comment on this issue.