ReviewOS

home-lang/home

Native readdir: preserve buffer Dirent names and recursive name encodings

#897
Open chris opened this 8 days ago ·
8 days ago

Native directory encoding checkpoint (2026-10-14): source 68ac671bf through 5e3cdf8c8 and 2,285 Git-verified evidence artifacts are on main through 9f99fd4f8. Native readdir now returns real Buffer Dirent names and plain Buffer arrays, preserves Buffer root/sibling parent identities and root spelling, applies recursive async encodings, and builds encoded parent displays without changing filesystem traversal bytes. Private worker StringImpl copies, GC roots, conversion cleanup, bulk code-unit copies and a bounded ASCII UTF-8 fast path cover ownership and performance. SDK class/global layouts are unchanged. Final build 56/56; Node/Home encoding/concurrency plus traversal controls pass; ABI checks 10/10 with 58 assertions and Pickier pass. The final unchanged 140-file filesystem/FileHandle batch and all validators pass: 393 cases, 126 process checks, zero failures, 36 original case skips and three complete-file skips. Initial compile/control failures, an unreachable-code file abort, diagnostic memory-ceiling failure and later deadline failures remain recorded. Original iterations/assertions/ten-second deadlines are unchanged. #897 stays open for Linux raw-byte, Windows and lifecycle acceptance; APFS rejects the raw invalid-UTF8 fixture with EILSEQ. #898 tracks the independent pure-path UTF-16 gap; #896 retains robust stress/pinned-reference acceptance. The full feature/suite/all-platform objective remains open.

A same-filesystem Node v26.10.0 / installed Bun 1.4.2 / Home native probe on Darwin arm64 identified two directory-data gaps. Home and installed Bun both expose undefined Dirent.name for fs.readdirSync and callback/promise readdir with { withFileTypes: true, encoding: "buffer" }, including recursive calls. Node returns Buffers containing the actual filename bytes. Home and installed Bun also ignore requested latin1/hex encodings in recursive asynchronous Dirent names while synchronous calls encode names.

The pinned buffer Dirent original only checks the callback error, so its passing process check does not prove name-value correctness. The unchanged opendir test itself describes the existing buffer Dirent gap. Home’s native opendir carrier already preserves raw buffer names, but the native readdir result carrier does not.

Acceptance:

  • Implement native buffer name materialization with correct ownership, raw POSIX filename bytes, Windows UTF-8 conversion, and GC-safe result construction.
  • Preserve parentPath/path types, values and alias behavior against actual Node controls, including recursive directories and Buffer path arguments.
  • Apply requested name encoding consistently in recursive asynchronous traversal, while traversing actual filesystem names.
  • Preserve real Dirent kind methods, Unicode/invalid-byte names, callback/promise/sync semantics and shutdown/error cleanup.
  • Keep original corpus bytes/assertions/deadlines unchanged and validate the wider filesystem selection after the related native changes.

This probe uses installed Bun as a reference, not a rebuilt pinned-source binary. No production change or full-suite acceptance is claimed by the probe.

8 days ago

Parent #66; related #896 and #878. Probe and native regression evidence is published on main. Node v26.10.0 returns Buffer names; Home and installed Bun 1.4.2 return undefined names for the buffer cases tested. The probe records sync and promise modes, Unicode names and recursive latin1/hex behavior. The current native readdir regression passes 140 unchanged files with the explicit original skip gaps retained; it does not prove the missing name/encoding semantics. Production implementation remains pending.

8 days ago

The native batch is implemented locally and queued for compilation: withFileTypes now selects the Dirent carrier before plain buffer arrays, raw buffer names have native ownership, descendant parent Buffers share identity, Buffer root arguments retain caller identity, root strings retain spelling, and recursive async name encoding is applied consistently. Descendant display parents use the encoded relative directory while native traversal keeps filesystem bytes. UTF-16 parent construction preserves lone surrogates. Result arrays are rooted during materialization and remaining native entries are released on conversion failure.

The expanded Node v26.10.0 control passes sync/callback/promise paths, recursive/nonrecursive options, all nine encoding names, string/Buffer roots and trailing separators, parent identity, kinds, retained buffers after GC and a lone-surrogate UTF16 parent. APFS rejects the invalid UTF-8 filename fixture with EILSEQ; that host limitation is recorded and Linux raw-byte acceptance remains required. Native compilation/Home execution and wider regression are pending. No SDK class/global layout fields are added and no corpus assertion/deadline is changed.

8 days ago

The native batch compiles (56/56) and the Node/Home controls now pass all encoding names, sync/callback/promise modes, root Buffer identity, sibling parent identity, plain Node Buffer arrays, multilevel parents, GC retention and lone-surrogate parent strings. APFS’s EILSEQ raw-filename limitation remains explicit.

The first wider 140-file batch is not green: the complete unchanged fs.test.ts process aborts with an unreachable-code panic, while the remaining launches/validators succeed or retain their original skips. The abort has no completed case report and receives no passing case credit. A review found shared StringImpl references in the recursive worker parent builder; the local fix gives each worker a private code-unit copy and adds concurrent traversal coverage. Its cause is still under investigation. A symbols-enabled native rebuild is active so a recurring abort can be located precisely. The prior stripped abort, successful controls and correction history are retained. Full regression/native lifecycle acceptance remains open.

The independent pure-path surrogate gap is tracked in #898. No original assertion, iteration or deadline was changed.

8 days ago

Native directory encoding checkpoint (2026-10-14): source 68ac671bf through 5e3cdf8c8 and 2,285 Git-verified evidence artifacts are on main through 9f99fd4f8. Native readdir now returns real Buffer Dirent names and plain Buffer arrays, preserves Buffer root/sibling parent identities and root spelling, applies recursive async encodings, and builds encoded parent displays without changing filesystem traversal bytes. Private worker StringImpl copies, GC roots, conversion cleanup, bulk code-unit copies and a bounded ASCII UTF-8 fast path cover ownership and performance. SDK class/global layouts are unchanged. Final build 56/56; Node/Home encoding/concurrency plus traversal controls pass; ABI checks 10/10 with 58 assertions and Pickier pass. The final unchanged 140-file filesystem/FileHandle batch and all validators pass: 393 cases, 126 process checks, zero failures, 36 original case skips and three complete-file skips. Initial compile/control failures, an unreachable-code file abort, diagnostic memory-ceiling failure and later deadline failures remain recorded. Original iterations/assertions/ten-second deadlines are unchanged. #897 stays open for Linux raw-byte, Windows and lifecycle acceptance; APFS rejects the raw invalid-UTF8 fixture with EILSEQ. #898 tracks the independent pure-path UTF-16 gap; #896 retains robust stress/pinned-reference acceptance. The full feature/suite/all-platform objective remains open.

Sign in to comment on this issue.