ReviewOS

home-lang/home-os

Phase 0 exit: link the Appendix A set into one image (188 undefined symbols)

#41
Closed chris opened this 9 days ago ·
9 days ago

Both per-file ratchets are at 39/39 — every Appendix A file typechecks clean and reaches codegen. They have never been linked together. Doing so is the actual Phase 0 exit gate, and it is a different question with a different answer.

First link of the whole set: 13 duplicate symbols, 188 undefined symbols.

Tracked by the new scripts/mvk-links.sh ratchet: duplicates must stay at zero, undefined is a ceiling that must fall.

Duplicates — fixed

Every one was a real pair of distinct functions. serial.home and vga.home both export writeChar, writeString, writeHex, writeDec, and init; heap.home exports init too; kernel_shutdown, kernel_reboot, scheduler_init, scheduler_tick, fb_console_putchar, fb_console_write, getStats, printStats all appear twice.

export marks a module's public API, not a global C name — callers reach these as serial.writeChar and vga.writeChar, so the module alias is the namespace. Fixed in home-lang/home@b9d0582 by emitting <module>__<name>, with the boot entry points (kernel_main, main, _start) keeping bare names since hand-written assembly calls them. 13 → 0.

Note that some of those pairs are also genuine source-level duplication — kernel_shutdown and kernel_reboot exist in both main.home and core/kernel_init.home, like the duplicate scheduler_wake found earlier. Module scoping makes them link, but two implementations of one kernel operation is still a decision someone has to make.

Undefined — 188, two kinds

1. Functions that were never written. core/foundation.home is called for atomic_add_u64, cpu_halt, cpu_reboot, get_ticks, get_lapic_id — and defines none of them. Same class as socket.home calling a tcp.tcp_accept that did not exist (fixed in #21f5fdc). Code written against an interface nobody implemented.

Worst offenders by module: core_foundation (14), core_process (14), core_memory (10), core_filesystem (10), security_capabilities (9).

2. Appendix A is not import-closed. The set imports modules it does not contain: drivers/ata, drivers/nvme, drivers/e1000, mm/memcg, mm/swap, net/dhcp, net/dns, lib/pantry, lib/den, lib/craft, block_io/scheduler, perf/async_io. Worst: lib_pantry (28), lib_den (23), lib_craft (10).

The plan states Appendix A's "Known gaps: none — every candidate role resolved to a real file". That is wrong, and it was invisible until the set was linked.

What has to happen

  • Decide per module: implement the missing function, or add the module to Appendix A, or cut the call
  • lib/pantry, lib/den, lib/craft are 61 of the 188 and look out of scope for a minimum viable kernel — the MVK probably should not reach into the package manager or the GUI toolkit at all
  • Resolve the genuine duplicate implementations (kernel_shutdown, kernel_reboot, scheduler_init, scheduler_tick)
  • Correct Appendix A's "known gaps: none" claim
  • Then: link, boot, and the Phase 0 gate closes

Blocks #22.

9 days ago

188 → 116. Progress, and the shape of what's left.

Closed so far

ChangeUndefined
First whole-set link188
integration.home dropped from Appendix A (#0c4bbd3)130
core/foundation.home's 14 missing primitives (#a053dea)116

Duplicate symbols remain at 0.

integration.home was pulling the package manager, the den shell, and the Craft GUI toolkit into a minimum viable kernel — 61 symbols, via a single call in main.home commented "legacy integration". Those are Phase 3 and Phase 4 concerns; removed with the reason recorded at the call site so it does not quietly come back.

foundation was called by fourteen names it did not define: cpu_halt, cpu_reboot, read_cr2, get_lapic_id, irq_save, irq_restore, pic_eoi, atomic_add_u64, memory_barrier, mem_copy, mem_set, get_ticks, kernel_panic, serial_write_hex64. Every one is a primitive the rest of the kernel cannot be written without.

What remains — 116, by module

14 core_process        10 core_memory       10 core_filesystem
 9 security_capabilities  6 security_ptr_safety  6 perf_async_io
 5 mm_memcg             4 net_tcp            4 mm_swap
 4 drivers_nvme         4 drivers_ata        4 core_driver_init
 3 perf_boot_opt        3 net_dns            3 drivers_e1000

Two piles, needing different decisions:

  • Missing functions in modules that ARE in the set — , , , . Same as foundation: implement them.
  • Modules NOT in the set — , , , , , , , . Each needs a call: add to Appendix A, or cut the call. Most look like the same over-reach as — a minimum viable kernel probably should not be initialising NVMe, e1000, DNS, and swap.

A checker bug found on the way

Adding four locals to produced five "use of possibly uninitialized variable" warnings in unrelated functions. The checker's may-be-uninitialized set is keyed by name with no scope of its own, so a var value: u64 in one function marked every later value in the file — including parameters, which are initialized by definition. Fixed in home-lang/home@f133a24 by clearing the set at each function boundary.

9 days ago

188 → 95. Duplicates still 0.

ChangeUndefined
First whole-set link188
integration.home dropped from Appendix A130
core/foundation.home — 14 missing primitives116
Networking dropped from the entry's imports109
core/process.home — 15 missing functions95

Appendix A is now 35 files, down from 41. Both per-file ratchets remain at every file in the set, and the boot gate still passes.

Networking removal was the plan's own advice

Appendix A already said net/socket, net/tcp and net/udp were "included only because main.home imports them ... shrinking the entry's imports instead is an acceptable MVK simplification". Linking showed what those imports cost: tcp and udp were imported and never used, and socket was used once — in an IRQ branch for vectors 48–55, network interrupts a kernel with no NIC driver cannot receive. Servicing it dragged drivers/e1000 in behind it.

A real conflict inside process.home

Two context_switch definitions coexisted with different argument conventions: a private one taking slot indices that only reloaded CR3, and the exported name scheduler.home was calling with pids. Whichever the linker picked, one caller was passing the wrong thing. Unified on pids.

Also worth noting: every accessor now goes through a slot_of(pid) lookup rather than indexing the table by pid. Slots are reused and a pid is not its index — indexing by pid reads a different process.

Remaining 95

10 core_memory      10 core_filesystem    9 security_capabilities
 6 security_ptr_safety  6 perf_async_io   5 mm_memcg
 4 mm_swap           4 drivers_nvme       4 drivers_ata
 4 core_driver_init  3 perf_boot_opt      3 core_vfs_permissions
 2 security_audit

core_memory and core_filesystem are in the set and need their missing functions written, same as foundation and process. The rest are modules the set imports but does not contain, each needing the same call as integration.home got: is this actually minimum-viable-kernel work, or Phase 2+?

9 days ago

Progress on the two known gaps (measured at a315094):

Gap 1 — import closure: resolved by addition. Appendix A grew from 37 to 49 files; every added module reaches clean codegen: perf/boot_opt, security/{aslr,smep_smap,audit}, core/vfs_permissions, drivers/{ata,nvme}, fs/{devfs,procfs,sysfs,tmpfs}, perf/async_io, net/dhcp, lib/den_lib. net/dns deferred (26 type diagnostics) — returns with Phase 2 net-echo.

Gap 2 — never-written functions: in progress. Fixed along the way:

  • foundation: volatile MMIO helpers landed in core/memory instead
  • signal_deliver_pending implemented (lowest-signum-first, lock-held dequeue)
  • seccomp_check + syscall_dispatch wrappers over the real filter/dispatch cores
  • caps.home: local bounding/permitted/effective bitmask store
  • den_lib den_execute was moving the Command struct by value through @ptrFromInt — now passes addresses

Compiler (home-lang/home): CallExpr through a fn-typed variable now lowers to an indirect call — wakeup_callback/watchdog_callback/callback were emitting phantom direct-call symbols.

Current ratchets:

parse-rate:    410/410 (100%)
mvk-compiles:  49/49 reach codegen
typecheck:     48/49 zero errors
mvk-links:     0 duplicates, 74 undefined symbols (ceiling 74)
boot-qemu:     PASS

Remaining 74 undefined symbols are the path from "each file compiles" to "the image links": mostly kernel_init's calls into memcg/swap/driver_init and the security depth modules (ptr_safety, capabilities) that still have codegen or type debt.

9 days ago

188 → 31. Duplicates still 0.

ChangeUndefined
First whole-set link188
integration.home dropped from Appendix A130
core/foundation.home — 14 primitives116
Networking dropped from the entry's imports109
core/process.home — 15 functions95
core/memory.home (5) + core/filesystem.home (12)57
12 import-closure modules added to Appendix A48
foundation/serial/pci/timer/memcg — 16 functions38
drivers/pci + drivers/e1000 added31

Appendix A is now 66 files. typecheck 56/56, mvk-compiles 56/56, boot gate green throughout.

Two structural findings

The import-closure treadmill is real. Adding a module resolves its symbols and introduces its own. net/socket.home was tried and reverted: it resolves nine and introduces ten, all calls into net/tcp.home, which this plan deliberately excluded along with the rest of networking. Adding socket without tcp trades one unresolved module for another; adding both undoes a scoping decision made on its merits. Networking returns together at the Phase 2 net-echo gate.

A near-miss worth recording. My check for "is this module already listed" matched §2's honest-baseline table rather than the Appendix A file list, so net/tcp.home looked present when it was not. The link ratchet caught the consequence immediately — the count went up — which is the entire argument for a ratchet that can move in both directions.

Genuine source defects found on the way

  • drivers/pci.home had a hand-written forward declaration of pci_scan_bus alongside the real one. It existed so the parser would accept the recursive call; the backend collects function names in a pre-pass, so it was redundant and had become a duplicate-symbol error.
  • drivers/wifi.home calls a bus/device/function-addressed PCI family — pci_read_bar(bus, device, func, idx), pci_read_irq, pci_enable_bus_master, pci_read_device — none of which exist. Home has no overloading, so this cannot coexist with the handle-addressed pci_read_bar(dev, index) the rest of the tree uses. Wi-Fi is Phase 7a; it needs its own decision.
  • mm/memcg.home referenced usage.swap_current and limits.swap; the fields are usage.swap and limits.swap_max.

Remaining 31

core_driver_init (4)   mm_swap (4)        security_random (3)
perf_boot_opt (3)      drivers_nvme (2)   net_tcp__tcp_init (1)
+ ~14 singletons

core/driver_init has 72 unlowered constructs, mm/swap and security/random have type errors — each needs work before it can join the set. The rest are single missing functions in modules already present.

9 days ago

188 → 20. Duplicates 0 throughout. Appendix A is now 68 files; typecheck 58/58, mvk-compiles 58/58, boot gate green.

ChangeUndefined
First whole-set link188
integration.home dropped130
core/foundation.home — 14 primitives116
Networking dropped from the entry's imports109
core/process.home — 15 functions95
core/memory.home + core/filesystem.home — 1757
12 import-closure modules added48
foundation/serial/pci/timer/memcg — 16 functions38
drivers/pci + drivers/e1000 added31
9 more functions across 6 modules23
mm/swap + security/random added20

Real defects this found

Worth listing, because none of them were visible to a green per-file build:

An operator-precedence bug that disabled a check entirely. mm/swap.home had four instances of

if (entry.flags & SWAP_ENTRY_USED == 0)

== binds tighter than &, so this reads flags & (SWAP_ENTRY_USED == 0) — a bitwise AND against a boolean. The intended test never ran; every swap entry was classified by whatever that expression produced.

A duplicate function with incompatible signatures. security/random.home defined random_add_entropy twice — once taking a word of entropy, once taking a buffer and a length. Both were genuinely called. Whichever the linker kept, the other's callers were passing the wrong arguments; a caller passing one word to the buffer form would have read a length off the stack.

A redundant forward declaration of pci_scan_bus, written so the parser would accept a recursive call. The backend collects function names in a pre-pass, so it was unnecessary and had become a duplicate symbol.

Two context_switch definitions with different argument conventions — one taking slot indices, one taking pids, with the scheduler calling the wrong one.

Struct fields that do not exist: memcg referenced usage.swap_current and limits.swap; the real names are usage.swap and limits.swap_max.

Pointer arithmetic on an array: random.home indexed entropy_pool + 64 on a [u8; 256].

Remaining 20

core/driver_init (4, has 72 unlowered constructs), perf/boot_opt (3), net/tcp__tcp_init, net/socket__socket_init, net/udp__udp_receive, block_io/scheduler (file does not exist), security/smep_smap__cpu_security_init, and a handful of unqualified names (scheduler_tick, serial_interrupt_handler, async_read_complete) that are called without a module prefix.

The networking three stay until the Phase 2 net-echo gate brings that layer back together.

9 days ago

Zero. 64 objects, 0 duplicate symbols, 0 undefined symbols, one ELF.

mvk-links: 64 objects; 0 duplicate symbol(s), 0 undefined symbol(s)
LINKED: the whole Appendix A set produced one image.
StepUndefined
First whole-set link188 (+ 13 duplicate)
integration.home dropped from Appendix A130
core/foundation.home — 14 primitives116
Networking dropped from the entry's imports109
core/process.home — 15 functions95
core/memory.home + core/filesystem.home — 1757
12 import-closure modules added48
foundation/serial/pci/timer/memcg — 16 functions38
drivers/pci + drivers/e1000 added31
9 more functions across 6 modules23
mm/swap + security/random added20
foundation SMAP/CR/CPUID, swap, boot_opt — 1415
block/io_scheduler added7
core/driver_init + its four boot phases3
net/socket, net/tcp, net/arp + 4 TCP ops0

Appendix A went 41 → 74 files as it became genuinely import-closed. All other gates green throughout: parse-rate 410/410, typecheck 64/64, mvk-compiles 64/64, stub-register, and the boot gate.

What per-file compilation could not see

Every one of these was invisible to a green build, and none were found by reading code:

  • An operator-precedence bug that disabled a check entirely. mm/swap.home had four instances of if (entry.flags & SWAP_ENTRY_USED == 0) — == binds tighter than &, so the intended test never ran.
  • A duplicate function with incompatible signatures. random_add_entropy was defined twice, once taking a word and once a buffer plus length. Both genuinely called; whichever the linker kept, the other's callers passed the wrong arguments.
  • Two context_switch definitions with different argument conventions — one taking slot indices, one pids, with the scheduler calling the wrong one.
  • A duplicate scheduler_wake, one a stub whose own comment described what the other already did.
  • 13 duplicate symbols from a language-level gap: export emitted a bare symbol, so serial.writeChar and vga.writeChar collided.
  • Four broken import paths hidden by the Zig-style @import form, which binds a name without resolving anything.
  • Struct fields that do not exist: usage.swap_current, limits.swap, req.base.req_type, bottlenecks[].name.

Compiler work this required

Upstreamed to home-lang/home: module-scoped symbols, type-alias resolution when sizing, indirect calls through function-pointer fields, module-qualified constants in member position, function addresses carrying their module prefix, for/do-while bodies walked when reserving locals, and two analysis-state leaks across function boundaries (may-be-uninitialized and ownership).

What this does not mean

The set links. It has not been booted as a whole — the boot gate still runs mvk_poc.home. Linking 64 modules and running them are different claims, and the second one is the remaining Phase 0 gate item (#22).

Sign in to comment on this issue.