We have a working VPN engine. We do not yet have a VPN platform. This issue separates those two things, surveys how the industry standardizes the second one, and lays out the step-by-step path from here to there — including the macOS UI in system-cleaner and a us-west server fleet.
Follow-up to #28 / #31. Engine gaps stay in #30; this issue is strictly about what sits above the datapath.
1. Where we actually stand
Verified locally on zig 0.17.0-dev.2163, bun 1.4.3-canary.1:
| Check | Result |
|---|---|
bun run build:native | clean |
bun run test:native | 50 Zig tests pass |
bun test packages | 81 pass, 2 skip, 0 fail |
vpn:selftest | keygen / IKpsk2 / transport / replay all ok |
vpn:demo (real UDP) | handshake 1.8 ms, 100/100 both ways, ~15.7k msg/s |
vpn:mesh-demo | coordinator assigned 100.100.0.2/.3, 100/100 through the mesh |
That is a real WireGuard v1 implementation: ~2.8k lines of dependency-free Zig (libltvpn) with Noise_IKpsk2, ChaCha20-Poly1305, RFC-6479 replay, DoS cookies and TUN I/O, plus ~2.3k lines of TS control plane, 7 wired CLI commands and IaC deploys. The commit history is hardening, not greenfield.
The engine is done. Everything in the rest of this issue is the part that isn't.
2. How VPN platforms standardize
There are thousands of VPN products and approximately six planes. Every product is a recombination of the same stack, and knowing which plane you are missing is more useful than knowing which competitor you resemble.
| Plane | What it does | Industry convergence |
|---|---|---|
| 1. Protocol / data | Encrypt and move packets | Settled on WireGuard (Noise_IKpsk2). OpenVPN is legacy, IKEv2 is enterprise/MDM |
| 2. Control | Key distribution, network map, IP allocation | Tailscale coordination server; NetBird splits management + signal; Headscale reimplements Tailscale's server protocol |
| 3. NAT traversal | Get two peers talking through NAT | STUN/ICE for discovery, encrypted relay fallback (Tailscale DERP, NetBird TURN). Relays see ciphertext only |
| 4. Identity & policy | Who may join, which device, reaching what | OIDC/SSO + pre-auth "setup keys" for headless, ACL-as-code, device key expiry + re-auth |
| 5. Name & routing services | Reach peers by name; subnet routers; exit nodes; split DNS | MagicDNS-style host.tailnet.ts.net, advertised routes, exit-node flag |
| 6. Client & fleet | Per-OS clients, server fleet, regions, observability, billing | Per-OS tunnel providers; fleet of exit nodes; RAM-only/no-logs posture + third-party audits |
The two product archetypes
This matters because they imply different roadmaps, and we are currently ambiguous between them:
- Archetype A — mesh / private network (Tailscale, NetBird, ZeroTier, Nebula, Headscale). Peer-to-peer, identity-centric, ACL-driven. The coordinator is the product. Sold per-user/per-device. Hard part: plane 4.
- Archetype B — consumer privacy VPN (Mullvad, Proton, Nord, IVPN). Client → exit node, fleet-centric, no-logs/RAM-only/audits. The server fleet is the product. Sold per-subscription. Hard part: plane 6.
We have the engine for both and the product for neither. Our coordinator shape points at A; the immediate "us-west exit node I actually use" goal is B. Both are reachable from here, and Phase 1 is shared, but the ordering after that differs. This is the first decision below.
De facto standards worth conforming to
Adopting these is mostly free and buys interoperability and user familiarity:
wg(8)conf format +wg-quicksemantics for config interchange — we already emitclient-lt.conf- CGNAT space
100.64.0.0/10(RFC 6598) for tunnel addresses. Our100.100.0.0/16default already sits inside it ✅ - OIDC device authorization grant (RFC 8628) for headless enrollment, plus pre-auth keys for CI/servers
- ACL-as-code policy file, versioned in git, compiled down to per-peer allowed-ips
- STUN (RFC 5389) / ICE (RFC 8445) / TURN (RFC 8656) for traversal
- Device key expiry defaults (Tailscale uses 180 d) with explicit rotation and revocation
- Relay carries ciphertext only — ours already does ✅
Honest plane-by-plane audit
| Plane | Status | Gap |
|---|---|---|
| 1. Protocol/data | ✅ Done | Live kernel-WG interop still unproven (#30) |
| 2. Control | 🟡 Prototype | VpnCoordinator is an in-memory Map. No persistence. IP allocation is a monotonic nextHost++ that never reclaims |
| 3. NAT traversal | 🟡 Partial | Hole punching (MSG_PUNCH) and relay fallback both work and are tested. But endpoint discovery reflects the WebSocket source address + self-reported port — not STUN. Correct for LAN/same-NAT; symmetric NAT falls to relay. Docs oversell this slightly |
| 4. Identity & policy | 🔴 Missing | VpnCoordinator has no authentication whatsoever — any node that connects is registered and handed an IP. No users, orgs, devices, ACLs, expiry or revocation. This is the single biggest gap |
| 5. Name & routing | 🟡 Partial | Exit node + RoutingTable + cryptokey routing done. No DNS/naming, no subnet routers, no advertised-route plumbing |
| 6. Client & fleet | 🔴 Mostly missing | CLI only. Deploy is one box, one region, hardcoded Hetzner fsn1 (Germany). No macOS/Windows GUI, no fleet, no health/latency, no observability |
3. The roadmap
Phase 0 — close the engine's last gaps (prereq, small)
- 0.1 Live interop vs kernel WireGuard /
wireguard-go, both directions — see #30. Ourwg_ref.pybyte-for-byte vectors are strong evidence but are a second implementation by the same author, not the canonical one. Docs claim stock-client compatibility in two places; this either proves it or finds the one byte that's off - 0.2 Replace WS-reflected endpoints with real STUN binding discovery, so symmetric-NAT peers get a direct path instead of silently relaying
- 0.3 Reconcile docs with reality on NAT traversal (
docs/features/vpn.mdfeature summary vs thecoordinator.tscomment) - 0.4 (defer unless Windows is in scope) wintun — #30 item 3
Phase 1 — control plane: identity, persistence, policy (the real platform work)
This is the phase that turns the engine into a platform. Shared by both archetypes.
- 1.1 Persistent store. SQLite behind the coordinator:
orgs,users,devices,device_keys,ip_leases,policies,audit_log. Replaces the in-memoryMap - 1.2 Authentication — close the open coordinator. Staged: (a) pre-auth/setup keys (single-use + reusable, TTL, tag-scoped) which unblocks CI and servers immediately; (b) OIDC device flow (RFC 8628) for interactive enrollment
- 1.3 Device lifecycle. Enrollment, stable device identity, key expiry (default 180 d), rotation, revocation that actually evicts from peers' network maps
- 1.4 ACL-as-code. Policy file (HuJSON-ish), git-versioned, compiled to per-peer allowed-ips. Default-deny for new orgs
- 1.5 IP lease management. Stable per-device addresses, reclamation, explicit subnet config. Fixes the
nextHost++leak - 1.6 Network-map push. Incremental map updates on peer join/leave/revoke rather than full re-broadcast
Phase 2 — network services
- 2.1 DNS / naming — MagicDNS equivalent:
device.<net>.<domain>, split-DNS resolver config per platform - 2.2 Subnet routers — advertise and approve CIDRs so one node fronts a whole LAN
- 2.3 Exit-node advertisement in the control plane (today it's a
vpn:upflag), so clients can choose an exit - 2.4 Standalone relay — DERP-equivalent as its own deployable, with a relay map served by the coordinator and latency-based home-relay selection
Phase 3 — server fleet, us-west first
- 3.1 Region abstraction in
deploy/. Todaydeploy/config.tshardcodesHETZNER_LOCATION = 'fsn1'and defaultsAWS_REGIONtous-east-1. Introduce--regionwith a provider→location→size map - 3.2 us-west box. Options below; Hetzner
hil(Hillsboro, OR) is the price/perf pick, AWSus-west-2(Oregon) if we want to stay in AWS - 3.3 Multi-node fleet state.
deploy/.state.jsonholds one box; make it a fleet keyed by region, soverify/destroyact per node - 3.4 Health + latency endpoint per node so clients pick the nearest exit
- 3.5 Hardening posture. Diskless/RAM-only where the provider allows, no-logs by construction, firewall defaults, unattended upgrades, reproducible provisioning
Region gotchas worth knowing before 3.2. Hetzner's US locations (
hilHillsboro OR,ashAshburn VA) are cloud-only and do not carry thecxIntel line we currently use —BOX_SIZES.hetzner = 'cx23'will not provision inhil. US needs thecpx(shared AMD) orccx(dedicated AMD) lines. More importantly for a VPN: US boxes include ~1 TB egress vs ~20 TB in the EU, which is the binding constraint on an exit node, not CPU. On AWS,us-west-2works but egress is ~$0.09/GB, which is the expensive way to run a tunnel. Hetznerhil+cpxis the recommendation, with the 1 TB cap monitored.
Phase 4 — clients
- 4.1 Headless/CLI — ✅ done
- 4.2 macOS GUI via
system-cleaner— see section 4 - 4.3 Windows (needs 0.4), Linux GUI, iOS/Android — later, and mobile means a NetworkExtension provider on Apple and a
VpnServiceon Android
Phase 5 — platform surface
- 5.1 Admin API + web dashboard (device list, ACL editor, key management)
- 5.2 Observability — per-node metrics, handshake success rate, relay-vs-direct ratio, egress accounting
- 5.3 Billing/licensing, if this becomes commercial
- 5.4 Transparency posture — audit trail, documented data retention, eventually third-party audit. Note the industry lesson: RAM-only hardware proves nothing about what the software ships off-box, and an audit is a point-in-time check
4. The macOS VPN UI in system-cleaner
system-cleaner is a Stacks app with a Craft desktop launcher — already the right shape for this, and it already does privileged system work.
Existing patterns to follow: resources/components/*Panel.stx + resources/views/app/*.stx + an entry in resources/components/AppRail.stx + POST routes in routes/api.ts. Data binds client-side through x-data.
Three constraints from its CLAUDE.md that shape the design:
- The bundle serves prerendered HTML — nothing machine-specific may come from a
<script server>block, or it freezes to the build machine forever. All VPN state must travel over the API and bind client-side, exactly like/api/dashboard-statsdoes. x-dataliterals: no//comments, no double quotes, and never mutate reactive state in place (assign a new value;concat, notpush).- stx only — signals/composables, Crosswind utilities, Iconify icons. No vanilla JS, no hand-rolled SVG.
The privilege question is the real design decision. A macOS app that moves packets has two viable shapes:
| Option | How | Trade-off |
|---|---|---|
(a) Privileged helper + utun ← recommended for v1 | A launchd root helper runs the existing lt vpn:up datapath; the UI talks to it over loopback/XPC | Reuses the engine as-is. No Apple entitlement needed. Fits the current Developer ID signing and the existing NSSystemAdministrationUsageDescription. Does not appear in macOS's own VPN settings |
| (b) NetworkExtension system extension | A NEPacketTunnelProvider packaged as a system extension with com.apple.developer.networking.networkextension = packet-tunnel-provider-systemextension | The "proper" integration — shows up in System Settings, survives reboots, no root helper. But the provider must be a Swift/ObjC system extension; Bun cannot be it. Developer ID builds require the -systemextension suffix (app-extension packaging is Mac App Store only). Significant new surface |
Recommend (a) for v1 and treat (b) as a Phase 4 follow-up if we want App Store distribution or native VPN-settings integration.
Staged delivery:
- 4.2.1 Read-only status panel — no privilege required.
VpnPanel.stx+resources/views/app/vpn.stx+ AppRail entry (i-f7-lock-shield-fill). Card row (state / exit region / tunnel IP / throughput) and a peer table, followingProcessesPanel.stx. Backed by a newPOST /api/vpn-statusthat shells the existinglt vpn:*commands. Ships a real screen with zero new privileged code - 4.2.2 Connect / disconnect — the root helper from (a), plus
POST /api/vpn-connect//api/vpn-disconnect, with the live indicator reusing thelive-indicator/feedConnectedpattern already inProcessesPanel.stx - 4.2.3 Region picker —
POST /api/vpn-regionsfed by the Phase 3.4 health/latency endpoint, so the user picks "us-west" and sees latency - 4.2.4 Identity & keys — surface
vpn:keygenidentity, device name, key expiry; wire to Phase 1 enrollment
Note system-cleaner is a separate repo — these items will be mirrored into an issue there; this list is the contract between the two.
5. us-west servers
Immediate, independent of everything above, and the thing to do first since it's the stated goal:
- 5.a Parameterize region in
deploy/config.ts(--region, provider→location→size map), replacing the hardcodedHETZNER_LOCATION = 'fsn1' - 5.b Switch the US size to the
cpxline —cx23cannot provision in Hetzner US locations - 5.c Deploy
localtunnels-vpn-uswto Hetznerhil(Hillsboro, OR) and runbun run verify:vpnagainst it - 5.d Add egress monitoring — the ~1 TB US allowance is the real limit on an exit node
- 5.e Confirm
deploy/lt-e2e.sh(netns two-host ping + exit routing) passes inhil, which also partly discharges #30 items 2 and 4
6. Decisions needed before Phase 1
- Archetype A or B — identity-driven mesh, or exit-node fleet? Phase 1 is shared; Phase 2 vs Phase 3 ordering flips on the answer. (Current signal: coordinator says A, the us-west goal says B.)
- Self-hosted-first or hosted service? Decides whether the coordinator needs multi-tenancy in Phase 1.1 or can stay single-org.
- Wire-compat as a hard guarantee? If stock WireGuard clients must keep working, 0.1 becomes a CI gate rather than a one-off check.
- macOS: ship (a) and stop, or commit to (b)? Only (b) gets App Store distribution and native VPN-settings integration.
Sources for the standardization survey
Tailscale DERP servers · NAT traversal · policy file syntax · connect to devices · mesh VPN comparison 2026 · NetBird vs Headscale · Apple Network Extension entitlement · appex vs sysex on macOS · Hetzner locations · cx not available in US · RAM-only servers
🤖 Generated with Claude Code