ReviewOS

craft-native/craft

[S3-017] Harden native and browser checkpoint contracts

#396
Open chris opened this 8 days ago ·
8 days ago

S3-017: Harden native and browser checkpoint contracts

Parent and ownership

Parent epic: https://github.com/stacksjs/the-settlers-iii-remake/issues/6 Source owner: craft-native/craft. Master roadmap: https://github.com/stacksjs/the-settlers-iii-remake/issues/4.

Problem and observable result

Specify ordered atomic checkpoint completion. Preserve the previous valid checkpoint after interrupted writes. Propagate cancellation and storage failures.

The current application is a browser prototype. Passing the Riverlands campaign does not establish original-game or native parity. This issue is complete only when its stated behavior has retained evidence.

Evidence and interfaces

Craft owns graphics, native hosting, and platform capabilities. stx owns compilation/reactivity. ts-css owns style semantics. Stacks owns build and release orchestration. The game consumes published contracts.

Pin the inspected revisions and fixture hashes in the implementation evidence. Original-game observations govern rules and timing. JSettlers is implementation evidence, not an authoritative balance specification. Unknown values remain explicitly unverified.

Acceptance criteria

  • Specify ordered atomic checkpoint completion.
  • Preserve the previous valid checkpoint after interrupted writes.
  • Propagate cancellation and storage failures.
  • The visible result agrees with the underlying simulation or platform state, including failure paths.
  • Tests exercise the maintained public interface and retain reproducible fixtures or execution receipts.

Test matrix

  • Race save, cancellation, and disposal.
  • Exercise quota, backgrounding, and failed writes.
  • Exercise invalid input, cancellation or interruption, and repeated execution where this interface permits them.
  • Verify determinism/conservation for simulation changes and resource ownership/disposal for platform changes.
  • Run owning-repository lint, relevant type checks and tests. Capture actual native output for native capability claims.

Scope constraints

Implement this behavior as a complete verifiable slice. Do not silently expand into adjacent tickets or substitute simplified gameplay. Keep the shared TypeScript simulation, shared stx signals, and ts-css conventions. Native gameplay and HUD must not depend on a WebView. HomeOS reuse must not introduce a kernel dependency into the game. Existing unrelated edits and authored guidance must be preserved.

Blocked by

Upstream release and adoption

Fix and test the owning source repository, run its existing bun run release:patch when it is a Bun package repository, verify published artifacts, then install into Stacks and release Stacks when affected. Update the game from verified published packages. HomeOS uses its own build/source provenance.

No node_modules edits, copied runtimes, generated-source string patches, fabricated globals, successful no-op implementations, or skipped release gates. Push tested conventional commits as Chris to the owning origin/main without coauthor trailers or force pushes.

Completion evidence

  • Implementation commits and relevant upstream issue links.
  • Exact checks and results, fixture hashes, and screenshots/native execution evidence where applicable.
  • Published package versions, downstream lockfile identity, and release receipts when affected.
  • Remaining limitations stated explicitly. Partial work does not close this issue.
9 hours ago

Android process-interruption coverage is now implemented in child #440 and pushed to main through eca85e883e875b088be9fba8627ce8e5779b0c8b. It uses public native-screen snapshot promises and unchanged generated storage backends, with an authored debug-only preload at the real atomic rename boundary.

The fresh local API35 Play arm64 run confirmed complete new staging and unchanged previous data, pending acknowledgement with responsive native UI, force-stop plus new-process recovery of the complete previous checkpoint, and acknowledged retry surviving another restart. A valid pass-through negative control recovered the newer acknowledged value. Resource cleanup and owning checks passed. Exact source CI and hosted Android artifacts are still pending; full details and retained hashes are in #440.

This parent remains open. The new slice does not establish every interruption point, power-loss/eviction durability, the full ordering/background/quota/cancellation matrix, native game acceptance or downstream adoption. Already-admitted native writes still have no new cancellation rollback guarantee.

8 days ago

Checkpoint contract progress on codex/issues-388-400-344:

  • Existing Android atomic staging is covered by packages/android/src/snapshot-file.test.ts and preserves the previous checkpoint when staging or replacement fails.
  • 266540de aligns Android with iOS on bounded snapshot names and JSON validation, rejecting invalid writes before posting optimistic state.
  • Android native-shell coverage is 16/16, including failed-write rollback and invalid input with no SYNC_WRITE posted.

The issue remains open for the full cross-platform cancellation/quota/background execution matrix and native receipts; this is not being marked complete from contract tests alone.

8 days ago

Additional cancellation evidence: upstream commit 00751a4e is present on this branch and its maintained browser contract suite passes 13/13. It covers queued Storage.get/set/remove/clear cancellation by caller and host disposal, preserving the previous checkpoint, plus cancellation while account cleanup continues. Together with the native atomic/rollback fixtures and the new Android input validation in 266540de, the stale #394 blocker is now removed from this issue. Remaining release evidence is the native device matrix (quota/background execution) before closure.

8 days ago

Checkpoint verification update:

  • iOS simulator PersistentRecordingCheckpointUnitTests: included in the native unit target and passed in the run (7/7).
  • Android atomic snapshot fixture and native-shell checkpoint tests remain green from the targeted runs.
  • Browser cancellation suite remains 13/13 passed, including caller/host cancellation and prior-checkpoint preservation.

The remaining gap is the full Android device matrix and background/process-death evidence; no Android device is connected on this host.

8 days ago

Additional owning-repo checks: bun run typecheck passed; browser checkpoint cancellation 13/13 (38 expects); Android native-shell checkpoint validation 16/16 (94 expects); Android builder 100/100. Native device/process-death evidence remains pending because this host has no Android SDK/device.

8 days ago

Dependency metadata is now current: the stale blocked by #394 edge has been removed; #394 is closed.

Verification remains:

  • Browser checkpoint cancellation suite 13/13.
  • Android native-shell checkpoint validation 16/16 and atomic replacement fixture passing.
  • iOS persistent checkpoint unit coverage passed 7/7.
  • Typecheck passes; Android device/process-death evidence is unavailable on this host.

Implementation commit: 266540de. The issue remains open until the native device/background/process-death matrix is captured.

8 days ago

Android checkpoint teardown hardening is now on codex/issues-388-400-344 at 86baf40b and pushed.

  • Snapshot SYNC_WRITE ids are tracked per screen.
  • Closing a screen rejects outstanding checkpoint promises with a single SYNC_RESULT { ok: false } before the isolate port closes.
  • A late writer-thread completion cannot publish a second result.
  • Generator assertions cover the tracking/removal/close path; native-STX docs describe the cancellation behavior.

Focused verification: Android generator lifecycle test passed; changed-file Pickier passed. Android device/process-death evidence is still unavailable here, so #396 remains open.

8 days ago

iOS parity follow-up is now pushed at d56c95d8. Native craft.snapshots.set and clear completions check runtimeDisposed on the script thread; if the writer finishes after screen disposal, the pending promise receives false rather than a success callback into a dead runtime.

Focused iOS generator lifecycle test passed and changed-file Pickier passed. This complements Android 86baf40b; #396 remains open for device/process-death and background execution evidence.

8 days ago

Fresh iOS simulator receipt after d56c95d8: PersistentRecordingCheckpointUnitTests passed 7/7 on iPhone 17 Pro (iOS 26.5), including atomic checkpoints, deletion feedback, disk failure recovery, and public storage routing. The suite emitted one expected legacy-migration collision diagnostic but all seven tests passed. Android device/process-death evidence remains unavailable, so the issue stays open.

8 days ago

Fresh Android contract receipt: packages/android/src/snapshot-file.test.ts and native-shell.test.ts passed 17/17 (94 expectations). This covers atomic staging/replacement rollback, invalid checkpoint input, optimistic rollback, cross-isolate updates, and the host prelude contract. No Android device or emulator is installed (adb is unavailable), so process-death/background evidence remains the only native verification gap.

8 days ago

Pushed 85f21899 on codex/issues-388-400-344 and verified the iOS generator suite (129 passed).

This closes a lifecycle edge in craft.snapshots: a late writer completion now rejects when the screen is disposed or already deallocated. The previous weak-self check treated a deallocated controller as success, which could resolve into a dead JavaScript runtime. The contract test and native guide now cover the stronger failure semantics.

8 days ago

Latest verification on codex/issues-388-400-344: Android builder/native-shell/checkpoint suites passed 117/117; the Android Gradle atomic replacement fixture passed; iOS persistent-checkpoint simulator receipt passed 7/7 on iPhone 17 Pro / iOS 26.5. Typecheck also passed. The only expected simulator log is the legacy migration collision diagnostic exercised by the failure-recovery test.

8 days ago

Owning-source checkpoint hardening is now on main through 6fdd8eb3. Both the complete source CI and Full Mobile matrix passed on that exact source.

Original job artifacts were downloaded and checked against their published artifact digests. The actual result bundles were decoded independently:

  • Android API 34 and 35: 74 tests passed on each, including queued storage cancellation and repeated explicit Back/native KEYCODE_BACK teardown.
  • iOS UIKit: 252 tests passed, including all nine persistent checkpoint cases and retained closed-controller storage rejection.
  • iOS bridge: both 13-case legs passed. Metal and Windows surface fixtures also passed, but they do not establish native game checkpoint acceptance.

The iOS storage lifetime guard rejects calls retained after screen close while permitting the genuine synchronous unmount callback. Queued operations do not report success after cancellation or disposal. A parallel macOS preferences test collision was fixed by giving each test its own owned domain, without changing production storage behavior.

The existing bun run release:patch completed successfully and published source/tag v0.0.341 at 0dba496e37f94fcc5e76c55503c4786bab30a590. Its publication workflow is still being qualified; the game has not adopted it. Full game background/process-death acceptance and release artifact qualification remain separate gates, so this issue stays open.

8 days ago

Exact v0.0.341 release follow-up at 0dba496: source CI completed successfully, but Full Mobile E2E failed and the release gate correctly refused publication.

The original failed iOS Metal artifact was downloaded and verified against GitHub digest 003d685e622e57b13b5188a469cda77d7480e69e78a7518499756087c5d6d3cb, with ZIP CRC passing. Its single verification receipt shows the first xcrun simctl list runtimes --json command reached its existing 120000ms limit, received SIGTERM/143 and produced no stdout/stderr. This happened before any native build, pixel or frame acceptance. Workspace and process groups were disposed. The cause of that simulator-service stall remains unverified.

Full Mobile: https://github.com/craft-native/craft/actions/runs/40157747311 . Release gate: https://github.com/craft-native/craft/actions/runs/40157747428 . Source CI: https://github.com/craft-native/craft/actions/runs/40157747263 . Prior 6fdd native acceptance remains separate evidence. Version341 is not qualified or adopted into the game. Web gameplay remains the current priority.

8 days ago

Exact v0.0.341 follow-up: the unchanged-source retry of Full Mobile E2E and the maintained release rerun are now fully green at 0dba496. The original attempt1 simulator-query timeout remains retained separately; it was not relabeled as success or bypassed.

Published artifacts were independently qualified: the original npm artifact12585048450 matches GitHub SHA256fce7d1d123f330f1c99485e796335373ea4310b7ef258df32e2c565b6919c4b0 and ZIP CRC; all8 original package archives match complete public registry bytes and metadata. All4 native ZIPs match the original manifest and GitHub digests, pass CRC and executable architecture checks, including both macOS executables and the Windows loader. The original macOS CLI executed --version successfully and reported0.0.341. A normal Stacks installation resolved341, and all216 installed craft-native files match the original archive.

Full Mobile: https://github.com/craft-native/craft/actions/runs/40157747311 . Releaser: https://github.com/craft-native/craft/actions/runs/40157747428 . Stacks adoption and its next maintained patch release are in progress. This qualifies the source/package/runtime cohort; actual native game save/lifecycle and HUD acceptance remain open, so this issue stays open.

4 days ago

Pushed two tested commits directly to main on top of the latest upstream release commit a7b7a46:

  • 35714ce: retain browser checkpoint locks through storage publication (3 files).
  • d929f2a: real-browser checkpoint recovery and competing-tab acceptance (5 files).

The new real Firefox test exposed lost updates: two tabs acknowledged 40 successful CAS increments, but reload recovered only 33. The initial Firefox run was 6 passed / 2 failed; the second failure was a quota exception also escaping as a page error despite caller handling. Chromium initially passed all 8 tests.

The native Web Lock callback is now async and keeps ownership through the next task boundary, allowing task-local localStorage snapshots to publish before another tab acquires the lock. Exceptions propagate as promise rejections. No browser preference, storage mock, timeout relaxation or fallback was used to make acceptance pass. All competing writers must obey the same lock; raw localStorage writes outside Craft are not protected.

Acceptance bundles the maintained createBrowserPlatform source, serves it on loopback and launches isolated persistent browser profiles. Eight scenarios per engine cover reload plus graceful browser restart, competing setIfAbsent/CAS winners, repeated CAS increments, input capture, queued reload/abort/disposal, and real QuotaExceededError recovery. Profiles are removed, while traces and JSON receipts with actual browser versions and bundle hashes are retained. CI explicitly runs Chromium and Firefox with the locked Playwright dependency, uploads receipts, and gates publish-commit on both jobs.

Validation:

  • Final Chromium: 8 passed, 0 failed.
  • Final Firefox: 8 passed, 0 failed.
  • Firefox contention soak: 20 passed across 10 repetitions, no lost increments.
  • SDK: 814 passed, 10 skipped, 0 failed.
  • Release/workflow suite: 413 passed. Typechecks (including the new harness), package builds, 43 package checks, frozen lock install and pinned Zig formatting passed. Lint: 0 errors, 26 existing warnings.
  • After the concurrent upstream version-only v0.0.387 commit, regenerated the lock and reran both full browser suites, typechecks, builds/package checks, release tests, formatting and lint successfully.

Receipts: /tmp/craft-396-firefox.log (original failure), /tmp/craft-396-firefox-soak.log, /tmp/craft-396-387-chromium.log, /tmp/craft-396-387-firefox.log. Actual traces/results are at the temporary directories printed in those logs. Fixture SHA256: 87f756f1d8b1f9bb799058b3155d487ff19ea523d323ddbc8ac058777340d90e (scripts/browser-checkpoints.test.ts).

Remote CI: https://github.com/craft-native/craft/actions/runs/40868743838 (pending at handoff). No release or tag created. Keeping #396 open: this does not prove browser process-crash/power-loss durability, storage eviction, WebKit, Android, or downstream game adoption.

5 days ago

Pushed a concrete iOS checkpoint-lifetime fix to main in 47c416c (four files), on top of the latest upstream changes.

A retained closed screen could still call craft.snapshots.get/set/clear. The old completion guard rejected the promise only after the writer had overwritten or deleted the shared checkpoint. The new native regression reproduced the disk/cache changes in both main-thread and background-thread modes before the fix.

Snapshot calls now use the existing synchronized screen-lifetime admission guard: post-close reads, writes, deletions and clears fail with SCREEN_CLOSED before reaching the cache/writer. The genuine synchronous unmount callback retains its cleanup allowance. Already-admitted writes can still finish on disk after disposal; a rejected completion is not a rollback guarantee, now documented explicitly.

Validation:

  • Full generated iOS 26.5 / iPhone 17 Pro simulator unit suite: 286 passed, 0 failures, 0 skips, including all 11 persistent-checkpoint tests and retained-host ownership tests.
  • New regressions cover unchanged disk/cache after post-close calls and final synchronous unmount cleanup, each in both script-thread modes.
  • iOS generator: 142 passed. SDK: 800 passed / 8 skipped / 0 failed.
  • Typecheck, npm builds, 43 package checks, pinned Zig formatting, 399 release-tooling tests, and lint passed (0 errors / 26 existing warnings).

Local receipts are retained at /tmp/craft-396-red.xcresult and /tmp/craft-396-green.xcresult. Fixture SHA256 (HeartRateUnitTests.swift): 6108dbf35de797feb0ce45c4fc7c642c2b8531621ae8c52abec25c5662642faf.

New remote CI is pending. No release/tag was created. Keeping #396 open: this is screen-disposal protection, not new game background/process-death, Android, or downstream adoption evidence.

5 days ago

Two more focused commits are pushed to main (three files each):

  • 979b8cd: serialize snapshot admission with disk-queue submission. Previously revisions/pending cache were assigned under a lock, but work was enqueued after unlocking. Concurrent callers could reorder disk writes relative to cached revisions. The native stress regression reproduced this at round 46: all writes succeeded, but cache retained writer 30 while disk held writer 10. Queue submission now occurs within the same admission lock; disk I/O remains asynchronous.
  • de6a439: public snapshot recovery UI fixture. It queues 32 distinct 64-KiB checkpoints, verifies completion order, saves on an actual app-background lifecycle event, terminates/relaunches the process, validates the complete recovered payload, and verifies acknowledged deletion survives another relaunch. A changing process-local session identifier confirms this is not merely a new screen over the old cache. The fixture renders UIKit, with no WebView, and retains a recovery screenshot.

Validation on iOS 26.5 / iPhone 17 Pro simulator:

  • Full native unit suite: 287 passed, 0 failures/skips, including 3,200 concurrent writes and existing failed-write preservation/retry and disposal cases.
  • Full native renderer UI suite: 9 passed, 0 failures/skips.
  • iOS generator: 142 passed. SDK: 800 passed / 8 skipped on unchanged full retry. The first SDK run hit an existing 10-second emulator-fixture subprocess timeout during fresh simulator boot; unchanged focused retry passed 26/26 before the full retry.
  • 399 release-tooling tests; typecheck; npm builds and 43 package checks; pinned Zig formatting; lint (0 errors, 26 existing warnings).

Retained local receipts: /tmp/craft-396-order-red.xcresult, /tmp/craft-396-order-green.xcresult, /tmp/craft-396-recovery-ui.xcresult. Fixture SHA256:

  • HeartRateUnitTests.swift: 443f4489a266ef3d055c1b949783292e9c41fddfeabf6c51f46db6335432266d
  • NativeRenderUITests.swift: a30a7dcd4fc3d1076a67c2565fc258f58078d70929e33c6c3125e71b80e91fef
  • render-test.js: 3564ceaafff6b0f5073dd5e22798065dd1e2dbc77d35842562bf4d5c82a7df81

Remote CI is pending. No release/tag created. Keeping #396 open: acknowledged-save recovery and actual background handling do not prove termination at a selected disk-write instruction, OS memory-pressure eviction, power-loss durability, Android acceptance, or downstream game adoption.

5 days ago

Pushed two focused commits to main: 13b9f61 (3 files) and 83ba2fe (4 files).

Added deterministic iOS process-interruption evidence for the existing snapshot backend:

  • The generated simulator app alone interposes Foundation's real atomic replacement. It pauses the snapshot worker immediately before rename, verifies the entire staged 256-KiB payload for generation 2 and unchanged generation 1 on disk, and keeps its UI responsive.
  • XCTest requires that confirmed boundary, terminates the app while the write is in flight, then checks that a new process recovers the complete generation 1. A subsequent generation 3 save must survive another process relaunch.
  • This uses the existing CraftSnapshots backend; the snapshot implementation in the generated app was compared byte-for-byte with source. No production runtime change or shipped fault hook was necessary. Earlier public-JS burst/background recovery coverage remains in the same suite.
  • Negative control: disabling only .atomic in a disposable generated app caused the test to fail: the replacement was not intercepted and relaunch recovered generation 2 rather than the required previous value. That failed result is retained, not counted as a pass.

Validation:

  • Final iOS 26.5 / iPhone 17 Pro simulator run: 304 passed, 0 failed/skipped (294 native unit tests + 10 UI tests).
  • SDK: 800 passed, 8 skipped, 0 failed. iOS generator: 142 passed. Runner/fixture command tests: 7 passed.
  • Release-tooling/contract suite: 400 passed. Typecheck, package builds, 43 package checks, pinned Zig formatting, and lint passed (0 errors / 26 existing warnings).
  • Full checks also exposed two prior test problems: the SVG source assertion still expected the pre-timestamp frame callback signature; the audio weak-lifetime fixture drained notification temporaries but not registration temporaries. The assertion is aligned, and registration/notification/close now share an autorelease scope, checking deallocation across 16 cycles without sleeps or an audio runtime change. The initial full run's 303-pass/1-failure result is retained separately.

Local execution receipts:

  • /tmp/craft-396-interruption-verified.xcresult — final green run with retained boundary/recovery screenshots.
  • /tmp/craft-396-nonatomic-control.xcresult — deliberate non-atomic negative control.
  • /tmp/craft-396-interruption-full.xcresult — initial audio lifetime assertion failure.

Fixture SHA256:

  • CheckpointInterruption.c: ecbf8d42e12292e62b0330de78b026742acbe3e625b6dc5b1a76a2804ca1294e
  • CheckpointInterruption.swift: c0198eb4a4845a89ce0bc3017c0e86ef376d17a34a45585b2eeda6ba54600234
  • NativeRenderUITests.swift: 935bb219f5701fc38bd6a733b34443a8f77d65945c8e8e6d7bec3725f3d1fd74
  • NativeAudioUnitTests.swift: 644ee99ed8b5434de977e797cf7471e4279511b2ddb70f284b5db38370c0f438

Remote CI is pending. No release/tag created. Keeping #396 open: this establishes controlled termination at the iOS pre-replacement boundary, not every interruption point, OS memory-pressure eviction, device power-loss durability, Android process-interruption acceptance, or downstream game adoption.

5 days ago

Pushed ec2f74a to main: browser checkpoint writes now capture their key and JSON at invocation, before account cleanup or Web Lock waits. Later simulation mutations cannot change or retarget a queued save. CAS captures both expected and replacement JSON (including toJSON). Cancellation and account identity are still checked before commit.

Evidence: four new regressions failed before the fix. After the fix, 35 focused browser storage/session tests pass, including public platform calls, caller cancellation and host disposal, malformed JSON, account changes during serialization, quota errors, and retries. Full SDK: 812 passed, 8 skipped, 0 failed. Typecheck, package builds, package verification (43 checks), pinned Zig formatting, release rehearsals (411 tests), and lint (0 errors, 26 existing warnings) pass.

This is a four-file implementation/test/documentation commit. These are browser-host fixtures, not evidence of real browser crash recovery or Android acceptance. Keeping this broad issue open; not yet published.

Sign in to comment on this issue.