ReviewOS

stacks/the-settlers-iii-remake

[S3-092] Make state checksums incremental for long replays and large maps

#106
Closed chris opened this 9 days ago ·
9 days ago

Problem

checksum() hashes every map layer and a canonical serialization of all entities. That is fine for saves and occasional checks, but replays record a checksum after every tick, and multiplayer will compare checksums continuously. On a 96 by 96 map a 420-tick replay test takes several seconds; original maps reach 640 by 640, where per-tick full checksums are too slow for live play or long replays.

Acceptance criteria

  • Maintain checksums incrementally (or per region and entity class) so per-tick cost scales with what changed, not with map size.
  • Prove equivalence: the incremental checksum equals a full recomputation at every tick across long mixed scenarios, saves and restores.
  • Keep the first-divergent-tick report and identical results across hosts.
  • Record measured per-tick cost on the largest supported map.

Context

Relevant to S3-092 (replay diagnostics), S3-094 (authoritative multiplayer) and S3-104 (performance budgets). The current implementation is in resources/game/engine/save.ts.

9 days ago

Completed in 275dc1a.

  • Incremental maintenance: the checksum combines a hash per 32×32 chunk of the map and fog layers with the canonical entity text (resources/game/engine/digest.ts). Every place the simulation writes a map or fog tile marks its chunk:

    • harvest, planting, growth, mining, levelling
    • territory claims, pioneers and capture
    • scenario setup and mission reveals
    • fog, which is rebuilt into a scratch layer so only tiles whose sight changed are marked

    session.checksum() rehashes only marked chunks. checksum(state) recomputes everything.

  • Equivalence: tests/unit/engine-checksum.test.ts requires both to agree on every tick:

    • a 6,000-tick economy with pioneers, a mission reveal and six restores
    • combat and a capture that redraws borders

    A write that bypasses change tracking is shown to be caught by the comparison.

  • Replays and hosts: replays record and verify with the incremental checksum and still report the first divergent tick (existing replay tests unchanged). Hashing uses only integer operations (Math.imul, shifts) over little-endian values, as before. A live browser/native/Bun comparison remains with S3-092 (#94), which needs the native host.

  • Saves: format 3 changes only the checksum. Format 2 saves are verified under their own checksum and then written as format 3, tested with a fixture written by the previous engine (73e7abe).

  • Measured cost on the largest supported map (1024×1024, MAX_MAP_SIZE): full checksum ≈ 88-100 ms; incremental ≈ 0.42-0.49 ms per tick. Apple M3 Pro, Bun 1.4.2, under heavy unrelated machine load; printed by the test, which also asserts the incremental cost is under a fifth of the full one.

Sign in to comment on this issue.