also looking at this
Add deterministic test sharding for distributed CI
#22
10 files
+322
-5
| @@ -95,6 +95,8 @@ This will automatically discover and run all `*.test.zig` files in the `tests` d | ||
| 95 | 95 | - `--no-recursive` - Disable recursive directory search |
| 96 | 96 | - `--bail` - Stop on first failure |
| 97 | 97 | - `--filter <text>` / `--grep <text>` - Run only Zig tests whose names contain the text |
| 98 | - `--shard-index <N>` - Run one one-based file shard | |
| 99 | - `--shard-count <N>` - Set the total number of file shards | |
| 98 | 100 | - `--no-color` - Disable color in child Zig test processes |
| 99 | 101 | - `--verbose` - Show detailed output |
| 100 | 102 | |
| @@ -117,6 +119,34 @@ zig-test --test-dir tests --no-recursive | ||
| 117 | 119 | |
| 118 | 120 | # Stop on first failure |
| 119 | 121 | zig-test --test-dir tests --bail |
| 122 | ||
| 123 | # Run the second of four deterministic file shards | |
| 124 | zig-test --test-dir tests --shard-index 2 --shard-count 4 | |
| 125 | ``` | |
| 126 | ||
| 127 | #### Distributed CI sharding | |
| 128 | ||
| 129 | Sharding is opt-in and file-based. Both options are required, shard indices are | |
| 130 | one-based, and `1 <= shard-index <= shard-count`. Test files are assigned from | |
| 131 | their normalized relative paths with a fixed FNV-1a 64 hash, so the same paths | |
| 132 | and shard count always produce the same assignment on every platform. Each file | |
| 133 | belongs to exactly one shard; a valid shard with no matching files succeeds. | |
| 134 | ||
| 135 | Discovery and file-pattern selection happen before sharding. `--filter` is then | |
| 136 | forwarded to `zig test` for every file in the selected shard. Coverage and bail | |
| 137 | behavior apply only to that shard. The summary prints the shard, selected file | |
| 138 | count, and total discovered file count. | |
| 139 | ||
| 140 | ```yaml | |
| 141 | jobs: | |
| 142 | test: | |
| 143 | strategy: | |
| 144 | matrix: | |
| 145 | shard: [1, 2, 3, 4] | |
| 146 | steps: | |
| 147 | - uses: actions/checkout@v4 | |
| 148 | - run: zig build | |
| 149 | - run: zig build run -- --test-dir tests --shard-index ${{ matrix.shard }} --shard-count 4 | |
| 120 | 150 | ``` |
| 121 | 151 | |
| 122 | 152 | ### Configuration files |
| @@ -135,6 +165,10 @@ precedence over configured values. | ||
| 135 | 165 | "recursive": true, |
| 136 | 166 | "filter": "database" |
| 137 | 167 | }, |
| 168 | "sharding": { | |
| 169 | "index": 1, | |
| 170 | "count": 4 | |
| 171 | }, | |
| 138 | 172 | "coverage": { |
| 139 | 173 | "enabled": false, |
| 140 | 174 | "output_dir": "coverage" |
| @@ -142,8 +176,8 @@ precedence over configured values. | ||
| 142 | 176 | } |
| 143 | 177 | ``` |
| 144 | 178 | |
| 145 | The accepted top-level sections are `test`, `parallel`, `reporter`, `snapshot`, | |
| 146 | `watch`, `memory`, `ui`, and `coverage`. Options retain the same execution-mode | |
| 179 | The accepted top-level sections are `test`, `sharding`, `parallel`, `reporter`, | |
| 180 | `snapshot`, `watch`, `memory`, `ui`, and `coverage`. Options retain the same execution-mode | |
| 147 | 181 | limitations as their CLI equivalents; unsupported combinations fail explicitly. |
| 148 | 182 | |
| 149 | 183 | ### Parallel execution semantics |
| @@ -607,6 +641,8 @@ zig-test --no-color | ||
| 607 | 641 | | `--test-dir <dir>` | | Directory to search for tests | |
| 608 | 642 | | `--pattern <pattern>` | | Test file pattern (default: *.test.zig) | |
| 609 | 643 | | `--no-recursive` | | Disable recursive directory search | |
| 644 | | `--shard-index <N>` | | Run one one-based file shard | | |
| 645 | | `--shard-count <N>` | | Set the total number of file shards | | |
| 610 | 646 | | `--coverage` | | Enable code coverage collection | |
| 611 | 647 | | `--coverage-dir <dir>` | | Coverage output directory (default: coverage) | |
| 612 | 648 | | `--coverage-tool <tool>` | | Coverage tool to use (kcov or grindcov) | |