also looking at this
Add reproducible randomized test order
#29
18 files
+487
-42
| @@ -34,6 +34,9 @@ reporters, including the configured JUnit output path. | ||
| 34 | 34 | child process. |
| 35 | 35 | - Every logical result owns ordered attempt history. A failure followed by a |
| 36 | 36 | successful retry is classified as `flaky`, distinct from an ordinary pass. |
| 37 | - Optional randomized execution uses one run seed across the selected plan. | |
| 38 | The seed is carried by reporters and `TestResults`; see | |
| 39 | [Randomized test order](randomized-order.md) for mode-specific guarantees. | |
| 37 | 40 | - Summaries always come from `TestResults`, rather than executor-local counters. |
| 38 | 41 | |
| 39 | 42 | ## Extension points |
| @@ -53,10 +56,11 @@ model without coupling the execution API to JSON. | ||
| 53 | 56 | Existing programmatic APIs remain source compatible: |
| 54 | 57 | |
| 55 | 58 | - `TestRunner.init`, `runTests`, and `runTestsWithOptions` are unchanged. |
| 56 | - `RunnerOptions` gains optional reporter, timeout, retry, repeat, and flaky-exit | |
| 59 | - `RunnerOptions` gains optional reporter, timeout, retry, repeat, flaky-exit, | |
| 60 | shuffle, and seed | |
| 57 | 61 | fields with backward-compatible defaults. |
| 58 | 62 | - `runDiscoveredTests` and `LoaderOptions` remain available; optional reporter, |
| 59 | JUnit path, timeout, retry, repeat, flaky-exit, and writer fields default to | |
| 63 | JUnit path, timeout, retry, repeat, flaky-exit, shuffle, seed, and writer fields default to | |
| 60 | 64 | the previous spec behavior. |
| 61 | 65 | |
| 62 | 66 | CLI hosts can inject a stdout writer with `reporter_writer`. Embedded callers |