ReviewOS

also looking at this

zig-utils/zig-test-framework

Define stable JSON event protocol

#27
Merged glennmichael123 wants to merge codex/issue-10-json-events into main
11 files +442 -177
docs/json-event-protocol.mdadded+71-0
Changes to docs/json-event-protocol.md
@@ -0,0 +1,71 @@
1# JSON event protocol
2
3The framework exposes one versioned event model for the JSON reporter, live UI,
4and saved test history. Protocol version 1 uses this envelope for every event:
5
6```json
7{
8 "protocol_version": 1,
9 "type": "test_end",
10 "data": {
11 "name": "adds two numbers",
12 "suite": "math",
13 "status": "passed",
14 "duration_ns": 42000,
15 "error_message": null
16 }
17}
18```
19
20All strings are encoded by Zig's standard JSON serializer. Consumers should
21decode JSON rather than depending on whitespace or object-field order.
22
23## Transports
24
25- The JSON reporter writes an ordered JSON array of event envelopes.
26- The live UI sends each envelope as a server-sent event. The SSE event name is
27 the same as the envelope's `type`.
28- Test-history files contain an ordered JSON array using the same envelopes.
29
30Events preserve execution order. A run begins with `run_start` and finishes
31with `run_end`; suite and test start/end events are nested in between. Retry
32events are emitted in attempt order immediately before their `test_end`.
33
34## Version 1 events
35
36| Type | Data fields |
37| --- | --- |
38| `run_start` | `total` |
39| `run_end` | `total`, `passed`, `flaky`, `failed`, `skipped`, `duration_ns` |
40| `suite_start`, `suite_end` | `name` |
41| `test_start` | `name`, optional `suite` |
42| `test_end` | `name`, optional `suite`, `status`, `duration_ns`, optional `error_message` |
43| `hook` | `kind`, optional `suite`, optional `test_name`, `status`, `duration_ns`, optional `error_message` |
44| `output` | `stream`, `text`, optional `test_name` |
45| `retry` | `name`, `attempt`, `repetition`, `status`, `duration_ns`, optional `error_message` |
46| `coverage` | optional `line_percent`, `function_percent`, `branch_percent`, `report_path` |
47
48Statuses are `pending`, `running`, `passed`, `flaky`, `failed`, or `skipped`.
49Hook kinds are `before_all`, `before_each`, `after_each`, or `after_all`.
50Output streams are `stdout` or `stderr`.
51
52Optional fields are serialized as `null` when no value is available. Durations
53are unsigned integer nanoseconds. Percentages are numbers from 0 through 100.
54
55## Compatibility policy
56
57`protocol_version` is the major wire-format version. Version 1 may gain new
58event types, enum values, and optional data fields without changing the version.
59Consumers must ignore event types and fields they do not understand. Existing
60fields will not change meaning or type within a major version.
61
62Removing or renaming an event or field, changing a field's type or semantics,
63or changing the envelope shape requires a new major version. Producers emit one
64major version per stream. Consumers should reject an unsupported major version
65with a clear error instead of interpreting it as an older version.
66
67## Zig API
68
69`event_protocol.Event` is the tagged union for all event types, and
70`event_protocol.encodeAlloc` returns a standards-compliant encoded envelope.
71The root module also exports these as `ProtocolEvent` and `protocol_version`.