Port to latest zig-master

docs/ARCHITECTURE.mdmodified+68-24

Changes to docs/ARCHITECTURE.md

		@@ -2,7 +2,7 @@
2	2
3	3	## Overview
4	4
5		-The zig-regex library implements a regular expression engine using **Thompson's NFA construction** algorithm combined with **thread-based simulation** for pattern matching. This architecture provides linear time complexity O(n*m) for matching, where n is the input length and m is the pattern size.
	5	+The zig-regex library implements a regular expression engine using **Thompson's NFA construction** algorithm combined with **thread-based simulation** for pattern matching. This architecture provides linear time complexity O(n\*m) for matching, where n is the input length and m is the pattern size.
6	6
7	7	## Design Philosophy
8	8
		@@ -55,9 +55,11 @@ The zig-regex library implements a regular expression engine using **Thompson's
55	55	## Module Breakdown
56	56
57	57	### 1. common.zig (174 lines)
	58	+
58	59	**Purpose:** Shared types and utilities
59	60
60	61	**Key Components:**
	62	+
61	63	- `CharRange` - Represents a character range (e.g., 'a'-'z')
62	64	- `CharClass` - Set of character ranges with negation support
63	65	- `CharClasses` - Predefined classes (\d, \w, \s, etc.)
		@@ -65,13 +67,16 @@ The zig-regex library implements a regular expression engine using **Thompson's
65	67	- `Span` - Source position for error reporting
66	68
67	69	**Design Notes:**
	70	+
68	71	- Character classes use ranges for memory efficiency
69	72	- Negation handled at match time, not during construction
70	73
71	74	### 2. ast.zig (267 lines)
	75	+
72	76	**Purpose:** Abstract Syntax Tree representation
73	77
74	78	**Node Types:**
	79	+
75	80	```zig
76	81	pub const NodeType = enum {
77	82	literal, // Single character 'a'
		@@ -90,20 +95,24 @@ pub const NodeType = enum {
90	95	```
91	96
92	97	**Key Features:**
	98	+
93	99	- Tagged union for type-safe node data
94	100	- Recursive structure for nested expressions
95	101	- Memory management via allocator
96	102	- Span tracking for error messages
97	103
98	104	**Design Notes:**
	105	+
99	106	- Each node owns its children (tree ownership)
100	107	- `destroy()` recursively frees the entire tree
101	108	- Factory methods ensure proper initialization
102	109
103	110	### 3. parser.zig (395 lines)
	111	+
104	112	**Purpose:** Lexical analysis and parsing
105	113
106	114	**Architecture:**
	115	+
107	116	```
108	117	Lexer (Tokenizer)
109	118	│
		@@ -122,21 +131,25 @@ Parser (Recursive Descent)
122	131	```
123	132
124	133	**Operator Precedence (high to low):**
	134	+
125	135	1. Literals, groups, character classes
126		-2. Quantifiers (*, +, ?, {m,n})
	136	+2. Quantifiers (\*, +, ?, {m,n})
127	137	3. Concatenation (implicit)
128	138	4. Alternation (|)
129	139
130	140	**Design Notes:**
	141	+
131	142	- Single-pass parsing
132	143	- Error recovery not implemented (fails fast)
133	144	- Lookahead limited to one token
134	145	- Character classes parsed separately
135	146
136	147	### 4. compiler.zig (455 lines)
	148	+
137	149	**Purpose:** NFA construction via Thompson's algorithm
138	150
139	151	**NFA Components:**
	152	+
140	153	```zig
141	154	pub const State = struct {
142	155	id: StateId,
		@@ -155,23 +168,26 @@ pub const Transition = struct {
155	168
156	169	**Thompson Construction Rules:**
157	170
158		-| Pattern | NFA Fragment |
159		-|---------|--------------|
160		-| Literal 'a' | `s0 --a--> s1` |
161		-| Concatenation 'ab' | `s0 --a--> s1 --ε--> s2 --b--> s3` |
	171	+| Pattern | NFA Fragment |
	172	+| ------------------ | ---------------------------------------------------------------------------- |
	173	+| Literal 'a' | `s0 --a--> s1` |
	174	+| Concatenation 'ab' | `s0 --a--> s1 --ε--> s2 --b--> s3` |
162	175	| Alternation 'a\|b' | `s0 --ε--> (s1 --a--> s2)<br>&nbsp;&nbsp;&nbsp;&nbsp;└--ε--> (s3 --b--> s4)` |
163		-| Star 'a*' | `s0 --ε--> s1 --a--> s2 --ε--> s1<br>&nbsp;&nbsp;&nbsp;&nbsp;└--ε--> accept` |
	176	+| Star 'a\*' | `s0 --ε--> s1 --a--> s2 --ε--> s1<br>&nbsp;&nbsp;&nbsp;&nbsp;└--ε--> accept` |
164	177
165	178	**Design Notes:**
	179	+
166	180	- Each AST node compiles to a Fragment (start, accept states)
167	181	- Epsilon transitions used for control flow
168	182	- Greedy matching achieved through simulation order
169	183	- Capture groups marked on states, not transitions
170	184
171	185	### 5. vm.zig (340 lines)
	186	+
172	187	**Purpose:** NFA simulation engine
173	188
174	189	**Thread-Based Simulation:**
	190	+
175	191	```
176	192	Thread = {
177	193	state: StateId,
		@@ -190,29 +206,35 @@ Algorithm:
190	206	```
191	207
192	208	**Key Features:**
	209	+
193	210	- **Greedy Matching:** Continues matching to find longest match
194	211	- **Epsilon Closure:** Precomputed before consuming input
195	212	- **Capture Tracking:** Each thread maintains capture positions
196	213	- **No Backtracking:** All paths explored simultaneously
197	214
198	215	**Time Complexity:**
199		-- O(n * m) where n = input length, m = number of states
200		-- Worst case: O(n * 2^p) where p = number of alternations
	216	+
	217	+- O(n \* m) where n = input length, m = number of states
	218	+- Worst case: O(n \* 2^p) where p = number of alternations
201	219
202	220	**Space Complexity:**
203		-- O(m * c) where m = states, c = capture groups
	221	+
	222	+- O(m \* c) where m = states, c = capture groups
204	223	- Thread list grows with state complexity
205	224
206	225	**Design Notes:**
	226	+
207	227	- Threads represent possible execution paths
208	228	- Visited states tracked to avoid infinite loops
209	229	- Anchors checked during epsilon closure
210	230	- Match preference: longer match > earlier match
211	231
212	232	### 6. regex.zig (385 lines)
	233	+
213	234	**Purpose:** Public API and convenience functions
214	235
215	236	**API Surface:**
	237	+
216	238	```zig
217	239	// Compilation
218	240	Regex.compile(allocator, pattern) -> Regex
		@@ -230,15 +252,18 @@ regex.split(allocator, input) -> [][]const u8
230	252	```
231	253
232	254	**Design Notes:**
	255	+
233	256	- Immutable compiled patterns (can reuse)
234	257	- All allocations explicit and user-controlled
235	258	- Match results contain slices (no copies)
236	259	- Iterator pattern not yet implemented
237	260
238	261	### 7. errors.zig (67 lines)
	262	+
239	263	**Purpose:** Error types and reporting
240	264
241	265	**Error Types:**
	266	+
242	267	```zig
243	268	pub const RegexError = error{
244	269	// Parse errors

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -260,6 +285,7 @@ pub const RegexError = error{ formatting only
260	285	```
261	286
262	287	**Error Context:**
	288	+
263	289	- Position tracking in pattern
264	290	- Human-readable messages
265	291	- Pointer to error location

Changes to docs/ARCHITECTURE.md

		@@ -269,12 +295,14 @@ pub const RegexError = error{
269	295	### Thompson's NFA Construction
270	296
271	297	**Why Thompson?**
	298	+
272	299	1. Simple to implement
273	300	2. Guaranteed linear time matching
274	301	3. No catastrophic backtracking
275	302	4. Naturally supports alternation
276	303
277	304	**How it works:**
	305	+
278	306	1. Each regex operator has a standard NFA fragment pattern
279	307	2. Fragments compose recursively
280	308	3. Epsilon transitions connect fragments
		@@ -283,16 +311,19 @@ pub const RegexError = error{
283	311	### Thread-Based Simulation
284	312
285	313	**Why Threads?**
	314	+
286	315	- Simulates NFA without building DFA (space efficient)
287	316	- Handles large state spaces
288	317	- Natural support for captures
289	318
290	319	**Greedy Matching:**
	320	+
291	321	- Continue simulation even after finding a match
292	322	- Keep longest match found
293	323	- Ensures `a*` matches "aaa" not ""
294	324
295	325	**Epsilon Closure:**
	326	+
296	327	- Precompute all states reachable via ε-transitions
297	328	- Check anchors during closure
298	329	- Avoids redundant state exploration

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -300,6 +331,7 @@ pub const RegexError = error{ formatting only
300	331	## Memory Management
301	332
302	333	**Ownership Model:**
	334	+
303	335	```
304	336	Regex
305	337	└─ owns NFA

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -315,6 +347,7 @@ Matches formatting only
315	347	```
316	348
317	349	**Allocation Points:**
	350	+
318	351	1. Pattern string duplication
319	352	2. AST node creation (freed after compilation)
320	353	3. NFA states and transitions

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -322,6 +355,7 @@ Matches formatting only
322	355	5. Match results and captures
323	356
324	357	**Guidelines:**
	358	+
325	359	- User provides allocator for all operations
326	360	- No hidden global allocations
327	361	- Clear ownership semantics

Changes to docs/ARCHITECTURE.md

		@@ -331,27 +365,28 @@ Matches
331	365
332	366	### Time Complexity
333	367
334		-| Operation | Complexity | Notes |
335		-|-----------|------------|-------|
336		-| Compile | O(p) | p = pattern length |
337		-| isMatch | O(n * m) | n = input, m = states |
338		-| find | O(n * m * k) | k = positions to try |
339		-| findAll | O(n * m) | Amortized |
340		-| replace | O(n + r) | r = replacement length |
341		-| split | O(n * m) | Plus allocation |
	368	+| Operation | Complexity | Notes |
	369	+| --------- | ------------ | ---------------------- |
	370	+| Compile | O(p) | p = pattern length |
	371	+| isMatch | O(n \* m) | n = input, m = states |
	372	+| find | O(n _ m _ k) | k = positions to try |
	373	+| findAll | O(n \* m) | Amortized |
	374	+| replace | O(n + r) | r = replacement length |
	375	+| split | O(n \* m) | Plus allocation |
342	376
343	377	### Space Complexity
344	378
345		-| Component | Complexity | Notes |
346		-|-----------|------------|-------|
347		-| AST | O(p) | Temporary |
348		-| NFA | O(p) | Persistent |
349		-| VM Threads | O(m * c) | c = captures |
350		-| Matches | O(r) | r = result count |
	379	+| Component | Complexity | Notes |
	380	+| ---------- | ---------- | ---------------- |
	381	+| AST | O(p) | Temporary |
	382	+| NFA | O(p) | Persistent |
	383	+| VM Threads | O(m \* c) | c = captures |
	384	+| Matches | O(r) | r = result count |
351	385
352	386	### Optimization Opportunities
353	387
354	388	**Not Yet Implemented:**
	389	+
355	390	1. DFA construction for static patterns
356	391	2. State minimization
357	392	3. Literal prefix extraction

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -362,6 +397,7 @@ Matches formatting only
362	397	## Testing Strategy
363	398
364	399	**Test Categories:**
	400	+
365	401	1. **Unit Tests** - Each module tested independently
366	402	2. **Integration Tests** - End-to-end pattern matching
367	403	3. **Edge Cases** - Empty strings, boundary conditions

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -369,6 +405,7 @@ Matches formatting only
369	405	5. **Property Tests** - Invariants (future)
370	406
371	407	**Current Coverage:**
	408	+
372	409	- 35+ tests across all modules
373	410	- All basic features tested
374	411	- Edge cases covered

Changes to docs/ARCHITECTURE.md

		@@ -377,18 +414,21 @@ Matches
377	414	## Future Enhancements
378	415
379	416	### Phase 6: Quality & Testing
	417	+
380	418	- Comprehensive fuzzing
381	419	- Property-based tests
382	420	- Performance regression tests
383	421	- Memory leak detection
384	422
385	423	### Phase 7: Performance
	424	+
386	425	- DFA construction option
387	426	- Literal prefix optimization
388	427	- State minimization
389	428	- Benchmark suite
390	429
391	430	### Phase 8: Extended Features
	431	+
392	432	- Named capture groups
393	433	- Backreferences
394	434	- Lookahead/lookbehind

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -396,6 +436,7 @@ Matches formatting only
396	436	- Unicode properties
397	437
398	438	### Phase 9: API Improvements
	439	+
399	440	- Iterator interface
400	441	- Streaming matches
401	442	- Partial matching

Changes to docs/ARCHITECTURE.md

		@@ -404,10 +445,12 @@ Matches
404	445	## References
405	446
406	447	**Algorithms:**
	448	+
407	449	- Thompson, Ken (1968). "Regular Expression Search Algorithm"
408	450	- Cox, Russ (2007). "Regular Expression Matching Can Be Simple And Fast"
409	451
410	452	**Implementations:**
	453	+
411	454	- RE2 (Google) - Inspiration for architecture
412	455	- Rust regex crate - API design reference
413	456	- PCRE - Feature completeness reference

formatting only · 7 hidden lines

Changes to docs/ARCHITECTURE.md

		@@ -415,6 +458,7 @@ Matches formatting only
415	458	## Contributing
416	459
417	460	When modifying the architecture:
	461	+
418	462	1. Maintain phase separation
419	463	2. Keep allocations explicit
420	464	3. Preserve linear time guarantees