Following @mvdan's question about upstreaming improvements, I'd like to discuss what path forward makes sense for the community.

Related issues:

• regexp: investigate further performance improvements #26623 - regexp: optimize performance (opened 2018, 180+ comments, still open)
• proposal: regexp: Optimize fixed-length patterns #21463 - regexp: performance improvements
• regexp: port RE2's DFA matcher to the regexp package #11646 - regexp: consider DFA-based implementation
• simd/archsimd: architecture-specific SIMD intrinsics under a GOEXPERIMENT #73787 - Go native SIMD support (relevant for future optimizations)

Background

Go's regexp package intentionally uses a simple NFA implementation to guarantee O(n) time complexity with no backtracking. This design prevents catastrophic backtracking but results in slower performance compared to other languages.

From the mariomka/regex-benchmark referenced in #26623:

Year	Go Total	Rust Total	Gap
2018 (original)	1562 ms	70 ms	22x
2024 (update)	632 ms	6 ms	105x

Go has improved (~2.5x faster), but Rust improved even more with regex crate optimizations. The relative gap has actually widened.

Issue #26623 has been open for 7 years. Go team has made incremental improvements, but architectural limitations remain.

Implementation

I've developed coregex — a pure-Go regexp alternative inspired by Rust's regex crate architecture.

Architecture Comparison

Aspect	stdlib regexp	coregex
Engine	Single NFA (PikeVM)	Multi-engine (11 strategies)
DFA	None	Lazy DFA with caching
SIMD	None	AVX2/SSSE3 prefilters
Literal optimization	Basic	Reverse suffix/inner search
Multi-pattern	None	Teddy algorithm
Memory	Minimal	Adaptive (trade memory for speed)
O(n) guarantee	✅ Yes	✅ Yes (no backtracking)

Strategy Selection

Strategy	Use Case	Speedup
ReverseInner	.*keyword.* patterns	1000-3000x
ReverseSuffix	.*\.txt suffix patterns	100-400x
CharClassSearcher	[\w]+ character classes	20-25x
Teddy	foo|bar|baz alternations	15-240x
LazyDFA	Complex patterns with literals	10-50x
OnePass	Anchored patterns with captures	10x
BoundedBacktracker	Small patterns	2-5x

Current Status (v0.8.22)

• API: Drop-in compatible with regexp.Regexp
• Platforms: All Go platforms (SIMD with pure-Go fallback)
• Testing: Battle-tested via GoAWK integration (15+ edge cases found and fixed)
• CI: Automated benchmarks on every commit

Benchmarks

CI-verified results from kolkov/regex-bench (GitHub Actions Ubuntu, 6MB input):

Pattern	Go stdlib	Go coregex	Rust regex	vs stdlib	vs Rust
uri	257 ms	1.3 ms	0.8 ms	192x faster	1.6x slower
email	259 ms	1.5 ms	1.5 ms	172x faster	tie
suffix	240 ms	1.5 ms	1.3 ms	166x faster	~tie
inner_literal	232 ms	1.5 ms	0.6 ms	153x faster	2.5x slower
char_class	550 ms	26 ms	52 ms	21x faster	2x faster
literal_alt	473 ms	31 ms	0.8 ms	15x faster	39x slower
ip	493 ms	163 ms	12 ms	3x faster	14x slower

Analysis

• vs stdlib: 3-192x faster on all patterns
• vs Rust: Competitive on most patterns, wins on char_class
• Known gaps: literal_alt (need Aho-Corasick), ip (complex alternations)

Questions for the community

Before investing more effort, I'd like to understand actual community needs:

1. Is stdlib regexp performance a real problem for you?

• 👍 Yes, it's a bottleneck in my applications
• 👎 No, stdlib performance is acceptable for my use cases

2. If faster regexp is needed, what delivery would you prefer?

• (a) Standalone library is fine — I can import external packages
• (b) Would prefer stdlib integration — fewer dependencies matter
• (c) Either option works for me

3. What patterns cause performance issues in your applications?

• Log parsing (error|warning|fatal)
• Email/URL validation
• Configuration file processing
• Other (please describe)

4. What would prevent you from using coregex today?

• API compatibility concerns
• Trust/stability concerns
• Don't want external dependencies
• Performance is not my bottleneck
• Other

Compatibility

• API: Drop-in replacement for regexp.Regexp (same method signatures)
• Behavior: Same matching semantics, O(n) guarantees preserved
• Breaking changes: None if used as external library
• Platforms: All Go platforms (SIMD with pure-Go fallback)

Upstream Discussion

Following @mvdan's question — yes, I'm open to upstreaming, but the approach matters:

Option A: Incremental PRs to stdlib

Individual optimizations that could be PRed separately:

• Literal prefix/suffix fast paths
• Better memory pooling in NFA execution
• Integration with Go's upcoming SIMD (simd/archsimd: architecture-specific SIMD intrinsics under a GOEXPERIMENT #73787)

Pros: Lower review burden, gradual improvement
Cons: Architectural limits — some optimizations (lazy DFA, multi-engine) require deeper changes

Option B: New regexp/v2 package

Similar to encoding/json/v2 approach — new package with better performance, same API.

Pros: Clean break, full optimization potential
Cons: Ecosystem fragmentation, maintenance burden

Option C: Continue as external library

Community uses coregex when performance matters, stdlib for simplicity.

Pros: No Go team burden, faster iteration
Cons: External dependency, less visibility

Regarding Go SIMD (#73787)

coregex currently uses hand-written AVX2/SSSE3 assembly. Once #73787 lands, these could be rewritten in pure Go with math/simd, making the code:

• More maintainable (no assembly)
• Portable to more platforms
• Easier to upstream

I'm watching #73787 closely and willing to migrate once it's available.

Call for testers

We urgently need testers with real-world projects — similar to how @benhoyt tested coregex with GoAWK.

That integration alone uncovered 15+ edge cases that benchmarks never caught: Unicode handling, empty match semantics, capture group edge cases, anchoring behavior, and more. Each bug found makes the library more robust.

If you have a project that heavily uses regexp, please consider:

1. Running your test suite with coregex as a drop-in replacement
2. Reporting any failures or unexpected behavior
3. Sharing benchmark comparisons from your real workloads

Real-world testing is the fastest path to production quality. Synthetic benchmarks can only go so far.

What I'm asking

I'm not advocating for any specific path. I genuinely want to understand:

1. Is there community demand for faster regexp?
2. What delivery method would be most valuable?
3. Would the Go team consider architectural changes to stdlib regexp?

If the answer is "stdlib is fine, use external libraries when needed" — that's a valid answer too.

Links

• Repository: https://github.com/coregx/coregex
• Benchmarks (CI): https://github.com/kolkov/regex-bench
• GoAWK integration: Coregex actually significantly slower than stdlib regexp in most cases coregx/coregex#29
• Organization: https://github.com/coregx
• Author: @kolkov