rare icon indicating copy to clipboard operation
rare copied to clipboard

Why not use grok?

Open CameronNemo opened this issue 6 years ago • 5 comments

Hello, grok is a generally common log parsing language that allows for a clear combination of regular expressions. It is used in tools like logstash and vector. I was just curious why you opted for traditional regex and match groups rather than using grok.

Thanks, Cam.

CameronNemo avatar Dec 04 '19 17:12 CameronNemo

No reason other than it was an easy way to start, and is logically compatible with most tools it's trying to work with (grep, ag, etc). If there's a strong argument and a cross-platform library for grok parsing, I would be open to its implementation.

zix99 avatar Dec 08 '19 19:12 zix99

There are pros and cons when it comes to trust other people's/system's regex hells and heavens, cf. e.g. https://github.com/vjeantet/grok/blob/master/patterns.go For instance, on the one hand I'd be surprised if using grok patterns like these are as fast as well chosen, short and dedicated regex expressions. On the other hand there's the comfort of prefabricated patterns for some kinds of log files. My 2c.

xrat avatar May 28 '21 11:05 xrat

For ease, I'd definitely be open to some sort of preprocessing of regex like vjeantet/grok. I think it'd add a little of accessibility/ease (Which is really what rare aims for at the end of the day). I'll take a look into it, and either use it directly, or maybe borrow the idea (TBD)

zix99 avatar Jun 01 '21 00:06 zix99

To circle back on this: I did end up writing a proof of concept on a local branch to test out grok expressions, borrowing the specs from logstash's legacy library. I got it mostly working, but what got me was the lack of compatibility for regex lookahead/back that logstash uses heavily and go doesn't support. This effectively means that a good chunk of their expressions don't work (If you look at vjeantet/grok, there's a bunch commented out for this reason).

After all that, I didn't feel like supporting grok added a ton of value to rare, especially given the tools it's trying to compare to (grep/ack/sed/etc), while also adding a bunch of technical bloat and multiple ways of doing things.

I could still probably be convinced otherwise, and I'd consider pushing the branch if anyone is interested, but at this point that's where I stand.

zix99 avatar Jul 10 '21 16:07 zix99

That makes sense to me. If you do not think it adds a lot of value, probably best to not merge the support.

CameronNemo avatar Jul 10 '21 20:07 CameronNemo