log: implement a crdb-v2 log entry parser and use it in debug merge-logs #65504
Description
Since v21.1 the log files use a new file format called crdb-v2.
- Previously (in
crdb-v1) entries were not delimited properly, so the entry parser was using heuristics to find the entry boundary and had a maximum size of 64K - In the new format, entries can be split across multiple lines, with a continuation delimiter. There is no maximum size.
However, the new format is not recognized by the tool debug merge-log which we routinely use in tech support. This makes it difficult/impossible to merge log files properly.
We want to create an entry parser and integrate it into debug merge-log. The new parser should be combined with the following new functionality in debug merge-log:
- load multi-line entries as a single entry in memory for the purpose of merging log files together.
- convert from one format to another. In particular we want to offer the ability to emit v2 entries in the
crdb-v1format for compatibility with previous-version tools
The new parser should exist alongside the v1 parser (i.e. it is an addition, not a replacement)
At this point, the work described in this issue is not an invitation to update the GetLogs HTTP API to use the new parser. We will need to discuss later (in a separate issue) what we want to do with that API; it's possible we will need to change it entirely so it remains out of scope here..