Motivation
We want to move to ECS log format and get benefits of a uniform logging schema on all the products. This allows us to develop ECS aware visualization of behavior like correlating metrics or events between products. The change in format is a breaking change and we should target it for 8.0.
History of logging in beats
Initially, beats were using a custom logging library and after we partially moved to uber's zap library. I believe the initial reason to move to this library was to reduce the scope of things we have to maintain, JSON output, better namespace support and also get a speed increase benefit when we are actively writing a lot of log events.
Note, I did use the word partially because we never completed the initial intended complete migration. We wanted to remove all global loggers instantiation and correctly create namespace per concerns, sadly we still have a lot of global loggers and not all logger instances have the right context attached to it which complicate debugging and support.
How we will migrate to ECS?
I think we have a few options to migrate to the ECS log format.
Option 1
Keep using the uber Zap library and add a custom formatter on top of it to add 10-15 ECS fields to each logs statement.
| Pros |
Cons |
| Low impact on current code |
Hard to extend when more typed fields are added (IP Address, Source) |
| Rely on external library reduced changes |
Do not improve error reporting, the logs are still flat and we don't have an easy way to add contextual data. |
|
When using the Structured export format it doesn't necessarily mean that we produce valid JSON. |
Option 2
A few month ago @urso, @ruflin and I talked about how we could better target ECS log format. With that @urso created a POC called ECSlog.
| Pros |
Cons |
| Can use any fields From ECS |
It's a complete library to maintain (exported, file rotation) |
| Typesafety (generated from ECS definition) |
Not stable yet. |
| Allow both machine readable error message and human readable error message |
|
| Strong emphasis on context embedded/traceability |
|
| Implement an ECS friendly error type |
|
| We can make the old logging statement works as is and gracefully upgrade |
|
| Enforce valid content with types |
|
Option 3
Could it be possible to mix Option 1 and Option 2, meaning that we delegate file rotation and exporter to Zap but we enforce everything on top of it. This actually mean that Zab mostly deal with raw bytes or string? I don't know if its possibe, WDYT @urso?
Next steps
In all the option above we need to write new codes, either we write a new formatter or bring ECSLog to stable, or a mix. We have to keep in mind, moving to a structured format mean that we will have to deal with backward compatibility.
I surely miss pros and cons and I would like your opinion @urso @graphaelli @ruflin @andresrc @webmat
related to #15129, elastic/apm-server#3155
Motivation
We want to move to ECS log format and get benefits of a uniform logging schema on all the products. This allows us to develop ECS aware visualization of behavior like correlating metrics or events between products. The change in format is a breaking change and we should target it for 8.0.
History of logging in beats
Initially, beats were using a custom logging library and after we partially moved to uber's zap library. I believe the initial reason to move to this library was to reduce the scope of things we have to maintain, JSON output, better namespace support and also get a speed increase benefit when we are actively writing a lot of log events.
Note, I did use the word partially because we never completed the initial intended complete migration. We wanted to remove all global loggers instantiation and correctly create namespace per concerns, sadly we still have a lot of global loggers and not all logger instances have the right context attached to it which complicate debugging and support.
How we will migrate to ECS?
I think we have a few options to migrate to the ECS log format.
Option 1
Keep using the uber Zap library and add a custom formatter on top of it to add 10-15 ECS fields to each logs statement.
Option 2
A few month ago @urso, @ruflin and I talked about how we could better target ECS log format. With that @urso created a POC called ECSlog.
Option 3
Could it be possible to mix Option 1 and Option 2, meaning that we delegate file rotation and exporter to Zap but we enforce everything on top of it. This actually mean that Zab mostly deal with raw bytes or string? I don't know if its possibe, WDYT @urso?
Next steps
In all the option above we need to write new codes, either we write a new formatter or bring ECSLog to stable, or a mix. We have to keep in mind, moving to a structured format mean that we will have to deal with backward compatibility.
I surely miss pros and cons and I would like your opinion @urso @graphaelli @ruflin @andresrc @webmat
related to #15129, elastic/apm-server#3155