-
Notifications
You must be signed in to change notification settings - Fork 5k
[WIP] Filebeat module issues found during ECS conversion #9208
Copy link
Copy link
Closed
Labels
FilebeatFilebeatFilebeatTeam:IntegrationsLabel for the Integrations teamLabel for the Integrations teammodulev7.0.0
Description
Please ignore for now.
This is just a collection of all the small problems we notice could be improved while performing the ECS transition (#8655). Some things are small enough that we can tackle them right in the ECS PRs.
Some things are out of scope, though. Here they are:
Usage of http.response.elapsed_time
This field is not in ECS and should be deprecated in favour of event.duration. Modules using it:
- kibana
Full version strings vs breakdown fields
- Ingest Node's UA parser gives us the version number in broken up in 3 fields (major/minor/patch), but no full version string (e.g.
10.14.1). We should populateuser_agent.versionanduser_agent.os.versionwith full version strings. Modules affected:- apache2.access
- iis.access
- nginx.access
- traefik.access
system.syslog
- Add integration test for the
convert_timezoneoption
system.auth
- Add integration tests for message formats other than
sshd,sudo,groupaddanduseradd. Right now it's only testing for those, and no test for the last GROK pattern that just populates the message field. - Patterns specific to
sshd,sudo,groupaddanduseradd- Do not set
process.name. They should. - Do not set
message. Although this point will be solved by Filebeat modules: keep raw message #8083.
- Do not set
- After performing field renames on auth messages, I wonder if the resulting events are missing something. For example the first pattern uses only ECS fields, if it wasn't for one last field that hasn't been renamed (system.auth.ssh.method, line 29), I would never know that its ECS event is a message about SSH.
nginx.access
- Current parsing of access log assumes people are using
$http_x_forwarded_forrather than the default$remote_addrfor their remote address. This means even though nginx can receive connections from unix sockets (e.g. common when polling for nginx stats), this module doesn't support these log events.
Message in error filesets
E.g. apache.error, nginx.error
- Currently outputs to
messagerather thanerror.message, since all Ingest Node documentation currently suggests outputting Ingest Node error messages aterror.message.- I'm not 100% convinced about using
error.messagein any case for userland error messages, by the way. We've been talking about always populatingmessagefor the timeline viewer.
- I'm not 100% convinced about using
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
FilebeatFilebeatFilebeatTeam:IntegrationsLabel for the Integrations teamLabel for the Integrations teammodulev7.0.0
Type
Fields
Give feedbackNo fields configured for issues without a type.