Convert apache2.access to ECS - Take 2#9245
Merged
webmat merged 14 commits intoelastic:masterfrom Nov 29, 2018
Merged
Conversation
Contributor
Author
|
@ruflin I brought your initial Apache2 access PR exactly in line with the other recent access log PRs. Expecting to have the build work the first time. |
ruflin
approved these changes
Nov 27, 2018
ruflin
reviewed
Nov 27, 2018
53da6b2 to
5d95880
Compare
5d95880 to
321c5a9
Compare
Contributor
Author
|
Only failure in Jenkins was a network hiccup on Ubuntu:
|
ruflin
reviewed
Nov 28, 2018
* Update ecs-migration.yml file * Update changelog * Update generated files * Link old fields Todo: * Add aliases for old fields
- remove `source.hostname` field definition - remove misspelled `http.request.referer`. The rreal one is already defined. - pipeline reverted to populate `apache2.access.remote_ip`. - add split to source.ip or source.domain as a separate grok.
- Output the ua string to the field not afflicted with a typo. - Fix user agent parsing - Add a log entry with a hostname as the remote host, instead of an IP - Add leniency to geoip filter, if remote host is a hostname instead of an IP (no ip field)
- Remove unneeded definition for `http.response.body_sent.bytes`. - Re-introduce `apache2.access.remote_ip` field definition. - Alias all `apache2.access.*` that have moved.
Dupe of definition in `libbeat/_meta/fields.ecs.yml`.
f5ddc46 to
54a294e
Compare
Contributor
Author
|
@ruflin Ok, ready for final review. Everything is green, and even fixes the test breakage in master. |
jsoriano
approved these changes
Nov 28, 2018
Member
|
For a future change, we might consider to rename the |
Contributor
Author
|
@jsoriano Actually "Apache" is the foundation. The webserver is actually called "httpd" ;-) |
Contributor
Author
|
But I totally agree on another part of your point. The number is problematic. Will "apache2" support Apache httpd v3? So yeah, the number has to go eventually |
Member
|
@webmat apart of the number I was mentioning that to use the same name as the metricbeat module, though we can also rename both to |
Contributor
Author
|
Ah I was not aware of that. Thanks for mentioning it! |
3 tasks
14 tasks
18 tasks
DStape
pushed a commit
to DStape/beats
that referenced
this pull request
Aug 20, 2019
- Convert many fields under `apache2.access.*` to ECS. Previous field names are field aliases towards the new corresponding ECS field: - apache2.access.user_name => user.name - apache2.access.method => http.request.method - apache2.access.url => url.original - apache2.access.http_version => http.version - apache2.access.response_code => http.response.status_code - apache2.access.referrer => http.request.referrer - apache2.access.agent => user_agent.original - read_timestamp => event.created - apache2.access.geoip.* => source.geo.* - apache2.access.user_agent.* => user_agent.* - Rename common field `url.href` to `url.original`, bringing it in line with ECS - Remove field defs which are no longer in ECS: - url.href.raw - source.hostname - Add a log entry with a hostname as the remote host, instead of an IP - Add leniency to geoip filter, if remote host is a hostname instead of an IP (no ip field) - Coerce int fields to ints: status_code, sent bytes - Fix unrelated test failure caused by generated code (under x-pack/metricbeat) that seems to be outdated in master
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Taking over #8901, and I don't have write access to @ruflin's repo.
TODO:
url.originalfromfilebeat/_meta/fields.common.yml. It's a dupe oflibbeat/_meta/fields.ecs.ymldefinition.https://travis-ci.org/elastic/beats/jobs/460918667Unrelated
Closes #8901, partially addresses #8655.