Rename source_ecs to source#8983
Conversation
ad51a65 to
93d9d6c
Compare
Update Suricata module to use `source` instead of `source_ecs` now that the Filebeat source field has been changed. This updates `libbeat/_meta/fields.ecs.yml` to include the ECS `source.*` fields. This updates the dashboards and visualizations to use the new `source` fields. And I also updated the saved searches to use `event.module` instead of `fileset.module`. Remove source.ip, source,port, and source.hostname from auditbeat's auditd module fields. Remove source.ip, source.port, and source.mac from packetbeat's flow fields.
93d9d6c to
3eb2749
Compare
libbeat/_meta/fields.ecs.yml
Outdated
|
|
||
| Can be one or multiple IPv4 or IPv6 addresses. | ||
|
|
||
| - name: hostname |
There was a problem hiding this comment.
@andrewkroh @webmat This does not exist anymore in ECS. Do we need it for suricata? Can you two sync up?
There was a problem hiding this comment.
The ECS fields I used are stale because I moved this source definition from filebeat to libbeat, rather than getting an entirely new copy from elastic/ecs. We should do a full update in a separate PR (we have to verify that any fields removed from ECS are still accounted for elsewhere in fields.yml files when we do this).
Auditbeat had source.hostname in its fields.yml, so I had to remove it because it was now duplicated. I think Auditbeat uses source.hostname in cases where the SSH daemon reports both the remote client's IP and the reverse DNS name of that IP.
We can proceed with a rename in Auditbeat or a change ECS. Similarly we are using source.geo.* which is not in ECS.
There was a problem hiding this comment.
Just to clarify, geo is a reuseable object, and source and destination are both expected places where geo can be nested. The template.json doesn't reflect it yet, but the official spec is the readme :-)
So source.geo.* is perfectly ECS compliant.
There was a problem hiding this comment.
Agreed that source.hostname should be replaced by source.domain. Same goes for destination.hostname.
If you want to store an ambiguous value as well, before determining if it's a hostname or an IP, ECS no longer has a field for this (at least for now). You may store it in a custom field, or you may add logic to save to .ip only when it's an IP, or save to .domain when it's a hostname/domain.
There was a problem hiding this comment.
Ok, I'll update Auditbeat to send source.domain instead of source.hostname.
There was a problem hiding this comment.
I removed the source.hostname and destination.hostname fields. And I added the source.geo.* and destination.geo.*.
I reverted the changes to Auditbeat's mapping w.r.t. source.hostname as I will do that in an independent PR since it's separate change.
| "disabled": false, | ||
| "index": "filebeat-*", | ||
| "key": "fileset.module", | ||
| "key": "event.module", |
There was a problem hiding this comment.
Not sure if we should update the dashboards here as I planned to this in bulk. Like this we have some dashboards converted but not all. But not sure if it will have an affect on the migration.
There was a problem hiding this comment.
I needed to update them to test my UI changes, but I could undo these lines. However if the bulk migration just does a find/replace on fileset.module this won't be a problem. I'm in favor of leaving it as is, but I'll follow your direction.
| "source.ip": "192.168.1.146", | ||
| "source.port": 52340, | ||
| "suricata.eve.alert.action": "allowed", | ||
| "suricata.eve.alert.category": "Not Suspicious Traffic", |
webmat
left a comment
There was a problem hiding this comment.
LGTM overall.
I'd just make sure to get rid of source.hostname and destination.hostname.
I don't have a strong opinion on whether or not to rename fileset.module now vs as part of Nic's PR. I agree that if it's done via a find/replace, it shouldn't matter :-)
| alias: true | ||
| copy_to: false | ||
|
|
||
| - from: source_ecs.geo.continent_name |
There was a problem hiding this comment.
Needlessly verbose, IMO. What about the following?
- from: source_ecs.geo
to: source.geoThere was a problem hiding this comment.
I don't know how ecs-migration.yml will be used so I can't say for sure whether this is needlessly verbose. But if I can speculate, I think the level of detail here is appropriate if this will be used to create aliases because aliases must target concrete fields, not objects like source.geo.
libbeat/_meta/fields.ecs.yml
Outdated
|
|
||
| Can be one or multiple IPv4 or IPv6 addresses. | ||
|
|
||
| - name: hostname |
There was a problem hiding this comment.
Agreed that source.hostname should be replaced by source.domain. Same goes for destination.hostname.
If you want to store an ambiguous value as well, before determining if it's a hostname or an IP, ECS no longer has a field for this (at least for now). You may store it in a custom field, or you may add logic to save to .ip only when it's an IP, or save to .domain when it's a hostname/domain.
Removes `source.hostname` and `destination.hostname` from fields.ecs.yml. Adds `source.geo` and `destination.geo` to fields.ecs.yml.
|
@andrewkroh Actually there's a mention of |
* Rename source_ecs to source Update Suricata module to use `source` instead of `source_ecs` now that the Filebeat source field has been changed. This updates `libbeat/_meta/fields.ecs.yml` to include the ECS `source.*` fields. This updates the dashboards and visualizations to use the new `source` fields. And I also updated the saved searches to use `event.module` instead of `fileset.module`. - Remove source.ip, source,port from auditbeat's auditd module fields (source.hostname remains, will remove separately). - Remove source.ip, source.port, and source.mac from packetbeat's flow fields. - Adds `source.geo` and `destination.geo` to fields.ecs.yml.
Update Suricata module to use
sourceinstead ofsource_ecsnow that the Filebeat sourcefield has been changed. This updates
libbeat/_meta/fields.ecs.ymlto include the ECSsource.*fields.This updates the dashboards and visualizations to use the new
sourcefields. And I also updatedthe saved searches to use
event.moduleinstead offileset.module.