Conversation
|
Does this need to be backported to 7.17? |
|
@ferullo Yes, this would need to be backported to |
|
Should we consider this a breaking change? @andrewkroh what do we usually do in a package when we change a field type? @bradenlpreston |
|
When the integration is upgraded by Fleet the data stream will be rolled over in order to apply the new mapping. Any data that is ingested after that will need to align to the type (keyword). In this case that won't be a problem since binary data must be base64 encoded and base64 is obviously valid for a keyword field. You may want to add a step to your Ingest Node pipelines to conditionally convert incoming base64 data to hexidecimal to account for Agents that are not yet upgraded. I think you'd need to use Painless to handle this (like https://www.elastic.co/guide/en/elasticsearch/painless/master/painless-api-reference-shared-java-util.html#painless-api-reference-shared-Base64-Decoder). The last issue to consider are queries. Are there any consumers (rules, alerts, dashboards, etc) of this field in the Elastic ecosystem that will be broken by the change or by receiving mixed results (base64 and hex) when hitting older indices? You should test this on a cluster having both a binary and keyword mapped indices to see if you encounter any other issues (like check Discover and check the Kibana index pattern / data view UI). |
pzl
left a comment
There was a problem hiding this comment.
the change here is ok
sounds like what's being sent from endpoint is not changing at all, just how it's stored, so that obviates some of the above complications.
This won't re-index the existing data, it will just index anything after package update. Not much to do there.
If this needs to be exceptionable, then it probably needs to be added to the Kibana repo here: x-pack/plugins/security_solution/public/common/components/exceptions/exceptionable_endpoint_fields.json
Change Summary
Reference: #233
Ransomware alerts may contain
Ransomware.files.dataand / orRansomware.child_processes.files.data, and we needed to update the type of these fields frombinarytokeyworddue to how we have traditionally been generating these alerts (with strings of hexadecimal character) and in order to allow for these fields to be exceptionable.Sample values
Complete example of Endpoint ransomware alert with Ransomware.files.data (click to expand)
Complete example of Endpoint ransomware alert with Ransomware.child_processes.files.data (click to expand)
Release Target
8.2For mapping changes:
makeafter making the schema changes, and committed any generated files (inschema/,generated/)