Refactor add_cloud_metadata to handle ECS fields easier#26438
Refactor add_cloud_metadata to handle ECS fields easier#26438ChrsMark merged 6 commits intoelastic:masterfrom
Conversation
Signed-off-by: chrismark <chrismarkou92@gmail.com>
|
Pinging @elastic/integrations (Team:Integrations) |
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
kaiyan-sheng
left a comment
There was a problem hiding this comment.
Looks good to me besides one naming issue :)
Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
jsoriano
left a comment
There was a problem hiding this comment.
Not blocking this PR as I think that the refactor in addMeta would be enough to avoid the different code to handle overwrites. 👍
But in the fetcher I still see some special code to handle cloud and orchestrator fields, I wonder if this could be unified too.
| extractECSField("orchestrator", out, meta) | ||
|
|
||
| meta.DeepUpdate(common.MapStr{"cloud": out}) |
There was a problem hiding this comment.
It still seems that some special handling is done for orchestrator vs cloud fields. And it may be a bit confusing to have an out, that is not an output of the function at the end.
Wdyt about setting the orchestrator values directly on meta, so they don't have to be moved from out to meta?
And in the same way, perhaps cloud fields could be directly set on meta["cloud"], so this intermediary out is not needed.
There was a problem hiding this comment.
👍🏼 Good point! I tried to get rid of the extra "cooking" and applied the schema for orchestrator fields on a different map, since unfortunately Apply cannot work in a "DeepUpdate" way so it's not possible to write directly to cloud.* in multiple places since cloud* will be overridden each time Apply is called.
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
(cherry picked from commit f8bb3a2)
…arwin-arm64 * upstream/master: (295 commits) Update urllib to 1.26.5. (elastic#26380) Update golang.org/x/crypto (elastic#26448) [Filebeat] Update Fortinet Ingest Pipeline (elastic#24816) Move parsers outside of filestream input so others can use them as well (elastic#26541) [Filebeat] Fix `threatintel.indicator.url.full` field not populating (elastic#26508) [Filebeat] Add network direction processor to Zeek and Suricata modules (elastic#24620) Logging code cleanup related to Nomad auto-discovery (elastic#26498) [Metricbeat] Add Couchbase's Sync Gateway module (elastic#25599) Refactor add_cloud_metadata to handle ECS fields easier (elastic#26438) [Elastic Agent] Improper casting of int64 (elastic#26520) [Elastic Agent] Enable configuring monitoring namespace (elastic#26439) [Heartbeat] configure permissions for synthetics config (elastic#26393) Osquerybeat: set the raw index name to supress the timestamp suffix (elastic#26545) [Heartbeat] add screenshots config to synthetics (elastic#26455) [Elastic Agent] Use http2 to connect to Fleet Server. (elastic#26474) Remove all docs about Beats central management (elastic#26399) update data.json for gcp billing (elastic#26506) Skip x-pack metricbeat tests (elastic#26537) [Elastic Agent] Fix issue with FLEET_CA not being used with Fleet Server in container (elastic#26529) Add changelog entry for elastic#26224 (elastic#26531) ...
What does this PR do?
This PR refactors add_cloud_metadata processor as a follow-up of #26056 so as to provide a better handling for ECS fields which will be reported under a different namespace other than
cloud.*. See more at #26337Why is it important?
So as to provide a better experience for ECS fields handling by the add_cloud_metadata processor.
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.How to test this PR locally
go test -v github.com/elastic/beats/v7/libbeat/processors/add_cloud_metadata/...Related issues