Skip to content

[8.0](backport #28620) Update to ECS 8.0#28815

Merged
adriansr merged 2 commits into8.0from
mergify/bp/8.0/pr-28620
Nov 5, 2021
Merged

[8.0](backport #28620) Update to ECS 8.0#28815
adriansr merged 2 commits into8.0from
mergify/bp/8.0/pr-28620

Conversation

@mergify
Copy link
Copy Markdown
Contributor

@mergify mergify bot commented Nov 4, 2021

This is an automatic backport of pull request #28620 done by Mergify.
Cherry-pick of dde7a1f has failed:

On branch mergify/bp/8.0/pr-28620
Your branch is up to date with 'origin/8.0'.

You are currently cherry-picking commit dde7a1f96.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.next.asciidoc
	modified:   NOTICE.txt
	modified:   auditbeat/auditbeat.reference.yml
	modified:   auditbeat/cmd/root.go
	modified:   auditbeat/docs/fields.asciidoc
	modified:   auditbeat/include/fields.go
	modified:   auditbeat/module/auditd/_meta/accept.json
	modified:   auditbeat/module/auditd/_meta/execve.json
	modified:   auditbeat/module/auditd/audit_linux.go
	modified:   auditbeat/module/auditd/testdata/chown.log-expected.json
	modified:   auditbeat/module/auditd/testdata/setuid.log-expected.json
	modified:   filebeat/docs/fields.asciidoc
	modified:   filebeat/filebeat.reference.yml
	modified:   filebeat/include/fields.go
	modified:   filebeat/module/auditd/fields.go
	modified:   filebeat/module/auditd/log/_meta/fields.yml
	modified:   filebeat/module/auditd/log/ingest/pipeline.yml
	modified:   filebeat/module/auditd/log/test/audit-rhel6.log-expected.json
	modified:   filebeat/module/auditd/log/test/audit-rhel7.log-expected.json
	modified:   filebeat/module/auditd/log/test/audit-ubuntu1604.log-expected.json
	modified:   filebeat/module/auditd/log/test/test.log-expected.json
	modified:   filebeat/module/santa/log/ingest/pipeline.yml
	modified:   filebeat/module/santa/log/test/santa.log-expected.json
	modified:   go.mod
	modified:   go.sum
	modified:   heartbeat/cmd/root.go
	modified:   heartbeat/docs/fields.asciidoc
	modified:   heartbeat/heartbeat.reference.yml
	modified:   heartbeat/include/fields.go
	modified:   journalbeat/cmd/root.go
	modified:   journalbeat/docs/fields.asciidoc
	modified:   journalbeat/include/fields.go
	modified:   journalbeat/journalbeat.reference.yml
	modified:   libbeat/_meta/config/processors.reference.yml.tmpl
	modified:   libbeat/_meta/fields.ecs.yml
	new file:   libbeat/ecs/agent.go
	new file:   libbeat/ecs/as.go
	new file:   libbeat/ecs/base.go
	new file:   libbeat/ecs/client.go
	new file:   libbeat/ecs/cloud.go
	new file:   libbeat/ecs/code_signature.go
	new file:   libbeat/ecs/container.go
	new file:   libbeat/ecs/data_stream.go
	new file:   libbeat/ecs/destination.go
	new file:   libbeat/ecs/dll.go
	new file:   libbeat/ecs/dns.go
	new file:   libbeat/ecs/ecs.go
	new file:   libbeat/ecs/elf.go
	new file:   libbeat/ecs/error.go
	new file:   libbeat/ecs/event.go
	new file:   libbeat/ecs/file.go
	new file:   libbeat/ecs/geo.go
	new file:   libbeat/ecs/group.go
	new file:   libbeat/ecs/hash.go
	new file:   libbeat/ecs/host.go
	new file:   libbeat/ecs/http.go
	new file:   libbeat/ecs/interface.go
	new file:   libbeat/ecs/log.go
	new file:   libbeat/ecs/network.go
	new file:   libbeat/ecs/observer.go
	new file:   libbeat/ecs/orchestrator.go
	new file:   libbeat/ecs/organization.go
	new file:   libbeat/ecs/os.go
	new file:   libbeat/ecs/package.go
	new file:   libbeat/ecs/pe.go
	new file:   libbeat/ecs/process.go
	new file:   libbeat/ecs/registry.go
	new file:   libbeat/ecs/related.go
	new file:   libbeat/ecs/rule.go
	new file:   libbeat/ecs/server.go
	new file:   libbeat/ecs/service.go
	new file:   libbeat/ecs/source.go
	new file:   libbeat/ecs/threat.go
	new file:   libbeat/ecs/tls.go
	new file:   libbeat/ecs/tracing.go
	new file:   libbeat/ecs/url.go
	new file:   libbeat/ecs/user.go
	new file:   libbeat/ecs/user_agent.go
	new file:   libbeat/ecs/version.go
	new file:   libbeat/ecs/vlan.go
	new file:   libbeat/ecs/vulnerability.go
	new file:   libbeat/ecs/x509.go
	modified:   libbeat/processors/add_docker_metadata/add_docker_metadata_test.go
	modified:   libbeat/processors/add_docker_metadata/config.go
	modified:   libbeat/processors/add_docker_metadata/docs/add_docker_metadata.asciidoc
	modified:   libbeat/processors/add_process_metadata/add_process_metadata.go
	modified:   libbeat/processors/add_process_metadata/add_process_metadata_test.go
	modified:   libbeat/processors/add_process_metadata/config.go
	modified:   libbeat/processors/add_process_metadata/docs/add_process_metadata.asciidoc
	modified:   libbeat/publisher/processing/default.go
	modified:   libbeat/publisher/processing/default_test.go
	modified:   metricbeat/cmd/root.go
	modified:   metricbeat/docs/fields.asciidoc
	modified:   metricbeat/include/fields/fields.go
	modified:   metricbeat/metricbeat.reference.yml
	modified:   metricbeat/module/system/fields.go
	modified:   metricbeat/module/system/process/_meta/fields.yml
	modified:   metricbeat/module/system/process/doc.go
	modified:   metricbeat/module/system/process/process.go
	modified:   packetbeat/cmd/root.go
	modified:   packetbeat/docs/fields.asciidoc
	modified:   packetbeat/include/fields.go
	modified:   packetbeat/packetbeat.reference.yml
	modified:   packetbeat/pb/ecs.go
	modified:   packetbeat/pb/event.go
	modified:   packetbeat/pb/event_test.go
	modified:   packetbeat/protos/dhcpv4/dhcpv4.go
	modified:   packetbeat/protos/http/event.go
	modified:   packetbeat/protos/http/event_test.go
	modified:   packetbeat/protos/http/http.go
	modified:   packetbeat/protos/icmp/icmp.go
	modified:   packetbeat/protos/tls/tls.go
	modified:   packetbeat/publish/publish_test.go
	modified:   packetbeat/tests/system/golden/http_basic_auth-expected.json
	modified:   packetbeat/tests/system/test_0006_wsgi.py
	modified:   winlogbeat/cmd/root.go
	modified:   winlogbeat/docs/fields.asciidoc
	modified:   winlogbeat/include/fields.go
	modified:   winlogbeat/winlogbeat.reference.yml
	modified:   x-pack/auditbeat/auditbeat.reference.yml
	modified:   x-pack/auditbeat/module/system/process/process.go
	modified:   x-pack/auditbeat/module/system/process/process_test.go
	modified:   x-pack/auditbeat/tests/system/test_metricsets.py
	modified:   x-pack/filebeat/filebeat.reference.yml
	modified:   x-pack/filebeat/module/barracuda/spamfirewall/config/liblogparser.js
	modified:   x-pack/filebeat/module/barracuda/spamfirewall/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/barracuda/waf/config/liblogparser.js
	modified:   x-pack/filebeat/module/bluecoat/director/config/liblogparser.js
	modified:   x-pack/filebeat/module/bluecoat/director/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp1.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp2.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp3.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp4.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp5.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp6.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/amp/test/cisco_amp7.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/cisco/asa/test/additional_messages.log-expected.json
	modified:   x-pack/filebeat/module/cisco/ios/config/pipeline.js
	modified:   x-pack/filebeat/module/cisco/ios/pipeline_test.go
	modified:   x-pack/filebeat/module/cisco/meraki/config/liblogparser.js
	modified:   x-pack/filebeat/module/cisco/meraki/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/cisco/nexus/config/liblogparser.js
	modified:   x-pack/filebeat/module/cisco/shared/ingest/asa-ftd-pipeline.yml
	modified:   x-pack/filebeat/module/cylance/protect/config/liblogparser.js
	modified:   x-pack/filebeat/module/cylance/protect/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/f5/bigipafm/config/liblogparser.js
	modified:   x-pack/filebeat/module/f5/bigipafm/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/f5/bigipapm/config/liblogparser.js
	modified:   x-pack/filebeat/module/f5/bigipapm/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/fortinet/clientendpoint/config/liblogparser.js
	modified:   x-pack/filebeat/module/fortinet/clientendpoint/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/fortinet/fortimail/config/liblogparser.js
	modified:   x-pack/filebeat/module/fortinet/fortimail/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/fortinet/fortimanager/config/liblogparser.js
	modified:   x-pack/filebeat/module/fortinet/fortimanager/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/imperva/securesphere/config/liblogparser.js
	modified:   x-pack/filebeat/module/imperva/securesphere/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/infoblox/nios/config/liblogparser.js
	modified:   x-pack/filebeat/module/infoblox/nios/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/iptables/log/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/iptables/log/test/icmp.log-expected.json
	modified:   x-pack/filebeat/module/iptables/log/test/iptables.log-expected.json
	modified:   x-pack/filebeat/module/iptables/log/test/ipv6.log-expected.json
	modified:   x-pack/filebeat/module/iptables/log/test/ubiquiti.log-expected.json
	modified:   x-pack/filebeat/module/juniper/junos/config/liblogparser.js
	modified:   x-pack/filebeat/module/juniper/junos/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/juniper/netscreen/config/liblogparser.js
	modified:   x-pack/filebeat/module/juniper/netscreen/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/microsoft/_meta/kibana/7/visualization/62f081c0-ca68-11ea-9d4d-9737a63aaa55.json
	modified:   x-pack/filebeat/module/microsoft/defender_atp/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/microsoft/defender_atp/test/defender_atp-test.json.log-expected.json
	modified:   x-pack/filebeat/module/microsoft/dhcp/config/liblogparser.js
	modified:   x-pack/filebeat/module/microsoft/dhcp/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/microsoft/m365_defender/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/microsoft/m365_defender/test/m365_defender-test.ndjson.log-expected.json
	modified:   x-pack/filebeat/module/mssql/log/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/mssql/log/test/test.log-expected.json
	modified:   x-pack/filebeat/module/netscout/sightline/config/liblogparser.js
	modified:   x-pack/filebeat/module/netscout/sightline/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_j002_28264_20201007122838056263426565.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_m000_25412_20201007115808319837620840.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_m002_28369_20201007123022392204603031.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_m003_24705_20201007113805036530435635.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_m004_29815_20201007130106080227473114.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_m004_29815_20201007130106085635422771.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_m005_24958_20201007115707242540239811.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_10651_20201007131744782913507561.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13448_20201007105245091084628324.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13448_20201007105751018661587803.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13765_20201007105751904399925443.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13779_20201007105801980871631378.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13788_20201007105802970031936241.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13790_20201007105803021897922657.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13791_20201007105803053277493103.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13792_20201007105804019827529526.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_13794_20201007105804045603856206.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_9667_20201007130533504494345257.aud.log-expected.json
	modified:   x-pack/filebeat/module/oracle/database_audit/test/ORCLCDB_ora_9680_20201007130542916949215077.aud.log-expected.json
	modified:   x-pack/filebeat/module/panw/panos/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/panw/panos/test/threat.log-expected.json
	modified:   x-pack/filebeat/module/panw/panos/test/traffic_nanos_time.log-expected.json
	modified:   x-pack/filebeat/module/panw/panos/test/userid.log-expected.json
	modified:   x-pack/filebeat/module/proofpoint/emailsecurity/config/liblogparser.js
	modified:   x-pack/filebeat/module/proofpoint/emailsecurity/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/radware/defensepro/config/liblogparser.js
	modified:   x-pack/filebeat/module/snort/log/config/liblogparser.js
	modified:   x-pack/filebeat/module/snort/log/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/sonicwall/firewall/config/liblogparser.js
	modified:   x-pack/filebeat/module/sonicwall/firewall/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/sophos/utm/config/liblogparser.js
	modified:   x-pack/filebeat/module/squid/log/config/liblogparser.js
	modified:   x-pack/filebeat/module/squid/log/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/tomcat/log/config/liblogparser.js
	modified:   x-pack/filebeat/module/tomcat/log/test/generated.log-expected.json
	modified:   x-pack/filebeat/module/zookeeper/audit/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/zookeeper/audit/test/audit.log-expected.json
	modified:   x-pack/filebeat/module/zookeeper/log/ingest/pipeline.yml
	modified:   x-pack/filebeat/module/zookeeper/log/test/zookeeper.log-expected.json
	modified:   x-pack/filebeat/module/zscaler/zia/config/liblogparser.js
	modified:   x-pack/filebeat/module/zscaler/zia/test/generated.log-expected.json
	modified:   x-pack/functionbeat/docs/fields.asciidoc
	modified:   x-pack/functionbeat/functionbeat.reference.yml
	modified:   x-pack/functionbeat/include/fields.go
	modified:   x-pack/heartbeat/heartbeat.reference.yml
	modified:   x-pack/heartbeat/include/fields.go
	modified:   x-pack/metricbeat/cmd/root.go
	modified:   x-pack/metricbeat/metricbeat.reference.yml
	modified:   x-pack/osquerybeat/cmd/root.go
	modified:   x-pack/osquerybeat/docs/fields.asciidoc
	modified:   x-pack/osquerybeat/include/fields.go
	modified:   x-pack/osquerybeat/osquerybeat.reference.yml
	modified:   x-pack/packetbeat/packetbeat.reference.yml
	modified:   x-pack/winlogbeat/winlogbeat.reference.yml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   x-pack/filebeat/module/cisco/ios/test/cisco-ios-syslog.log-expected.json
	both modified:   x-pack/filebeat/module/iptables/log/test/geo.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/global_protect.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/hipmatch.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/pan_inc_other.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/pan_inc_threat.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/pan_inc_traffic.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/pan_inc_traffic_ietf.log-expected.json
	both modified:   x-pack/filebeat/module/panw/panos/test/traffic.log-expected.json
	both modified:   x-pack/filebeat/module/sonicwall/firewall/test/general.log-expected.json
	both modified:   x-pack/filebeat/module/sophos/utm/test/generated.log-expected.json
	both modified:   x-pack/filebeat/module/squid/log/test/access1.log-expected.json

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

Updates the ECS field definitions to the current master (8.0-dev).

- add_process_metadata processor: Replace usage of deprecated `process.ppid` field with `process.parent.pid`.
- add_docker_metadata processor: Replace usage of deprecated `process.ppid` field with `process.parent.pid`.
- Auditbeat: Replace usage of deprecated `process.ppid` field with `process.parent.pid`.
- Auditbeat system/process metricset: Replace usage of deprecated `process.ppid` field with `process.parent.pid`.
- Filebeat all modules: Replace usages of deprecated ECS fields `process.ppid` and `log.original` with `process.parent.pid` and `event.original`.
- Filebeat: Replace usages of `host.user.*` fields with `user.*` in `cisco`, `microsoft` and `oracle` modules.
- Packetbeat http: The field `http.request.method` will maintain its original case.

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
(cherry picked from commit dde7a1f)

# Conflicts:
#	x-pack/filebeat/module/cisco/ios/test/cisco-ios-syslog.log-expected.json
#	x-pack/filebeat/module/iptables/log/test/geo.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/global_protect.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/hipmatch.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/pan_inc_other.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/pan_inc_threat.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/pan_inc_traffic.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/pan_inc_traffic_ietf.log-expected.json
#	x-pack/filebeat/module/panw/panos/test/traffic.log-expected.json
#	x-pack/filebeat/module/sonicwall/firewall/test/general.log-expected.json
#	x-pack/filebeat/module/sophos/utm/test/generated.log-expected.json
#	x-pack/filebeat/module/squid/log/test/access1.log-expected.json
@mergify mergify bot requested review from a team as code owners November 4, 2021 16:25
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Nov 4, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 4, 2021
@botelastic
Copy link
Copy Markdown

botelastic bot commented Nov 4, 2021

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Nov 4, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-11-04T18:13:52.770+0000

  • Duration: 251 min 23 sec

  • Commit: 558ffa7

Test stats 🧪

Test Results
Failed 0
Passed 53475
Skipped 5242
Total 58717

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

@adriansr adriansr self-requested a review November 4, 2021 18:22
@adriansr adriansr merged commit ed7ff56 into 8.0 Nov 5, 2021
@adriansr adriansr deleted the mergify/bp/8.0/pr-28620 branch November 5, 2021 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport conflicts There is a conflict in the backported pull request needs_team Indicates that the issue/PR needs a Team:* label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants