Skip to content

Add pipeline tests for AWS CloudTrail#408

Merged
andrewkroh merged 1 commit intoelastic:masterfrom
andrewkroh:feature/aws/cloudtrail-tests
Dec 2, 2020
Merged

Add pipeline tests for AWS CloudTrail#408
andrewkroh merged 1 commit intoelastic:masterfrom
andrewkroh:feature/aws/cloudtrail-tests

Conversation

@andrewkroh
Copy link
Copy Markdown
Member

@andrewkroh andrewkroh commented Nov 19, 2020

What does this PR do?

The tests revealed a few issues. There was an error in the pipeline for update-user-json.log because
serviceEventDetails was not present. This was the error

        "error": {
            "message": "Cannot invoke \\\"Object.getClass()\\\" because \\\"receiver\\\" is null"
        }

The aws.cloudtrail.read_only field was mapped as keyword but was actual a JSON boolean.
I changed the type to boolean, but do not plan to backport this change to Filebeat.

And lastly some ECS user_agent fields were missing.

This depends on elastic/elastic-package#177 to make the flattened fields pass
test validation.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all datasets collect metrics or logs.

Related issues

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Nov 19, 2020

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #408 updated

  • Start Time: 2020-12-01T23:05:17.679+0000

  • Duration: 24 min 57 sec

Test stats 🧪

Test Results
Failed 0
Passed 57
Skipped 0
Total 57

@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

Copy link
Copy Markdown
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As there were incompatible changes pushed to the master branch (PR: #410), please rebase this PR against the master branch.

You can fix go.sum with mage ModTidy.

The tests revealed a few issues. There was an error in the pipeline for update-user-json.log because
serviceEventDetails was not present. This was the error

            "error": {
                "message": "Cannot invoke \\\"Object.getClass()\\\" because \\\"receiver\\\" is null"
            }

The aws.cloudtrail.read_only field was mapped as keyword but was actual a JSON boolean.
I changed the type to boolean, but do not plan to backport this change to Filebeat.

And lastly some ECS user_agent fields were missing.

This depends on elastic/elastic-package#177 to make the flattened fields pass
test validation.
Copy link
Copy Markdown
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great.

@andrewkroh andrewkroh merged commit 270be37 into elastic:master Dec 2, 2020
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Jan 11, 2021
andrewkroh added a commit to elastic/beats that referenced this pull request Jan 25, 2021
* Sync changes to AWS CloudTrail

elastic/integrations#408

* Sync changes to CheckPoint Firewall

Change type of event.severity.

elastic/integrations#409

* Sync changes from Cisco ASA / FTD

elastic/integrations#414

* Sync changes from Cisco IOS

Make icmp and igmp fields strings because they are keywords.

elastic/integrations#416

* Sync changes to CrowdStrike Falcon

Fix some field types.

elastic/integrations#377

* Sync changes to Fortinet Firewall

Drop assignip if the value is "N/A".

elastic/integrations#437

* Sync changes to Juniper SRX

Convert event.risk values to float
Protect against missing event.timezone
Convert event.severity to long.

elastic/integrations#443

* Sync changes to Suricata EVE

Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457

* Sync changes to Zeek DNS

Fix usages of ignore_failure with convert processor.
Make DNS transaction ID a string.

elastic/integrations#448

* Add changelog
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Feb 16, 2021
* Sync changes to AWS CloudTrail

elastic/integrations#408

* Sync changes to CheckPoint Firewall

Change type of event.severity.

elastic/integrations#409

* Sync changes from Cisco ASA / FTD

elastic/integrations#414

* Sync changes from Cisco IOS

Make icmp and igmp fields strings because they are keywords.

elastic/integrations#416

* Sync changes to CrowdStrike Falcon

Fix some field types.

elastic/integrations#377

* Sync changes to Fortinet Firewall

Drop assignip if the value is "N/A".

elastic/integrations#437

* Sync changes to Juniper SRX

Convert event.risk values to float
Protect against missing event.timezone
Convert event.severity to long.

elastic/integrations#443

* Sync changes to Suricata EVE

Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457

* Sync changes to Zeek DNS

Fix usages of ignore_failure with convert processor.
Make DNS transaction ID a string.

elastic/integrations#448

* Add changelog

(cherry picked from commit bf46572)
adriansr pushed a commit to elastic/beats that referenced this pull request Feb 17, 2021
#24077)

* Sync fixes from Integration Package Testing (#23424)

* Sync changes to AWS CloudTrail

elastic/integrations#408

* Sync changes to CheckPoint Firewall

Change type of event.severity.

elastic/integrations#409

* Sync changes from Cisco ASA / FTD

elastic/integrations#414

* Sync changes from Cisco IOS

Make icmp and igmp fields strings because they are keywords.

elastic/integrations#416

* Sync changes to CrowdStrike Falcon

Fix some field types.

elastic/integrations#377

* Sync changes to Fortinet Firewall

Drop assignip if the value is "N/A".

elastic/integrations#437

* Sync changes to Juniper SRX

Convert event.risk values to float
Protect against missing event.timezone
Convert event.severity to long.

elastic/integrations#443

* Sync changes to Suricata EVE

Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457

* Sync changes to Zeek DNS

Fix usages of ignore_failure with convert processor.
Make DNS transaction ID a string.

elastic/integrations#448

* Add changelog

(cherry picked from commit bf46572)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants