Skip to content

Fix tls mapping in suricata module#19494

Merged
leehinman merged 4 commits intoelastic:masterfrom
leehinman:leh_suricata_sdh
Jul 2, 2020
Merged

Fix tls mapping in suricata module#19494
leehinman merged 4 commits intoelastic:masterfrom
leehinman:leh_suricata_sdh

Conversation

@leehinman
Copy link
Copy Markdown
Contributor

@leehinman leehinman commented Jun 29, 2020

What does this PR do?

Fixes tls mappings in suricata module. Specifically:

  • add suricata.eve.tls.ja3s.string field
  • add suricata.eve.tls.ja3s.hash field
  • add suricata.eve.tls.ja3.string field
  • add suricata.eve.tls.ja3.hash field
  • set default_field to false for suricata fields
  • map suricata.eve.tls.ja3.hash to tls.client.ja3
  • map suricata.eve.tls.ja3s.hash to tls.server.ja3s
  • perform suricata.eve.tls.* -> tls.* mappings for all event types

Why is it important?

  • If the tls.* mappings aren't filled in the event doesn't show up in
    the TLS tab in the SIEM
  • default_field to false to stay under 1000 fields

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=suricata mage -v pythonIntegTest

Related issues

@leehinman leehinman added bug Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM labels Jun 29, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 29, 2020

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 29, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 29, 2020 •
edited
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #19494 updated]

  • Start Time: 2020-07-02T13:37:32.333+0000

  • Duration: 27 min 53 sec

Test stats 🧪

Test Results
Failed 0
Passed 555
Skipped 128
Total 683

andrewkroh
andrewkroh approved these changes Jun 29, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I requested some changes but don't think it requires another review from me.

x-pack/filebeat/module/suricata/eve/config/eve.yml Outdated
Copy link
Copy Markdown
Member

@andrewkroh andrewkroh Jun 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ignore_failure: true
fail_on_error: false

x-pack/filebeat/module/suricata/eve/_meta/fields.yml Outdated
Copy link
Copy Markdown
Member

@andrewkroh andrewkroh Jun 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid changing the behavior for the existing fields can you mark this on the two new field groups instead.

@leehinman leehinman force-pushed the leh_suricata_sdh branch 2 times, most recently from 6f27ebc to 00956eb Compare June 30, 2020 19:49
leehinman added 3 commits July 1, 2020 09:34
@leehinman
Fix tls mapping in suricata module
5ec28ba 
- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for suricata fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492
@leehinman
incorporate feedback
2eb76ac
@leehinman
Fix tests
ff025dc
@leehinman
Copy link
Copy Markdown
Contributor Author

leehinman commented Jul 1, 2020

run tests

@leehinman leehinman merged commit afffe2b into elastic:master Jul 2, 2020
@leehinman leehinman mentioned this pull request Jul 2, 2020
Merged
4 tasks
@leehinman leehinman added v7.7.2 and removed needs_backport PR is waiting to be backported to other branches. labels Jul 2, 2020
@leehinman leehinman mentioned this pull request Jul 2, 2020
Merged
4 tasks
@leehinman leehinman mentioned this pull request Jul 2, 2020
Merged
4 tasks
leehinman added a commit to leehinman/beats that referenced this pull request Jul 2, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
3fb028d 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit afffe2b)
leehinman added a commit that referenced this pull request Jul 2, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (#19494) (#19609)
0f0fe45 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes #19492

(cherry picked from commit afffe2b)
leehinman added a commit to leehinman/beats that referenced this pull request Jul 2, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
ef7ebb9 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit afffe2b)
v1v added a commit to v1v/beats that referenced this pull request Jul 3, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/windows-o…
0ba6f3b 
…ne-beats

* upstream/master: (35 commits)
  [ci] fix env variable name for xpack filebeats (elastic#19617)
  Cache error responses for cloudfoundry apps metadata (elastic#19181)
  ci: user fixed type of agent (elastic#19625)
  Input v2 cursor testing (elastic#19573)
  Update Jenkinsfile to not inspect removed vendor (elastic#19610)
  Fix ordering and duplicate configs on autodiscover (elastic#19317)
  Prepare input/file for changes in the registrar (elastic#19516)
  Cursor input and manager implementation (elastic#19571)
  [Filebeat] Fix tls mapping in suricata module (elastic#19494)
  [Ingest Manager] Make Agent beta and Constraints experimental (elastic#19586)
  Accept prefix as metric_types for stackdriver metricset in GCP (elastic#19345)
  Implement memlog store operations (elastic#19533)
  introduce journalbeat/pkg in order to provide reusable shared code (elastic#19581)
  Add descriptions to HAProxy fields in Metricbeat (elastic#19561)
  ci: apm-server-update trigered only on upstream, comments, and manual triggered (elastic#19590)
  ci: enable upstream triggering on the packaging job (elastic#19589)
  ci: some jjbb improvements (elastic#19588)
  [MetricBeat] set tags correctly if the dimension value is ARN (elastic#19433)
  [Filebeat] Add default_fields: false to fields.yml in aws module (elastic#19568)
  Add publisher implementation for stateful inputs (elastic#19530)
  ...
leehinman added a commit to leehinman/beats that referenced this pull request Jul 6, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
bfa2bb3 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit afffe2b)
leehinman added a commit that referenced this pull request Jul 6, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (#19494) (#19608)
0f2144a 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes #19492

(cherry picked from commit afffe2b)
leehinman added a commit that referenced this pull request Jul 6, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (#19494) (#19607)
4f4baef 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes #19492

(cherry picked from commit afffe2b)
@leehinman leehinman deleted the leh_suricata_sdh branch October 5, 2020 19:21
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
@leehinman @melchiormoulin
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
89eea04 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494) (elasti…
a4fb441 
…c#19607)

* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit 362016d)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494) (elasti…
da3007a 
…c#19608)

* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit 362016d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

No one assigned

Labels

bug Filebeat Filebeat v7.7.2 v7.8.1 v7.9.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Filebeat] suricata fileset doesn't capture tls fields for alerts

3 participants

@leehinman @elasticmachine @andrewkroh