Automated tests need to be added to the security packages that we migrated over from Filebeat. This will prevent future regressions and save us from having to manually test. The tests should verify:

• Data can be collected through each supported input type.
• Fields contained in the final documents are defined in package.
• There are no data type conflicts with latest ECS release.

I think it would be ideal if all of this testing could be accomplished through system tests with the elastic-package tool. I'll work through setting up a test and see what additional features will be needed and work that via issues in the elastic-package repo.

Input Types

These are the different input types used in packages.

• logfile
• udp
• tcp
• tcp with tls
• aws-s3
• gcp-pubsub
• netflow
• httpjson
• o365audit
• windows event log (via .evtx file)

Data Streams to Test

This is every data stream and the inputs that they support.

• AWS
  • CloudTrail (pipeline test Add pipeline tests for AWS CloudTrail #408)
    • s3
  • VPC Flow
    • s3
• Auditd
  • Log
    • logfile (system & pipeline)
• CEF
  • Log (has pipeline test)
    • logfile (system test Add system tests for CEF #575)
    • syslog (system test Add system tests for CEF #575)
• CheckPoint
  • Firewall (pipeline test Add pipeline tests for CheckPoint Firewall #409)
    • logfile (system test Add logfile system test to CheckPoint Firewall #574)
    • udp (system test Add UDP and TCP Syslog tests for CheckPoint Firewall #553)
    • tcp (system test Add UDP and TCP Syslog tests for CheckPoint Firewall #553)
• Cisco
  • ASA
    • logfile Add pipeline tests to Cisco ASA #414
    • udp (system Add tests to Cisco package #578)
  • FTD
    • logfile (system Add tests to Cisco package #578)
    • udp (system Add tests to Cisco package #578)
  • IOS
    • logfile (system test Add system test for Cisco IOS #416)
    • syslog (system Add tests to Cisco package #578)
  • Meraki
    • logfile (system test Add system test for Cisco Meraki #428)
    • udp (system Add tests to Cisco package #578)
    • tcp (system Add tests to Cisco package #578)
  • Nexus
    • logfile (system Add tests to Cisco package #578)
    • udp (system Add tests to Cisco package #578)
    • tcp (system Add tests to Cisco package #578)
• CrowdStrike
  • Falcon
    • logfile (system test Add system test for CrowdStrike Falcon #429) (add it again Add system test to CrowdStrike Falcon #583)
• Fortinet
  • FortiClient Endpoint security
    • udp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • tcp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • logfile (system test Add system test for Fortinet client endpoint #432)
  • Firewall
    • udp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • tcp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • logfile (pipeline test Add pipeline test for Fortinet Firewall #437) (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
  • FortiMail
    • udp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • tcp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • logfile (system test Add system test for Fortinet Fortimail #438)
  • Manager/Analyzer
    • udp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • tcp (system Add system tests for Fortinet Firewall udp/tcp inputs #585)
    • logfile (system test Add system test for Fortinet Fortimanager #439)
• Google Workspace
  • admin
    • httpjson
    • logfile (system test ✅)
  • drive
    • httpjson
    • logfile (system test ✅)
  • groups
    • httpjson
    • logfile (system test ✅)
  • login
    • httpjson
    • logfile (system test ✅)
  • saml
    • httpjson
    • logfile (system test ✅)
  • user_accounts
    • httpjson
    • logfile (system test ✅)
• iptables
  • log (pipeline test ✅)
    • logfile (system test ✅)
    • syslog (udp) (system test ✅)
• Juniper
  • JUNOS (rsa2elk, no pipeline test))
    • udp (system test ✅ Add system tests to Juniper #588)
    • tcp (system test ✅ Add system tests to Juniper #588)
    • logfile (system test ✅ Add system tests to Juniper #588)
  • Netscreen (rsa2elk, no pipeline test)
    • udp (system test ✅ Add system tests to Juniper #588)
    • tcp (system test ✅ Add system tests to Juniper #588)
    • logfile (system test Add system test for Juniper Netscreen #444)
  • SRX (pipeline test Add pipeline test for Juniper SRX #443)
    • udp (system test ✅ Add system tests to Juniper #588)
    • tcp (system test ✅ Add system tests to Juniper #588)
    • logfile (system test ✅ Add system tests to Juniper #588)
• Microsoft
  • DHCP
    • logfile (system test Add system test for Microsoft DHCP #447)
• Netflow
  • Log
    • netflow (system tests ✅ Add system test for Netflow Log (udp) #590)
• O365
  • Audit
    • o365audit
    • logfile (system test Add system test for Office 365 audit #449)
• Okta
  • System
    • httpjson
    • logfile (system test Add system test for Okta System #452)
• PANW
  • PAN-OS
    • syslog (system test Add system test for panw pan-os (syslog input) #594)
    • logfile (system+pipeline Add system and pipeline test for Palo Alto PAN-OS #454)
• Suricata
  • EVE
    • logfile (system+pipeline Add system and pipeline tests for Suricata EVE #457)
• System
  • Auth
    • logfile (pipeliine Add pipeline test for System Auth #463)
• Windows
  • Forwarded
    • winlog
  • PowerShell
    • winlog
  • PowerShell Operational
    • winlog
  • Security
    • winlog
  • Sysmon
    • winlog
• Zeek (@leehinman)
  • logfile (systems test add zeek system tests #448)
• Zoom
  • Webhook (pipeline Add system and pipeline tests for Zoom #598)
    • http_endpoint (system Add system and pipeline tests for Zoom #598)