Skip to content

Possible regression in field values validation #1439

@ebeahan

Description

@ebeahan

Summary

In elastic-package v0.84.0+, elastic-package test pipeline -v no longer produces errors due if ECS field value fails validation.

The same package and version (in the below example, cisco_asa version 2.21.0) tested using v0.83.2 does produce validation errors.

Comparison

v0.83.2


$ go install github.com/elastic/elastic-package@v0.83.2

$ elastic-package version
2023/09/11 12:14:48  INFO New version is available - v0.86.1. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.86.1
elastic-package v0.83.2 version-hash undefined (build time: unknown)

$ elastic-package test pipeline -v
2023/09/11 12:14:58  WARN CommitHash is undefined, in both /Users/ericbeahan/.elastic-package/version and the compiled binary, config may be out of date.
2023/09/11 12:14:58 DEBUG Enable verbose logging
2023/09/11 12:14:58  INFO New version is available - v0.86.1. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.86.1
Run pipeline tests for the package
--- Test results for package: cisco_asa - START ---
FAILURE DETAILS:
cisco_asa/log test-additional-messages.log:
[0] parsing field value failed: field "event.type" value "change" is not one of the expected values (access, allowed, connection, denied, end, info, protocol, start) for any of the values of "event.category" (network)
[1] parsing field value failed: field "event.type" value "deletion" is not one of the expected values (access, allowed, connection, denied, end, info, protocol, start) for any of the values of "event.category" (network)
[2] parsing field value failed: field "event.type" value "error" is not one of the expected values (access, allowed, connection, denied, end, info, protocol, start) for any of the values of "event.category" (network)
cisco_asa/log test-anyconnect-messages.log:
[0] parsing field value failed: field "event.type" value "error" is not one of the expected values (access, allowed, connection, denied, end, info, protocol, start) for any of the values of "event.category" (network)
cisco_asa/log test-sample.log:
[0] parsing field value failed: field "event.type" value "deletion" is not one of the expected values (access, allowed, connection, denied, end, info, protocol, start) for any of the values of "event.category" (network)


╭───────────┬─────────────┬───────────┬──────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┬──────────────╮
│ PACKAGE   │ DATA STREAM │ TEST TYPE │ TEST NAME                    │ RESULT                                                                      │ TIME ELAPSED │
├───────────┼─────────────┼───────────┼──────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┼──────────────┤
│ cisco_asa │ log         │ pipeline  │ test-additional-messages.log │ FAIL: test case failed: one or more problems with fields found in documents │     22.607ms │
│ cisco_asa │ log         │ pipeline  │ test-anyconnect-messages.log │ FAIL: test case failed: one or more problems with fields found in documents │   5.322584ms │
│ cisco_asa │ log         │ pipeline  │ test-asa-fix.log             │ PASS                                                                        │  17.565792ms │
│ cisco_asa │ log         │ pipeline  │ test-asa-missing-groups.log  │ PASS                                                                        │  10.212708ms │
│ cisco_asa │ log         │ pipeline  │ test-asa.log                 │ PASS                                                                        │  89.401125ms │
│ cisco_asa │ log         │ pipeline  │ test-dap-records.log         │ PASS                                                                        │   6.171625ms │
│ cisco_asa │ log         │ pipeline  │ test-filtered.log            │ PASS                                                                        │   4.049042ms │
│ cisco_asa │ log         │ pipeline  │ test-hostnames.log           │ PASS                                                                        │     7.7675ms │
│ cisco_asa │ log         │ pipeline  │ test-non-canonical.log       │ PASS                                                                        │  19.610458ms │
│ cisco_asa │ log         │ pipeline  │ test-not-ip.log              │ PASS                                                                        │   7.265417ms │
│ cisco_asa │ log         │ pipeline  │ test-sample.log              │ FAIL: test case failed: one or more problems with fields found in documents │     43.255ms │
│ cisco_asa │ log         │ pipeline  │ test-sip.log                 │ PASS                                                                        │  10.267375ms │
╰───────────┴─────────────┴───────────┴──────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┴──────────────╯
--- Test results for package: cisco_asa - END   ---
Done
Error: one or more test cases failed

v0.84.0+


$ go install github.com/elastic/elastic-package@v0.84.0

$ elastic-package version
2023/09/11 12:15:25  WARN CommitHash is undefined, in both /Users/ericbeahan/.elastic-package/version and the compiled binary, config may be out of date.
2023/09/11 12:15:25  INFO New version is available - v0.86.1. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.86.1
elastic-package v0.84.0 version-hash undefined (build time: unknown)

$ elastic-package test pipeline -v
2023/09/11 12:15:30  WARN CommitHash is undefined, in both /Users/ericbeahan/.elastic-package/version and the compiled binary, config may be out of date.
2023/09/11 12:15:30 DEBUG Enable verbose logging
2023/09/11 12:15:30  INFO New version is available - v0.86.1. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.86.1
Run pipeline tests for the package
--- Test results for package: cisco_asa - START ---
╭───────────┬─────────────┬───────────┬──────────────────────────────┬────────┬──────────────╮
│ PACKAGE   │ DATA STREAM │ TEST TYPE │ TEST NAME                    │ RESULT │ TIME ELAPSED │
├───────────┼─────────────┼───────────┼──────────────────────────────┼────────┼──────────────┤
│ cisco_asa │ log         │ pipeline  │ test-additional-messages.log │ PASS   │   26.69075ms │
│ cisco_asa │ log         │ pipeline  │ test-anyconnect-messages.log │ PASS   │  15.653833ms │
│ cisco_asa │ log         │ pipeline  │ test-asa-fix.log             │ PASS   │  13.698042ms │
│ cisco_asa │ log         │ pipeline  │ test-asa-missing-groups.log  │ PASS   │  12.466291ms │
│ cisco_asa │ log         │ pipeline  │ test-asa.log                 │ PASS   │  99.582625ms │
│ cisco_asa │ log         │ pipeline  │ test-dap-records.log         │ PASS   │   4.954625ms │
│ cisco_asa │ log         │ pipeline  │ test-filtered.log            │ PASS   │    4.13625ms │
│ cisco_asa │ log         │ pipeline  │ test-hostnames.log           │ PASS   │    4.68525ms │
│ cisco_asa │ log         │ pipeline  │ test-non-canonical.log       │ PASS   │  17.809375ms │
│ cisco_asa │ log         │ pipeline  │ test-not-ip.log              │ PASS   │   5.496333ms │
│ cisco_asa │ log         │ pipeline  │ test-sample.log              │ PASS   │     60.208ms │
│ cisco_asa │ log         │ pipeline  │ test-sip.log                 │ PASS   │   3.503375ms │
╰───────────┴─────────────┴───────────┴──────────────────────────────┴────────┴──────────────╯
--- Test results for package: cisco_asa - END   ---
Done

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions