[system] Adds `tags.yml` file so they appear under the Security Solution UI and upgrades package spec to version 3.0.0 by marc-gr · Pull Request #8206 · elastic/integrations

marc-gr · 2023-10-16T08:40:57Z

Proposed commit message

Adds tags.yml file so they appear under the Security Solution UI and upgrades package spec to version 3.0.0. Required changes include:

Removes dangling tags in dashboards.
Adds validation skip file for dashboard filters and search references.
Change env object type to flattened.
Remove yaml objects dotted notation.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Related issues

Relates Add asset tags to security integrations #7483

elasticmachine · 2023-10-16T08:47:26Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

elasticmachine · 2023-10-16T08:57:05Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-10-18T06:03:25.964+0000
Duration: 16 min 12 sec

Test stats 🧪

Test	Results
Failed	0
Passed	151
Skipped	0
Total	151

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-10-16T08:57:17Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`3/3`)	💚
Files	100.0% (`4/4`)	💚 3.888
Classes	100.0% (`4/4`)	💚 3.888
Methods	63.415% (`52/82`)	👎 -27.939
Lines	99.863% (`2924/2928`)	👍 13.167
Conditionals	100.0% (`0/0`)	💚

marc-gr · 2023-10-16T09:11:53Z

/test

elasticmachine · 2023-10-16T12:55:29Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

marc-gr · 2023-10-16T13:08:05Z

@jsoriano @mrodm It seems to me the errors we got loading test case failed: reading config for test case failed (testCasePath: /var/lib/jenkins/workspace/est-manager_integrations_PR-8206/src/github.com/elastic/integrations/packages/system/data_stream/auth/_dev/test/pipeline/test-auth-ubuntu1204.log): can't unpack test configuration: /var/lib/jenkins/workspace/est-manager_integrations_PR-8206/src/github.com/elastic/integrations/packages/system/data_stream/auth/_dev/test/pipeline/test-auth-ubuntu1204.log-config.yml: can not convert 'object' into 'string' accessing 'dynamic_fields.event' (source:'/var/lib/jenkins/workspace/est-manager_integrations_PR-8206/src/github.com/elastic/integrations/packages/system/data_stream/auth/_dev/test/pipeline/test-auth-ubuntu1204.log-config.yml') that got fixed in b606d36 should not fail in the first place? Is this intended?

jsoriano · 2023-10-16T13:30:07Z

@marc-gr this is likely a side effect of elastic/elastic-package#1485. Try running elastic-package format on one of the affected packages, and you will see some changes in test files.

I am currently looking into making this automation a bit smarter. In the meantime you can workaround the issue by quoting the keys that the automation is converting to objects.

drewdaemon

Great work!

efd6 · 2023-10-17T23:40:50Z

packages/system/data_stream/auth/_dev/test/pipeline/test-auth-ubuntu1204.log-config.yml

 dynamic_fields:
-  event:
-    ingested: "^.*$"
+  event.ingested: "^.*$"


This is funny. I take it that this is needed because it's essentially treated as a literal key rather than a path. Is that correct?

[a few moments later]

I see this is correct in the next commit.

ishleenk17 · 2023-10-19T09:53:39Z

@marc-gr : The upgrade of system package to 3.0.0 is being done as part of this effort.
I am not sure if these two are overlap ?

ishleenk17 · 2023-10-19T10:04:06Z

@marc-gr : The upgrade of system package to 3.0.0 is being done as part of this effort. I am not sure if these two are overlap ?

Checked with infraobs team. Looks like this has some security related changes as well.
So we can take this change for system package

shmsr · 2023-10-19T10:09:39Z

packages/system/data_stream/auth/_dev/test/pipeline/test-timestamp.log-config.yml

+  event:
+    timezone: "+0000"


Suggested change

event:

timezone: "+0000"

event:

timezone: "+0000"

@jsoriano If I am not wrong this also needs to be quoted ("event.timezone"), right?

PR with similar changes: #8174

It seems to only fail for dynamic_fields

We have reverted the change in elastic-package that was requiring all these changes in test config files. So you can also revert all these changes, they should not be needed anymore.

elasticmachine · 2023-10-19T12:13:32Z

Package system - 1.45.0 containing this change is available at https://epr.elastic.co/search?package=system

marc-gr added Team:Security-External Integrations Integration:system System labels Oct 16, 2023

Upgrade to package spec 3.0.0

ff767db

marc-gr force-pushed the system-package-spec-3 branch from db2d4f5 to ff767db Compare October 16, 2023 08:47

marc-gr marked this pull request as ready for review October 16, 2023 08:47

marc-gr requested review from a team as code owners October 16, 2023 08:47

marc-gr requested review from belimawr and ycombinator October 16, 2023 08:47

marc-gr changed the title ~~[system] Upgrade to package spec 3.0.0~~ [system] Adds tags.yml file so they appear under the Security Solution UI and upgrades package spec to version 3.0.0 Oct 16, 2023

pierrehilbert added the Team:Elastic-Agent Platform - Ingest - Agent [elastic/elastic-agent] label Oct 16, 2023

Fix tests

b606d36

Format test config files

c1076d9

drewdaemon approved these changes Oct 17, 2023

View reviewed changes

cmacknz approved these changes Oct 17, 2023

View reviewed changes

efd6 approved these changes Oct 17, 2023

View reviewed changes

Merge remote-tracking branch 'upstream/main' into system-package-spec-3

af363f8

ishleenk17 approved these changes Oct 19, 2023

View reviewed changes

shmsr mentioned this pull request Oct 19, 2023

Migrate obs-infraobs-integrations to package-spec v3 #3 #8203

Merged

22 tasks

shmsr reviewed Oct 19, 2023

View reviewed changes

marc-gr merged commit 22b5c72 into elastic:main Oct 19, 2023

marc-gr deleted the system-package-spec-3 branch October 19, 2023 11:50

Conversation

marc-gr commented Oct 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Related issues

Uh oh!

elasticmachine commented Oct 16, 2023

Uh oh!

elasticmachine commented Oct 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented Oct 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

marc-gr commented Oct 16, 2023

Uh oh!

elasticmachine commented Oct 16, 2023

Uh oh!

marc-gr commented Oct 16, 2023

Uh oh!

jsoriano commented Oct 16, 2023

Uh oh!

drewdaemon left a comment

Choose a reason for hiding this comment

Uh oh!

efd6 Oct 17, 2023

Choose a reason for hiding this comment

Uh oh!

ishleenk17 commented Oct 19, 2023

Uh oh!

ishleenk17 commented Oct 19, 2023

Uh oh!

shmsr Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marc-gr Oct 19, 2023

Choose a reason for hiding this comment

Uh oh!

jsoriano Oct 19, 2023

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Oct 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

marc-gr commented Oct 16, 2023 •

edited

Loading

elasticmachine commented Oct 16, 2023 •

edited

Loading

elasticmachine commented Oct 16, 2023 •

edited

Loading

shmsr Oct 19, 2023 •

edited

Loading