Skip to content

[System][Process] Add dimensions metadata; remove duplicated fields#6407

Merged
tetianakravchenko merged 3 commits intoelastic:mainfrom
tetianakravchenko:system-process-dimensions
Jun 1, 2023
Merged

[System][Process] Add dimensions metadata; remove duplicated fields#6407
tetianakravchenko merged 3 commits intoelastic:mainfrom
tetianakravchenko:system-process-dimensions

Conversation

@tetianakravchenko
Copy link
Copy Markdown
Contributor

@tetianakravchenko tetianakravchenko commented May 31, 2023

What does this PR do?

This is a follow up PR of #6118 (comment)
In this PR:

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@tetianakravchenko
remove duplicated fields for the process data_stream; add dimensions
8ae309b 
Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
@tetianakravchenko tetianakravchenko requested a review from a team as a code owner May 31, 2023 15:43
@tetianakravchenko
fix pr link
11ba4b5 
Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
@tetianakravchenko tetianakravchenko mentioned this pull request May 31, 2023
Closed
11 tasks
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented May 31, 2023 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-06-01T11:14:48.642+0000

  • Duration: 14 min 28 sec

Test stats 🧪

Test Results
Failed 0
Passed 145
Skipped 0
Total 145

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented May 31, 2023 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (3/3) 💚
Files 100.0% (4/4) 💚
Classes 100.0% (4/4) 💚
Methods 60.759% (48/79) 👎 -32.991
Lines 99.536% (2790/2803) 👎 -0.464
Conditionals 100.0% (0/0) 💚

@tetianakravchenko tetianakravchenko mentioned this pull request May 31, 2023
Merged
4 tasks
ruflin
ruflin reviewed Jun 1, 2023
View reviewed changes
packages/system/changelog.yml Outdated
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.30.0"
Copy link
Copy Markdown
Contributor

@ruflin ruflin Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this correct? It seems 1.30.0 just went out here: https://github.com/elastic/integrations/pull/6118/files#diff-eff09b294da7e0e80815a8d268a540c955e39011ff3ddfea22fa0651195bfaa6R2

Copy link
Copy Markdown
Contributor Author

@tetianakravchenko tetianakravchenko Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed - e9fdfca, after merging other PRs should be adjusted accordingly

ruflin
ruflin reviewed Jun 1, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM except the version number.

@tetianakravchenko
resolve conflicts
e9fdfca 
Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
ruflin
ruflin reviewed Jun 1, 2023
View reviewed changes
packages/system/data_stream/process/fields/agent.yml

Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.'
example: 666777888999
dimension: true
Copy link
Copy Markdown
Contributor

@ruflin ruflin Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tetianakravchenko The agent file we are adding here, is this shared across all integrations that have adopted dimensions?

Copy link
Copy Markdown
Contributor Author

@tetianakravchenko tetianakravchenko Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean if the content of this file is the same for all tsdb data_streams? no, there are less fields defined for cloud providers - #5193 (comment)

Copy link
Copy Markdown
Contributor

@ruflin ruflin Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. But the same file is copy / pasted (manually) for the non cloud ones.

Copy link
Copy Markdown
Contributor Author

@tetianakravchenko tetianakravchenko Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least for data_streams within the same package I believe it should be the same. From what I've noticed fields folder structure is different for different packages. In general there might be some differences, like for example - in diskio there are some specific to this data_stream fields - https://github.com/elastic/integrations/blob/main/packages/system/data_stream/diskio/fields/agent.yml#L195-L205

agithomas
agithomas approved these changes Jun 1, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@agithomas agithomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

I had my doubts on multi-threaded process PIDs. Tested and validated

@tetianakravchenko tetianakravchenko merged commit f1dd51a into elastic:main Jun 1, 2023
@tetianakravchenko tetianakravchenko deleted the system-process-dimensions branch June 1, 2023 13:09
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jun 1, 2023

Package system - 1.31.0 containing this change is available at https://epr.elastic.co/search?package=system

sodhikirti07 pushed a commit that referenced this pull request Jun 15, 2023
@tetianakravchenko @sodhikirti07
[System][Process] Add dimensions metadata; remove duplicated fields (#…
d214340 
…6407)

* remove duplicated fields for the process data_stream; add dimensions

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

* fix pr link

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

---------

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ruflin ruflin ruflin left review comments

@leehinman leehinman leehinman approved these changes

@agithomas agithomas agithomas approved these changes

@ycombinator ycombinator Awaiting requested review from ycombinator ycombinator was automatically assigned from elastic/elastic-agent-data-plane

Assignees

No one assigned

Labels

Integration:system System

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@tetianakravchenko @elasticmachine @ruflin @leehinman @agithomas @andrewkroh