Skip to content

[O11y][Azure Metrics] Migrate Queue Storage Overview dashboard to lens#7662

Merged
kush-elastic merged 4 commits intoelastic:mainfrom
kush-elastic:7434-Queue-Storage-Overview-lens-migration
Sep 12, 2023
Merged

[O11y][Azure Metrics] Migrate Queue Storage Overview dashboard to lens#7662
kush-elastic merged 4 commits intoelastic:mainfrom
kush-elastic:7434-Queue-Storage-Overview-lens-migration

Conversation

@kush-elastic
Copy link
Copy Markdown
Contributor

@kush-elastic kush-elastic commented Sep 5, 2023 •
edited
Loading

Urgency

  • High

Activity Type

  • Enhancement

What does this PR do?

  • Manually migrate Azure Queue Storage visualizations to the lens in the current Kibana version 8.9.0 itself.

  • Statistics for Azure Queue Storage Lens migration:

Migration stats

  Before Migration     After Migration    
  Lens Visualization Old Input Controls Lens Visualization New control panels
[Azure Metrics] Queue Storage Overview 0 9 1 9 0 1

Checklist

  • I have added an entry to my package's changelog.yml file.
  • I have verified that panels are populated with data.
  • I have verified that panels are not distorted after being migrated to the lens.
  • I have updated screenshots of the dashboard.
  • I have verified that the data counts are matching and panel aggregations are the same as before.

Author's Checklist

  • Migrated panels should be removed from visualization folder.
  • Migrated visualizations are populating in current Kibana version 8.4.0 itself.

Related issues

Before Migration
image

After Migration
image

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Sep 5, 2023 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-09-12T13:03:20.438+0000

  • Duration: 15 min 5 sec

Test stats 🧪

Test Results
Failed 0
Passed 34
Skipped 0
Total 34

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Sep 5, 2023 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (4/4) 💚
Files 100.0% (4/4) 💚
Classes 100.0% (4/4) 💚
Methods 75.758% (25/33) 👍 9.091
Lines 100.0% (32/32) 💚
Conditionals 100.0% (0/0) 💚

@kush-elastic kush-elastic marked this pull request as ready for review September 5, 2023 12:36
@kush-elastic kush-elastic requested a review from a team as a code owner September 5, 2023 12:36
@kush-elastic kush-elastic requested a review from a team September 5, 2023 12:37
@kush-elastic
Merge branch 'main' of https://github.com/kush-elastic/integrations i…
c8d7e07 
…nto 7434-Queue-Storage-Overview-lens-migration
zmoog
zmoog reviewed Sep 8, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@zmoog zmoog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The queue and message count values have decimals:

CleanShot 2023-09-08 at 12 32 30@2x

Isn't this the latest value for this metric?

zmoog
zmoog requested changes Sep 8, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@zmoog zmoog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't these be integer values?

@kush-elastic
Copy link
Copy Markdown
Contributor Author

kush-elastic commented Sep 8, 2023

@zmoog
Both the visualizations were already using Average aggregation before migration, we will need to create new tickets to address this issue. As part of this PR scope, our focus will be only on migration.

Screenshot 2023-09-08 at 4 34 15 PM Screenshot 2023-09-08 at 4 36 30 PM

@zmoog
Copy link
Copy Markdown
Contributor

zmoog commented Sep 8, 2023

The old dashboard version displays the count values as integers; how does it work?

@kush-elastic
Copy link
Copy Markdown
Contributor Author

kush-elastic commented Sep 8, 2023

The old dashboard version displays the count values as integers; how does it work?

it's working there because there is a way we can specify the custom format in TSDB.

Screenshot 2023-09-08 at 5 07 48 PM

@zmoog
Copy link
Copy Markdown
Contributor

zmoog commented Sep 8, 2023

it's working there because there is a way we can specify the custom format in TSDB.

So, the old version displays the average value but uses a custom format to show it as an integer, right?

@kush-elastic
Copy link
Copy Markdown
Contributor Author

kush-elastic commented Sep 8, 2023

So, the old version displays the average value but uses a custom format to show it as an integer, right?

Yes

@zmoog
Copy link
Copy Markdown
Contributor

zmoog commented Sep 8, 2023

Displaying single count values with decimals doesn't feel right, but I'm not an expert.

@drewdaemon, can you or someone in your team help us understand the best way to represent this kind of data? It could be just me.

@drewdaemon
Copy link
Copy Markdown
Contributor

drewdaemon commented Sep 8, 2023

Custom formatter was added to Lens in 8.9. Does that cover this?

Screen.Recording.2023-09-08.at.7.45.37.AM.mov

@kush-elastic
Copy link
Copy Markdown
Contributor Author

kush-elastic commented Sep 11, 2023

Custom formatter was added to Lens in 8.9. Does that cover this?

Hey @zmoog @drewdaemon,
We are not able to export dashboards from 8.9.0 using the latest elastic-pacakge.
Error

@kush-elastic kush-elastic self-assigned this Sep 11, 2023
@drewdaemon
Copy link
Copy Markdown
Contributor

drewdaemon commented Sep 11, 2023

@kush-elastic good point. I had forgotten about elastic/kibana#161969.

The public release for 8.10 is tomorrow, so you could always wait a day or two. Up to @zmoog of course.

@kush-elastic
Copy link
Copy Markdown
Contributor Author

kush-elastic commented Sep 12, 2023

The public release for 8.10 is tomorrow, so you could always wait a day or two. Up to @zmoog of course.

@zmoog, what do you think? how should we proceed with this?
what I think is the issue itself is with the aggregation used in the current visualization. if we use the last value it should work just fine without any formatting. I don't see any reason to move dashboards to 8.10 just because of export issues.

We can merge this and create a new issue to track and resolve the wrong aggregation issue. that way we can stay on the current version and target more users at the same time.

@zmoog
Copy link
Copy Markdown
Contributor

zmoog commented Sep 12, 2023

We can merge this and create a new issue to track and resolve the wrong aggregation issue. that way we can stay on the current version and target more users at the same time.

Okay, sounds good.

This was referenced Sep 12, 2023
Merged
Merged
rajvi-patel-22
rajvi-patel-22 approved these changes Sep 12, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@rajvi-patel-22 rajvi-patel-22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kush-elastic kush-elastic merged commit 888f0df into elastic:main Sep 12, 2023
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Sep 12, 2023

Package azure_metrics - 1.0.41 containing this change is available at https://epr.elastic.co/search?package=azure_metrics

@kush-elastic kush-elastic mentioned this pull request Sep 13, 2023
Closed
8 tasks
@andrewkroh andrewkroh added the Integration:azure_metrics Azure Resource Metrics label Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zmoog zmoog zmoog approved these changes

+1 more reviewer

@rajvi-patel-22 rajvi-patel-22 rajvi-patel-22 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@kush-elastic kush-elastic

Labels

Integration:azure_metrics Azure Resource Metrics

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@kush-elastic @elasticmachine @zmoog @drewdaemon @rajvi-patel-22 @andrewkroh