Skip to content

[HA Proxy] Fix incorrect mapping of source.address field in stat datastream#7479

Merged
agithomas merged 6 commits intoelastic:mainfrom
agithomas:issue-7440
Oct 11, 2023
Merged

[HA Proxy] Fix incorrect mapping of source.address field in stat datastream#7479
agithomas merged 6 commits intoelastic:mainfrom
agithomas:issue-7440

Conversation

@agithomas
Copy link
Copy Markdown
Contributor

@agithomas agithomas commented Aug 21, 2023 •
edited
Loading

  • Bug

What does this PR do?

source.address field is mapped as text. It must be updated to keyword type.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

  • elastic-package build
  • elastic-package stack up -v -d --services package-registry
  • Dashboard verification
  • Index verification

Related issues

Priority : High

This PR is a blocker for TSDB enablement. So, prioritising it to high

@agithomas
Copy link
Copy Markdown
Contributor Author

agithomas commented Aug 21, 2023

Dashboard screenshots after package update

image

image
image
image

@agithomas agithomas marked this pull request as ready for review August 21, 2023 16:09
@agithomas agithomas requested a review from a team as a code owner August 21, 2023 16:09
@agithomas agithomas mentioned this pull request Aug 21, 2023
Closed
20 tasks
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Aug 21, 2023 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-10-10T06:01:07.320+0000

  • Duration: 15 min 41 sec

Test stats 🧪

Test Results
Failed 0
Passed 40
Skipped 0
Total 40

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Aug 21, 2023 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (1/1) 💚 3.313
Classes 100.0% (1/1) 💚 3.313
Methods 90.476% (19/21) 👎 -1.885
Lines 97.312% (181/186) 👍 8.898
Conditionals 100.0% (0/0) 💚

lalit-satapathy
lalit-satapathy reviewed Aug 22, 2023
View reviewed changes
@agithomas agithomas changed the title Fixed source.address type to keyword [HA Proxy] Fixed source.address type to keyword Aug 22, 2023
lalit-satapathy
lalit-satapathy reviewed Aug 22, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@lalit-satapathy lalit-satapathy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, since the old field mapping was not done correctly. Should this be a breaking change? IMO it should not be. A correction of a wrongly mapped field should be not considered breaking.

@ishleenk17 for any comments. Should we capture this anywhere apart from the change log?

@agithomas agithomas changed the title [HA Proxy] Fixed source.address type to keyword [HA Proxy] Fix incorrect mapping of source.address type in stat datastream Aug 22, 2023
@agithomas agithomas changed the title [HA Proxy] Fix incorrect mapping of source.address type in stat datastream [HA Proxy] Fix incorrect mapping of source.address field in stat datastream Aug 22, 2023
@agithomas
Copy link
Copy Markdown
Contributor Author

agithomas commented Aug 23, 2023

@ishleenk17 for any comments. Should we capture this anywhere apart from the change log?

@ishleenk17 , do you have any concerns?

@agithomas agithomas requested a review from ishleenk17 August 24, 2023 09:03
@agithomas
Copy link
Copy Markdown
Contributor Author

agithomas commented Aug 30, 2023

@ishleenk17 , do you have any concerns?

@ishleenk17 , can you please take a quick look to unblock this PR?

ishleenk17
ishleenk17 reviewed Aug 30, 2023
View reviewed changes
@agithomas
Copy link
Copy Markdown
Contributor Author

agithomas commented Sep 25, 2023

@ishleenk17 , @lalit-satapathy the Readme section is updated to have the troubleshooting section. Can you please review and approve if all looks good?

ishleenk17
ishleenk17 reviewed Sep 26, 2023
View reviewed changes
packages/haproxy/docs/README.md

## Troubleshooting

If `source.address` is shown conflicted under ``metrics-*`` data view, then this issue can be solved by [reindexing](https://www.elastic.co/guide/en/elasticsearch/reference/current/use-a-data-stream.html#reindex-with-a-data-stream) the `stat` data stream indices.
Copy link
Copy Markdown
Member

@ishleenk17 ishleenk17 Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we not mentioning the reindexing steps in the case of TSDS as well?
This seems to be only the normal reindexing.

Copy link
Copy Markdown
Contributor Author

@agithomas agithomas Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the HAProxy is not TSDB enabled. So, no need to have the TSDB related re-indexing to be added presently.

Once the testing of TSDB based re-indexing steps are validated, this section (along with similar sections in other packages), will be modified to include the section for reindexing TSDB enablement. It will be a separate issue , will be tracked separately.

Copy link
Copy Markdown
Member

@ishleenk17 ishleenk17 Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fine then

Copy link
Copy Markdown
Contributor Author

@agithomas agithomas Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created a backlog issue for tracking

ishleenk17
ishleenk17 approved these changes Sep 26, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@ishleenk17 ishleenk17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@agithomas
Copy link
Copy Markdown
Contributor Author

agithomas commented Sep 26, 2023

@lalit-satapathy , can i have your approval as well ?

lalit-satapathy
lalit-satapathy approved these changes Oct 10, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@lalit-satapathy lalit-satapathy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change, but a bug fix to correctly map the field type and also required for tsdb enablement.

@agithomas agithomas merged commit 67126ad into elastic:main Oct 11, 2023
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 11, 2023

Package haproxy - 1.8.4 containing this change is available at https://epr.elastic.co/search?package=haproxy

@agithomas agithomas deleted the issue-7440 branch February 12, 2025 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ishleenk17 ishleenk17 ishleenk17 approved these changes

@lalit-satapathy lalit-satapathy lalit-satapathy approved these changes

Assignees

No one assigned

Labels

Integration:haproxy HAProxy

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[haproxy] Correct the mapping of source.address to keyword type instead of text

5 participants

@agithomas @elasticmachine @lalit-satapathy @ishleenk17 @andrewkroh