Skip to content

[Metricbeat] [Prometheus] Add proper error handling for query dataset when the auth/ssl is misconfigured#44493

Merged
mykola-elastic merged 5 commits intoelastic:mainfrom
mykola-elastic:metricbeat-prometheus-error-handling
May 28, 2025
Merged

[Metricbeat] [Prometheus] Add proper error handling for query dataset when the auth/ssl is misconfigured#44493
mykola-elastic merged 5 commits intoelastic:mainfrom
mykola-elastic:metricbeat-prometheus-error-handling

Conversation

@mykola-elastic
Copy link
Contributor

@mykola-elastic mykola-elastic commented May 26, 2025

Proposed commit message

[Metricbeat] [Prometheus] Query dataset: make understandable error messages in case for http error codes

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

No impact

Author's Checklist

  • [ ]

How to test this PR locally

My test environment:

  • Metricbeat built locally
  • Prometheus running in docker, configured to listen on HTTPS with local self-signed certs
  • Cloud deployment of Elastic stack

Related issues

Screenshots & Logs

When running Metricbeat with wrong Prometheus' username and password the error message looks like that now:

Screenshot 2025-05-26 at 15 58 40

and the debug message to the logs

{"log.level":"debug","@timestamp":"2025-05-26T16:07:31.937+0300","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":101},"message":"error received from prometheus endpoint: Unauthorized\n","service.name":"metricbeat","ecs.version":"1.6.0"}

When the protocol is HTTP but the Prometheus is configured to use HTTPS:

Screenshot 2025-05-26 at 16 01 44

and the debug message to the logs:

{"log.level":"debug","@timestamp":"2025-05-26T16:06:09.560+0300","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":101},"message":"error received from prometheus endpoint: Client sent an HTTP request to an HTTPS server.\n","service.name":"metricbeat","ecs.version":"1.6.0"}

Which is now the same behavior as of collector submodule

@mykola-elastic mykola-elastic added enhancement Metricbeat Metricbeat Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team prometheus labels May 26, 2025
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels May 26, 2025
@github-actions
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@mykola-elastic mykola-elastic self-assigned this May 26, 2025
@mergify
Copy link
Contributor

mergify bot commented May 26, 2025

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @mykola-elastic? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

@mykola-elastic mykola-elastic marked this pull request as ready for review May 26, 2025 13:42
@mykola-elastic mykola-elastic requested a review from a team as a code owner May 26, 2025 13:42
@mykola-elastic mykola-elastic added backport-active-8 Automated backport with mergify to all the active 8.[0-9]+ branches backport-active-9 Automated backport with mergify to all the active 9.[0-9]+ branches labels May 27, 2025
@mykola-elastic mykola-elastic requested a review from ishleenk17 May 27, 2025 08:04
@mykola-elastic mykola-elastic requested a review from ishleenk17 May 28, 2025 07:10
@mykola-elastic mykola-elastic force-pushed the metricbeat-prometheus-error-handling branch from 61c6bf1 to be99757 Compare May 28, 2025 12:11
@mykola-elastic mykola-elastic merged commit ba454d5 into elastic:main May 28, 2025
58 checks passed
@github-actions
Copy link
Contributor

@Mergifyio backport 8.17 8.18 8.19 9.0

mergify bot pushed a commit that referenced this pull request May 28, 2025
… when the auth/ssl is misconfigured (#44493)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)
mergify bot pushed a commit that referenced this pull request May 28, 2025
… when the auth/ssl is misconfigured (#44493)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)
mergify bot pushed a commit that referenced this pull request May 28, 2025
… when the auth/ssl is misconfigured (#44493)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)
mergify bot pushed a commit that referenced this pull request May 28, 2025
… when the auth/ssl is misconfigured (#44493)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)
mykola-elastic added a commit that referenced this pull request May 29, 2025
… when the auth/ssl is misconfigured (#44493) (#44532)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)

Co-authored-by: mykola-elastic <mykola.kmet@elastic.co>
mykola-elastic added a commit that referenced this pull request May 29, 2025
… when the auth/ssl is misconfigured (#44493) (#44531)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)

Co-authored-by: mykola-elastic <mykola.kmet@elastic.co>
mykola-elastic added a commit that referenced this pull request May 29, 2025
… when the auth/ssl is misconfigured (#44493) (#44530)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)

Co-authored-by: mykola-elastic <mykola.kmet@elastic.co>
mykola-elastic added a commit that referenced this pull request May 29, 2025
… when the auth/ssl is misconfigured (#44493) (#44529)

* [prometheus] query.go: check error response codes

* [prometheus] query.go: log error response body to debug log instead of repoter

* add entry to the changelog

* add unit test

* specify which endpoint failed

(cherry picked from commit ba454d5)

Co-authored-by: mykola-elastic <mykola.kmet@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-active-8 Automated backport with mergify to all the active 8.[0-9]+ branches backport-active-9 Automated backport with mergify to all the active 9.[0-9]+ branches enhancement Metricbeat Metricbeat prometheus Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Metricbeat] [Prometheus] Add proper error handling for query dataset when the auth/ssl is misconfigured

3 participants