Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

prom-wrapper: introduce src-prometheus, migrate alerts history#17602

Merged
bobheadxi merged 13 commits into
mainfrom
prom-wrapper/extract-prometheus
Jan 28, 2021
Merged

prom-wrapper: introduce src-prometheus, migrate alerts history#17602
bobheadxi merged 13 commits into
mainfrom
prom-wrapper/extract-prometheus

Conversation

@bobheadxi

@bobheadxi bobheadxi commented Jan 25, 2021

Copy link
Copy Markdown
Member

Was doing a review of the prom-wrapper code and remembered this old code I wrote lying around for the frontend making a direct query to Prometheus, which powers the alerts overview in the bug reports page. There's a few very manual API requests to the prom-wrapper as well. Sadly it seems I wrote some pretty bad code here and figured it might be a good idea to get this cleaned up a bit. Then I got into it and updated all the frontend->prom-wrapper touch points (more than I recalled) into a new package, src-prometheus

This PR:

  • removes the frontend->prometheus direct querying, and lets the prom-wrapper handle it (it already provides aggregated overviews for alerts.hideObservabilitySiteAlerts)
  • introduces the src-prometheus package for the prom-wrapper API, and replaces all touch points with the prom-wrapper with usage of this API
  • leverage the src-prometheus package in the prom-wrapper

Post-this PR I would advocate to move prom-wrapper out of docker-images and into a top-level cmd/prom-wrapper since doing this I realized there's a lot more to prom-wrapper than just a reverse-proxy for alertmanager and prometheus

@bobheadxi bobheadxi requested review from a team and pecigonzalo January 25, 2021 09:05
@bobheadxi bobheadxi force-pushed the prom-wrapper/extract-prometheus branch 3 times, most recently from ef9a18f to 16b30d1 Compare January 25, 2021 10:25
@bobheadxi bobheadxi changed the title prom-wrapper: extract prometheus query away from frontend prom-wrapper: extract prometheus query out of frontend, introduce srcprometheus Jan 25, 2021
@bobheadxi bobheadxi changed the title prom-wrapper: extract prometheus query out of frontend, introduce srcprometheus prom-wrapper: extract prometheus query out of frontend, introduce src-prometheus Jan 25, 2021
@bobheadxi bobheadxi force-pushed the prom-wrapper/extract-prometheus branch from 16b30d1 to e74e594 Compare January 25, 2021 10:36
@sourcegraph-bot

sourcegraph-bot commented Jan 25, 2021

Copy link
Copy Markdown
Contributor

Notifying subscribers in CODENOTIFY files for diff 1751289...092b79f.

Notify File(s)
@christinaforney doc/dev/background-information/observability/prometheus.md

@codecov

codecov Bot commented Jan 25, 2021

Copy link
Copy Markdown

Codecov Report

Merging #17602 (092b79f) into main (1751289) will decrease coverage by 0.03%.
The diff coverage is 27.36%.

@@            Coverage Diff             @@
##             main   #17602      +/-   ##
==========================================
- Coverage   51.19%   51.15%   -0.04%     
==========================================
  Files        1721     1720       -1     
  Lines       86071    86105      +34     
  Branches     7754     7763       +9     
==========================================
- Hits        44062    44045      -17     
- Misses      38119    38166      +47     
- Partials     3890     3894       +4     
Flag Coverage Δ
go 49.87% <27.36%> (-0.07%) ⬇️
integration 30.72% <ø> (-0.02%) ⬇️
storybook 30.67% <ø> (+0.04%) ⬆️
typescript 54.26% <ø> (+0.01%) ⬆️
unit 34.59% <ø> (ø)
Impacted Files Coverage Δ
client/web/src/site-admin/backend.ts 4.61% <ø> (ø)
cmd/frontend/graphqlbackend/site_monitoring.go 0.00% <0.00%> (-68.66%) ⬇️
docker-images/prometheus/cmd/prom-wrapper/main.go 0.00% <0.00%> (ø)
...r-images/prometheus/cmd/prom-wrapper/siteconfig.go 0.98% <0.00%> (-0.01%) ⬇️
internal/src-prometheus/types.go 0.00% <0.00%> (ø)
internal/src-prometheus/prometheus.go 8.69% <8.69%> (ø)
cmd/frontend/graphqlbackend/site_alerts.go 33.55% <36.36%> (-0.82%) ⬇️
...ocker-images/prometheus/cmd/prom-wrapper/status.go 34.31% <48.57%> (+34.31%) ⬆️
cmd/frontend/internal/app/debug.go 31.03% <78.57%> (+0.73%) ⬆️
... and 6 more

@uwedeportivo

Copy link
Copy Markdown
Contributor

i'll look at this tomorrow morning

@uwedeportivo uwedeportivo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

avoiding the dash for the package name is unfortunate :-), but i cannot come up with a different name

// Notify admins if critical alerts are firing.
AlertFuncs = append(AlertFuncs, observabilityActiveAlertsAlert(prometheusutil.PrometheusURL))
// Notify admins if critical alerts are firing, if Prometheus is configured.
promURL, err := srcprometheus.PrometheusURL()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i found this slightly weird to read. could srcprometheus.PrometheusURL() instead return the empty string when prometheus is not configured ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried to improve the flow here a bit with https://github.com/sourcegraph/sourcegraph/pull/17602/commits/841ac9c70b36c654f4c89cfa6d724a4520d35f79, now usages should just directly create a client and the client creation will return the appropriate ErrPrometheusNotAvailable that can be checked!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, lgtm

@bobheadxi

bobheadxi commented Jan 28, 2021

Copy link
Copy Markdown
Member Author

Did a final sanity check + one last fix (61b5b71):

image

@bobheadxi bobheadxi changed the title prom-wrapper: extract prometheus query out of frontend, introduce src-prometheus prom-wrapper: introduce src-prometheus, migrate alerts history Jan 28, 2021
@bobheadxi bobheadxi merged commit 7c55384 into main Jan 28, 2021
@bobheadxi bobheadxi deleted the prom-wrapper/extract-prometheus branch January 28, 2021 11:52
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants