Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

monitoring: 'flatten' observables (reimplement #11985)#17600

Merged
bobheadxi merged 8 commits into
mainfrom
monitoring/flatten-observables
Jan 27, 2021
Merged

monitoring: 'flatten' observables (reimplement #11985)#17600
bobheadxi merged 8 commits into
mainfrom
monitoring/flatten-observables

Conversation

@bobheadxi

@bobheadxi bobheadxi commented Jan 25, 2021

Copy link
Copy Markdown
Member

@bobheadxi bobheadxi requested review from a team and pecigonzalo January 25, 2021 06:10
@sourcegraph-bot

sourcegraph-bot commented Jan 25, 2021

Copy link
Copy Markdown
Contributor

Notifying subscribers in CODENOTIFY files for diff 1246425...8aa90f3.

Notify File(s)
@christinaforney doc/admin/observability/alert_solutions.md
doc/admin/observability/dashboards.md
@slimsag monitoring/definitions/prometheus.go
monitoring/monitoring/README.md
monitoring/monitoring/monitoring.go
@sourcegraph/distribution doc/admin/observability/alert_solutions.md
doc/admin/observability/dashboards.md
monitoring/definitions/prometheus.go
monitoring/monitoring/README.md
monitoring/monitoring/monitoring.go

@codecov

codecov Bot commented Jan 25, 2021

Copy link
Copy Markdown

Codecov Report

Merging #17600 (8aa90f3) into main (1246425) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main   #17600      +/-   ##
==========================================
- Coverage   51.58%   51.58%   -0.01%     
==========================================
  Files        1717     1717              
  Lines       85285    85285              
  Branches     7822     7761      -61     
==========================================
- Hits        43992    43991       -1     
- Misses      37419    37420       +1     
  Partials     3874     3874              
Flag Coverage Δ
go 50.47% <ø> (ø)
integration 30.72% <ø> (-0.03%) ⬇️
storybook 30.62% <ø> (ø)
typescript 54.24% <ø> (-0.01%) ⬇️
unit 34.59% <ø> (ø)
Impacted Files Coverage Δ
client/web/src/tree/TreeRoot.tsx 83.33% <0.00%> (-3.04%) ⬇️
client/web/src/nav/StatusMessagesNavItem.tsx 87.71% <0.00%> (+1.75%) ⬆️

Comment thread monitoring/monitoring/monitoring.go Outdated
@pecigonzalo

Copy link
Copy Markdown
Contributor

I would honestly prefer to remove the min and max 🤔 I wonder what is causing this issue as I have not had to wrap things up like this in general.

In any case, I would merge to fix the issues until we figure that out.

(not approving as I believe there is a typo which I mentioned)

Co-authored-by: Gonzalo Peci <pecigonzalo@users.noreply.github.com>
@bobheadxi

Copy link
Copy Markdown
Member Author

I would honestly prefer to remove the min and max 🤔

I think they're fine to add, not not fantastic, thing we are in the end achieving the same thing: alerting on the first occurrence of a value exceeding a threshold

I wonder what is causing this issue as I have not had to wrap things up like this in general.

This describes it well I think: https://github.com/sourcegraph/sourcegraph/issues/11571#issuecomment-654571953

@pecigonzalo

Copy link
Copy Markdown
Contributor

Thanks for that link, yeah, but we should not have those collisions, or it is rather rare, so I wonder if we have the wrong query or the wrong collection rule or something else altogether. Or maybe this is just more normal than I expect it to be :D

@bobheadxi

bobheadxi commented Jan 26, 2021

Copy link
Copy Markdown
Member Author

Our alert delivery, formatting, and entire model does not support per-series anyway, and there's a whole monitoring pillars section about why high-cardinality graphs are not ideal

If we want individual series alerting, I think they should just be separate panels, and instead of surfacing information through the alert itself IMO we should just push to have useful information about the panel be surfaced instead (e.g. a link to it, where more context can be derived), so I think this change is inline with that

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

monitoring: some observables end up with multiple metrics

3 participants