feat(tools): add debug printouts to rules unit testing#15196
Merged
Conversation
This makes it print out the tsdb state (both input_series and rules that are run) at the end of a test, making reasoning about tests much easier. Signed-off-by: David Leadbeater <dgl@dgl.cx>
# Conflicts: # cmd/promtool/main.go # cmd/promtool/unittest.go # cmd/promtool/unittest_test.go
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
krajorama
commented
Oct 22, 2024
| // Potentially a test can be specified at a time with fractional seconds, | ||
| // which PromQL cannot represent, so round up to the next whole second. | ||
| ts = (ts + time.Second).Truncate(time.Second) | ||
| expr := fmt.Sprintf(`{__name__=~".+"}[%v]`, ts) |
Member
Author
There was a problem hiding this comment.
The one issue here is that the engine is started with max samples == 10000, so I get an error when running with a lot of data:
$ ./promtool test rules --debug cmd/promtool/testdata/unittest.yml
DEBUG: Starting test unnamed#0
DEBUG: Dump of all data (input_series and rules) at 5m1s:
{__name__="ALERTS", alertname="AlwaysFiring", alertstate="firing"} =>
1 @[60000]
1 @[120000]
1 @[180000]
1 @[240000]
1 @[300000]
{__name__="ALERTS_FOR_STATE", alertname="AlwaysFiring"} =>
0 @[60000]
0 @[120000]
0 @[180000]
0 @[240000]
0 @[300000]
{__name__="fixed_data"} =>
1 @[60000]
1 @[120000]
1 @[180000]
1 @[240000]
1 @[300000]
{__name__="test_full"} =>
0 @[60000]
{__name__="test_histogram_increase"} =>
{count:3, sum:4.3, (0.5,1]:3} @[60000]
{count:4, sum:5.6, (0.5,1]:4} @[120000]
{__name__="test_histogram_repeat"} =>
{count:2, sum:3, (0.5,1]:2} @[60000]
{count:2, sum:3, (0.5,1]:2} @[120000]
{__name__="test_increase"} =>
2 @[60000]
3 @[120000]
{__name__="test_repeat"} =>
1 @[60000]
1 @[120000]
DEBUG: Test unnamed#0 finished, took 21.667828ms
DEBUG: Starting test unnamed#1
DEBUG: Dump of all data (input_series and rules) at 1s:
DEBUG: Test unnamed#1 finished, took 18.396051ms
DEBUG: Starting test unnamed#2
DEBUG: Dump of all data (input_series and rules) at 1h0m1s:
<cut out by krajorama>
DEBUG: Test unnamed#2 finished, took 32.819317ms
DEBUG: Starting test unnamed#3
DEBUG: Failed query exec, expr: "{__name__=~\".+\"}[24h0m1s]", err: query processing would load too many samples into memory in query execution
DEBUG: Test unnamed#3 finished, took 648.141484ms
DEBUG: Starting test unnamed#4
<rest cut out by krajorama>
Member
There was a problem hiding this comment.
We can fix in subsequent PR 👍🏽
bwplotka
approved these changes
Oct 22, 2024
| // Potentially a test can be specified at a time with fractional seconds, | ||
| // which PromQL cannot represent, so round up to the next whole second. | ||
| ts = (ts + time.Second).Truncate(time.Second) | ||
| expr := fmt.Sprintf(`{__name__=~".+"}[%v]`, ts) |
Member
There was a problem hiding this comment.
We can fix in subsequent PR 👍🏽
KeyOfSpectator
added a commit
to AliyunContainerService/prometheus
that referenced
this pull request
Oct 23, 2024
* master: (667 commits) NHCB scrape: refactor state handling and speed up scrape test (prometheus#15193) feat(tools): add debug printouts to rules unit testing (prometheus#15196) docs: add keep_firing_for in alerting rules api: Add rule group pagination to list rules api (prometheus#14017) Update scrape/scrape.go benchmark, rename parser omtext_with_nhcb goimports run Better docstring on test Remove omcounterdata.txt as redundant Fix failing benchmarks Add unit test to show that current wrapper is sub-optimal Rename convert_classic_histograms to convert_classic_histograms_to_nhcb More followup to prometheus#15164 Follow up prometheus#15178 Followup to prometheus#15164 test(cmd/prometheus): speed up test execution by t.Parallel() when possible feat: normalize "le" and "quantile" labels values upon ingestion scrape: provide a fallback format (prometheus#15136) Disallowing configure AM with the v1 api (prometheus#13883) feat: ProtobufParse.formatOpenMetricsFloat: improve float formatting by using strconv.AppendFloat instead of fmt.Sprint ... # Conflicts: # go.mod # go.sum
|
I think you meant to tag @dgl rather than dlg. |
julienduchesne
pushed a commit
to julienduchesne/prometheus
that referenced
this pull request
Dec 13, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This makes it print out the tsdb state (both input_series and rules that
are run) at the end of a test, making reasoning about tests much easier.
Originally by @dlg
Continued from #8349 .