Skip to content
This repository was archived by the owner on Aug 23, 2023. It is now read-only.

Rollup indicator cleanup#1508

Merged
Dieterbe merged 12 commits intomasterfrom
rollup-indicator-cleanup
Nov 4, 2019
Merged

Rollup indicator cleanup#1508
Dieterbe merged 12 commits intomasterfrom
rollup-indicator-cleanup

Conversation

@Dieterbe
Copy link
Copy Markdown
Contributor

@Dieterbe Dieterbe commented Oct 24, 2019

main changes:

  • cleaner output for users
  • also include archive interval, because interval may not match what schema says. e.g. for kafka input plugin.

I need to fix the unit tests, but it works.
verified with fakemetrics to feed some data at 5s, and some at 1s + a storage-schemas.conf like so:

[default-1]
pattern = some.*1
retentions = 1s:6h:2min:2,1min:35d:6h:1

[default-2]
pattern = some.*2
retentions = 2s:6h:2min:2,1min:35d:6h:1

[default-4]
pattern = some.*4
retentions = 4s:6h:2min:2,1min:35d:6h:1

[default-5]
pattern = some.*5
retentions = 5s:6h:2min:2,1min:35d:6h:1

[default]
pattern = .*
retentions = 10s:6h:2min:2


curl 'http://localhost:6061/render?target=sum(some.id.of.a.metric.*)&from=-30s&meta=true' | jsonpp | less
{
    "version": "v0.1",
    "meta": {
        "stats": {
            "executeplan.resolve-series.ms": 8,
            "executeplan.get-targets.ms": 11,
            "executeplan.prepare-series.ms": 0,
            "executeplan.plan-run.ms": 0,
            "executeplan.series-fetch.count": 300,
            "executeplan.points-fetch.count": 6600,
            "executeplan.points-return.count": 1800,
            "executeplan.cache-miss.count": 0,
            "executeplan.cache-hit-partial.count": 0,
            "executeplan.cache-hit.count": 0,
            "executeplan.chunks-from-tank.count": 300,
            "executeplan.chunks-from-cache.count": 0,
            "executeplan.chunks-from-store.count": 0
        }
    },
    "series": [
        {
            "target": "sumSeries(some.id.of.a.metric.*)",
            "datapoints": [
                [
                    9990.44220170934,
                    1571948135
                ],
                [
                    10350.177267618516,
                    1571948140
                ],
                [
                    9887.481073889445,
                    1571948145
                ],
                [
                    10312.657377260079,
                    1571948150
                ],
                [
                    2473.5966041257966,
                    1571948155
                ],
                [
                    null,
                    1571948160
                ]
            ],
            "meta": [
                {
                    "schema-name": "default-1",
                    "schema-retentions": "1s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 5,
                    "aggnum-norm": 1,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 20
                },
                {
                    "schema-name": "default-1",
                    "schema-retentions": "1s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 1,
                    "aggnum-norm": 5,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 119
                },
                {
                    "schema-name": "default-2",
                    "schema-retentions": "2s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 5,
                    "aggnum-norm": 1,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 17
                },
                {
                    "schema-name": "default-2",
                    "schema-retentions": "2s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 1,
                    "aggnum-norm": 5,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 18
                },
                {
                    "schema-name": "default",
                    "schema-retentions": "10s:6h:2min:2",
                    "archive-read": 0,
                    "archive-interval": 5,
                    "aggnum-norm": 1,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 35
                },
                {
                    "schema-name": "default",
                    "schema-retentions": "10s:6h:2min:2",
                    "archive-read": 0,
                    "archive-interval": 1,
                    "aggnum-norm": 5,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 35
                },
                {
                    "schema-name": "default-4",
                    "schema-retentions": "4s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 5,
                    "aggnum-norm": 1,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 15
                },
                {
                    "schema-name": "default-4",
                    "schema-retentions": "4s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 1,
                    "aggnum-norm": 5,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 15
                },
                {
                    "schema-name": "default-5",
                    "schema-retentions": "5s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 5,
                    "aggnum-norm": 1,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 13
                },
                {
                    "schema-name": "default-5",
                    "schema-retentions": "5s:6h:2min:2,1min:35d:6h:1",
                    "archive-read": 0,
                    "archive-interval": 1,
                    "aggnum-norm": 5,
                    "consolidate-normfetch": "AverageConsolidator",
                    "aggnum-rc": 0,
                    "consolidate-rc": "NoneConsolidator",
                    "count": 13
                }
            ]
        }
    ]
}

@Dieterbe
Retentions: add field to track original string input
d5f32fd 
This will become useful in response metadata when we want to include theschema used
@Dieterbe
comments
388b2bb
@Dieterbe Dieterbe force-pushed the rollup-indicator-cleanup branch from e05f9dc to 08945be Compare October 25, 2019 10:05
@Dieterbe Dieterbe force-pushed the rollup-indicator-cleanup branch from 08945be to 54b7fb2 Compare October 25, 2019 22:28
@Dieterbe Dieterbe force-pushed the rollup-indicator-cleanup branch from 54b7fb2 to fca78e1 Compare October 25, 2019 23:02
@Dieterbe Dieterbe changed the title WIP Rollup indicator cleanup Rollup indicator cleanup Oct 25, 2019
@Dieterbe
Copy link
Copy Markdown
Contributor Author

Dieterbe commented Oct 28, 2019

fix #1486

replay
replay reviewed Oct 28, 2019
View reviewed changes
replay
replay reviewed Oct 28, 2019
View reviewed changes
replay
replay approved these changes Oct 28, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@replay replay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one comment

@Dieterbe Dieterbe merged commit 38028b3 into master Nov 4, 2019
@Dieterbe Dieterbe deleted the rollup-indicator-cleanup branch November 4, 2019 13:56
Dieterbe added a commit that referenced this pull request Nov 4, 2019
@Dieterbe
fix build.
367c1bc 
I broke the build in #1508
@Dieterbe Dieterbe mentioned this pull request Nov 4, 2019
Merged
@Dieterbe Dieterbe mentioned this pull request Jan 22, 2020
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@replay replay replay approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Dieterbe @replay