Skip to content

storage: optimized fanoutAppenderV2#17976

Merged
bwplotka merged 3 commits intomainfrom
bwplotka/fanout
Feb 2, 2026
Merged

storage: optimized fanoutAppenderV2#17976
bwplotka merged 3 commits intomainfrom
bwplotka/fanout

Conversation

@bwplotka
Copy link
Member

@bwplotka bwplotka commented Jan 30, 2026

Relates to #17632

This should remove the last blocker to introduce AppenderV2 by default on scrape, found in #17959 (comment)

benchstat x.bwplotka/fanoutAppender2.txt x.bwplotka/fanoutAppender3.txt
goos: darwin
goarch: arm64
pkg: github.com/prometheus/prometheus/storage
cpu: Apple M4 Pro
                                    │ x.bwplotka/fanoutAppender2.txt │   x.bwplotka/fanoutAppender3.txt   │
                                    │             sec/op             │   sec/op     vs base               │
FanoutAppenderV2/both_works-2                            32.79µ ± 3%   32.69µ ± 6%        ~ (p=0.937 n=6)
FanoutAppenderV2/primary_errors-2                        46.65µ ± 2%   13.97µ ± 1%  -70.06% (p=0.002 n=6)
FanoutAppenderV2/exemplar_errors-2                       32.70µ ± 1%   32.75µ ± 1%        ~ (p=0.394 n=6)
FanoutAppenderV2/secondary_errors-2                      61.56µ ± 1%   32.78µ ± 1%  -46.74% (p=0.002 n=6)
geomean                                                  41.89µ        26.46µ       -36.84%

                                    │ x.bwplotka/fanoutAppender2.txt │   x.bwplotka/fanoutAppender3.txt    │
                                    │              B/op              │    B/op     vs base                 │
FanoutAppenderV2/both_works-2                             192.0 ± 0%   192.0 ± 0%        ~ (p=1.000 n=6) ¹
FanoutAppenderV2/primary_errors-2                        8192.0 ± 0%   192.0 ± 0%  -97.66% (p=0.002 n=6)
FanoutAppenderV2/exemplar_errors-2                        192.0 ± 0%   192.0 ± 0%        ~ (p=1.000 n=6) ¹
FanoutAppenderV2/secondary_errors-2                      8192.0 ± 0%   192.0 ± 0%  -97.66% (p=0.002 n=6)
geomean                                                 1.225Ki        192.0       -84.69%
¹ all samples are equal

                                    │ x.bwplotka/fanoutAppender2.txt │   x.bwplotka/fanoutAppender3.txt    │
                                    │           allocs/op            │ allocs/op   vs base                 │
FanoutAppenderV2/both_works-2                             4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
FanoutAppenderV2/primary_errors-2                      1004.000 ± 0%   4.000 ± 0%  -99.60% (p=0.002 n=6)
FanoutAppenderV2/exemplar_errors-2                        4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
FanoutAppenderV2/secondary_errors-2                    1004.000 ± 0%   4.000 ± 0%  -99.60% (p=0.002 n=6)
geomean                                                   63.37        4.000       -93.69%
¹ all samples are equal

Does this PR introduce a user-facing change?

NONE

@bwplotka bwplotka requested a review from a team as a code owner January 30, 2026 16:38
@bwplotka bwplotka requested review from jan--f, krajorama and ywwg January 30, 2026 16:38
@bwplotka bwplotka changed the title Bwplotka/fanout storage: optimized fanoutAppenderV2 Jan 30, 2026
Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
@bwplotka
Copy link
Member Author

Hm, pErr still allocates for that does not yield partial errors. I think we could do better.

image

@bwplotka
Copy link
Member Author

I didn't honestly expect this code to alloc on non-partial errors

var pErr *AppendPartialError
	if !errors.As(err, &pErr) {
		return e, err
	}

.... but whatever 🙃

I found some way.

Copy link
Member

@krajorama krajorama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved with nit comment, nice catch

Signed-off-by: bwplotka <bwplotka@gmail.com>
@bwplotka bwplotka enabled auto-merge (squash) January 30, 2026 17:42
@bwplotka bwplotka merged commit 9657c23 into main Feb 2, 2026
67 of 68 checks passed
@bwplotka bwplotka deleted the bwplotka/fanout branch February 2, 2026 07:04
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Will Bollock <wbollock@linode.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants