Skip to content

perf(internal/bits): Significantly speedup bitArray.PickRandom (backp… (backport #28)#32

Merged
czarcas7ic merged 1 commit intoosmo-v24/v0.37.4from
mergify/bp/osmo-v24/v0.37.4/pr-28
Apr 30, 2024
Merged

perf(internal/bits): Significantly speedup bitArray.PickRandom (backp… (backport #28)#32
czarcas7ic merged 1 commit intoosmo-v24/v0.37.4from
mergify/bp/osmo-v24/v0.37.4/pr-28

Conversation

@mergify
Copy link

@mergify mergify bot commented Apr 30, 2024

…ort cometbft#2841) (cometbft#2887)

This PR significantly speeds up bitArray.PickRandom which is used in VoteGossip and BlockPart gossip. We saw for a query serving full node, over an hour, this was a very large amount of RAM allocations. (75GB of RAM!)

image

This PR drops it down to 0 allocations, and makes the routine 10x faster on my machine.

OLD:

BenchmarkPickRandomBitArray-12           1545199               846.1 ns/op          1280 B/op          1 allocs/op

NEW:

BenchmarkPickRandomBitArray-12          22192857                75.39 ns/op            0 B/op          0 allocs/op

I think the new tests I wrote make this more tested than the old code that was here tbh, but pls let me know if theres more tests we'd like to see!




PR checklist

  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments

This is an automatic backport of pull request #28 done by [Mergify](https://mergify.com).

#28)

* perf(internal/bits): Significantly speedup bitArray.PickRandom (backport cometbft#2841) (cometbft#2887)

This PR significantly speeds up bitArray.PickRandom which is used in
VoteGossip and BlockPart gossip. We saw for a query serving full node,
over an hour, this was a very large amount of RAM allocations. (75GB of
RAM!)

![image](https://github.com/cometbft/cometbft/assets/6440154/755918a5-0cef-4e67-a47e-ce8a56aa1cd5)

This PR drops it down to 0 allocations, and makes the routine 10x faster
on my machine.

OLD:
```
BenchmarkPickRandomBitArray-12           1545199               846.1 ns/op          1280 B/op          1 allocs/op
```
NEW:
```
BenchmarkPickRandomBitArray-12          22192857                75.39 ns/op            0 B/op          0 allocs/op
```

I think the new tests I wrote make this more tested than the old code
that was here tbh, but pls let me know if theres more tests we'd like to
see!

---

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2841 done by
[Mergify](https://mergify.com).

---------

Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix failing test

* changelog

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
(cherry picked from commit f6abfce)
@czarcas7ic czarcas7ic merged commit 63def3b into osmo-v24/v0.37.4 Apr 30, 2024
@mergify mergify bot deleted the mergify/bp/osmo-v24/v0.37.4/pr-28 branch April 30, 2024 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant