Skip to content

perf(p2p/channel): Speedup NewDelimitedWriter#2949

Merged
melekes merged 3 commits intocometbft:mainfrom
osmosis-labs:dev/speedup_NewDelimitedWriter
May 2, 2024
Merged

perf(p2p/channel): Speedup NewDelimitedWriter#2949
melekes merged 3 commits intocometbft:mainfrom
osmosis-labs:dev/speedup_NewDelimitedWriter

Conversation

@ValarDragon
Copy link
Contributor

@ValarDragon ValarDragon commented Apr 30, 2024

Speeds up 5% of the non-IO time overhead from channel.WritePacketMsgTo. The CPU time overhead in this function is quite significant, CPU time is more than 3 times the syscall time for writing to the net buffer. Working on a github issue for more substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted. However we should do things that reduce this overhead further.

image


PR checklist

  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments
  • Title follows the Conventional Commits spec

@ValarDragon ValarDragon requested a review from a team as a code owner April 30, 2024 19:38
@ValarDragon ValarDragon requested a review from a team April 30, 2024 19:38
@ValarDragon ValarDragon changed the title Speedup NewDelimitedWriter perf(p2p/channel): Speedup NewDelimitedWriter Apr 30, 2024
Copy link
Collaborator

@melekes melekes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ValarDragon ❤️

@melekes melekes added backport-to-v0.37.x backport-to-v0.38.x Tell Mergify to backport the PR to v0.38.x labels May 1, 2024
@melekes melekes added this pull request to the merge queue May 2, 2024
Merged via the queue into cometbft:main with commit 8162aa2 May 2, 2024
mergify bot pushed a commit that referenced this pull request May 2, 2024
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

(cherry picked from commit 8162aa2)
mergify bot pushed a commit that referenced this pull request May 2, 2024
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

(cherry picked from commit 8162aa2)
mergify bot pushed a commit that referenced this pull request May 2, 2024
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

(cherry picked from commit 8162aa2)
melekes pushed a commit that referenced this pull request May 2, 2024
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request #2949 done by
[Mergify](https://mergify.com).

Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
melekes pushed a commit that referenced this pull request May 2, 2024
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request #2949 done by
[Mergify](https://mergify.com).

Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
melekes pushed a commit that referenced this pull request May 2, 2024
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request #2949 done by
[Mergify](https://mergify.com).

Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
czarcas7ic pushed a commit to osmosis-labs/cometbft that referenced this pull request May 2, 2024
cometbft#2969)

Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2949 done by
[Mergify](https://mergify.com).

Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
czarcas7ic added a commit to osmosis-labs/cometbft that referenced this pull request May 3, 2024
cometbft#2969) (#40)

Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2949 done by
[Mergify](https://mergify.com).

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
mergify bot added a commit to osmosis-labs/cometbft that referenced this pull request May 3, 2024
cometbft#2969) (#40)

Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2949 done by
[Mergify](https://mergify.com).

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
(cherry picked from commit 3d1b9dc)
mergify bot added a commit to osmosis-labs/cometbft that referenced this pull request May 3, 2024
cometbft#2969) (#40)

Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2949 done by
[Mergify](https://mergify.com).

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
(cherry picked from commit 3d1b9dc)
czarcas7ic added a commit to osmosis-labs/cometbft that referenced this pull request May 3, 2024
cometbft#2969) (#40) (#48)

Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2949 done by
[Mergify](https://mergify.com).

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
(cherry picked from commit 3d1b9dc)

Co-authored-by: Adam Tucker <adam@osmosis.team>
czarcas7ic added a commit to osmosis-labs/cometbft that referenced this pull request May 3, 2024
cometbft#2969) (#40) (#49)

Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.

![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request cometbft#2949 done by
[Mergify](https://mergify.com).

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
(cherry picked from commit 3d1b9dc)

Co-authored-by: Adam Tucker <adam@osmosis.team>
mattac21 pushed a commit that referenced this pull request Sep 9, 2025
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
mattac21 pushed a commit that referenced this pull request Sep 10, 2025
Speeds up 5% of the non-IO time overhead from
`channel.WritePacketMsgTo`. The CPU time overhead in this function is
quite significant, CPU time is more than 3 times the syscall time for
writing to the net buffer. Working on a github issue for more
substantial refactor / time eliminations, but this 3s is easy enough.

We don't even use this codepath, so this make slice is entirely wasted.
However we should do things that reduce this overhead further.


![image](https://github.com/cometbft/cometbft/assets/6440154/e02e45bf-d6ff-4e11-9983-e81ca1102dc8)

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v0.38.x Tell Mergify to backport the PR to v0.38.x

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants