-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Closed
Closed
Copy link
Labels
type/bugThe PR fixed a bug or issue reported a bugThe PR fixed a bug or issue reported a bug
Description
Search before reporting
- I searched in the issues and found nothing similar.
Read release policy
- I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
User environment
master branch version of Pulsar, 34f8657
Issue Description
There's a opportunity to significantly reduce memory allocations in Pulsar by caching parsed message metadata. In a simple profiling test case in #24623 I noticed that
there's a lot of allocations coming from org.apache.pulsar.common.protocol.Commands.peekAndCopyMessageMetadata.
The broker cache can be modified to support caching the message metadata together with the entries so that unnecessary parsing and allocations can be avoided.
Error messages
Reproducing the issue
With #24623 changes, enabling the BrokerEntryCacheMultiBrokerTest.testTailingReadsRollingRestart test
by modifying the @Test(enabled = false) annotation to @Test(invocationCount = 5) and then running
- Download Async Profiler from https://github.com/async-profiler/async-profiler/releases and after installing set
LIBASYNCPROFILER_PATHenvironment variable to the full path of thelibasyncProfiler.so(Linux) orlibasyncProfiler.dylib(MacOS) file. - You can use Eclipse Mission Control (https://adoptium.net/jmc) to open the JFR file and analyze it.
export LIBASYNCPROFILER_PATH=/full/path/to/async-profiler/lib/libasyncProfiler.so (or .dylib on MacOS)
mvn -PtestAsyncProfiler test -pl pulsar-broker -Dtest=BrokerEntryCacheMultiBrokerTestAdditional information
No response
Are you willing to submit a PR?
- I'm willing to submit a PR!
Metadata
Metadata
Assignees
Labels
type/bugThe PR fixed a bug or issue reported a bugThe PR fixed a bug or issue reported a bug