Skip to content

Conversation

@BewareMyPower
Copy link
Contributor

Motivation

It's never documented that the future of sendAsync is called in a single I/O thread. Without knowing it, it would be very easy to use sendAsync improperly, which might affect other producers or consumers in the same client, or even lead to deadlocks.

Modifications

Improve the JavaDocs of Producer#sendAsync.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@BewareMyPower BewareMyPower self-assigned this Aug 4, 2025
@BewareMyPower BewareMyPower added doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. release/4.0.7 release/3.0.14 release/3.3.9 labels Aug 4, 2025
@BewareMyPower BewareMyPower added this to the 4.1.0 milestone Aug 4, 2025
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
I think that there's a similar "feature" in using most async APIs of the Pulsar Java client. Would you mind checking other async methods? However, I guess sendAsync is the most commonly used async API since without using it, it's not possible to benefit from batching in many applications.

@BewareMyPower
Copy link
Contributor Author

Yes, sendAsync should be the most frequently used async API and it's a critical path. I don't have a chance to check all other async methods, but I have checked receiveAsync as well. Unlike sendAsync, receiveAsync switches to a different thread pool (PulsarClientImpl#internalExecutorProvider) from #10352, so it does not have the similar issue, but it could still affect other processing. Here are my notes when I read the code.


Currently a PulsarClient will create the following thread pools:

Thread Pool Name Number of Threads
internalExecutorProvider numIoThreads
externalExecutorProvider numListenerThreads
lookupExecutorProvider 1

internalExecutorProvider is used for:

  • Execute message listener, callbacks of receiveAsync() and batchReceiveAsync() for consumers
  • Many callbacks in TransactionMetaStoreHandler

externalExecutorProvider is used for:

  • Trigger message listener
  • Trigger consumer event listener

As you can see, transaction metadata processing could be affected. But I think it's because executors are used without any careful consideration. Before adding documents, we might need to think if the existing behavior should be documented.

@codecov-commenter
Copy link

codecov-commenter commented Aug 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.26%. Comparing base (d272825) to head (02deefc).
⚠️ Report is 12 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #24601      +/-   ##
============================================
+ Coverage     74.21%   74.26%   +0.04%     
+ Complexity    33142    33132      -10     
============================================
  Files          1881     1881              
  Lines        146770   146776       +6     
  Branches      16859    16859              
============================================
+ Hits         108922   108998      +76     
+ Misses        29181    29112      -69     
+ Partials       8667     8666       -1     
Flag Coverage Δ
inttests 26.65% <ø> (?)
systests 23.27% <ø> (-0.09%) ⬇️
unittests 73.75% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 103 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@BewareMyPower BewareMyPower merged commit dedba1a into apache:master Aug 5, 2025
73 of 77 checks passed
@BewareMyPower BewareMyPower deleted the bewaremypower/improve-send-docs branch August 5, 2025 03:29
lhotari pushed a commit that referenced this pull request Aug 5, 2025
lhotari pushed a commit that referenced this pull request Aug 5, 2025
lhotari pushed a commit that referenced this pull request Aug 5, 2025
nodece pushed a commit to ascentstream/pulsar that referenced this pull request Aug 6, 2025
gaozhangmin pushed a commit to gaozhangmin/pulsar that referenced this pull request Aug 13, 2025
poorbarcode pushed a commit to poorbarcode/pulsar that referenced this pull request Aug 14, 2025
ganesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 20, 2025
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 20, 2025
manas-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 20, 2025
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 26, 2025
KannarFr pushed a commit to CleverCloud/pulsar that referenced this pull request Sep 22, 2025
walkinggo pushed a commit to walkinggo/pulsar that referenced this pull request Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants