Skip to content

Conversation

@dao-jun
Copy link
Member

@dao-jun dao-jun commented Jun 14, 2024

Fixes #22866

Motivation

Optimize ConcurrentOpenLongPairRangeSet to prevent long GC pause

Modifications

  1. Bump RoaringBitmap version to 1.1.0
  2. Optimize ConcurrentOpenLongPairRangeSet by org.roaringbitmap.RoaringBitSet
  3. Make ConcurrentRoaringBitSet extends org.roaringbitmap.RoaringBitSet to make it concurrency.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@dao-jun dao-jun self-assigned this Jun 14, 2024
@dao-jun dao-jun added the category/performance Performance issues fix or improvements label Jun 14, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jun 14, 2024
@dao-jun dao-jun marked this pull request as ready for review June 15, 2024 15:27
@dao-jun dao-jun closed this Jun 15, 2024
@dao-jun dao-jun reopened this Jun 15, 2024
@lhotari
Copy link
Member

lhotari commented Jun 16, 2024

Please upgrade RoaringBitmap to latest version 1.0.6. it contains useful improvements such as RoaringBitmap/RoaringBitmap#713.

@dao-jun
Copy link
Member Author

dao-jun commented Jun 16, 2024

Please upgrade RoaringBitmap to latest version 1.0.6. it contains useful improvements such as RoaringBitmap/RoaringBitmap#713.

Thanks for reminding, I'll create another PR to do the upgrading

@dao-jun dao-jun marked this pull request as draft June 16, 2024 15:37
@dao-jun dao-jun marked this pull request as ready for review June 16, 2024 15:46
@dao-jun dao-jun closed this Jun 16, 2024
@dao-jun dao-jun reopened this Jun 16, 2024
@lhotari
Copy link
Member

lhotari commented Jun 16, 2024

Please upgrade RoaringBitmap to latest version 1.0.6. it contains useful improvements such as RoaringBitmap/RoaringBitmap#713.

Thanks for reminding, I'll create another PR to do the upgrading

The new features could be useful in this PR. That's why it could be useful to simply do in this PR.

@dao-jun
Copy link
Member Author

dao-jun commented Jun 17, 2024

@lhotari @mattisonchao @shibd PTAL

@dao-jun dao-jun requested a review from mattisonchao June 17, 2024 08:46
lhotari added a commit that referenced this pull request Jun 20, 2024
…gBitmap (#22908)

(cherry picked from commit 5b1f653)

# Conflicts:
#	distribution/server/src/assemble/LICENSE.bin.txt
#	pom.xml
lhotari pushed a commit that referenced this pull request Jun 20, 2024
lhotari pushed a commit that referenced this pull request Jun 20, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 21, 2024
…gBitmap (apache#22908)

(cherry picked from commit 5b1f653)
(cherry picked from commit f99040d)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 24, 2024
…gBitmap (apache#22908)

(cherry picked from commit 5b1f653)
(cherry picked from commit f99040d)
lhotari added a commit that referenced this pull request Jun 24, 2024
lhotari added a commit that referenced this pull request Jun 24, 2024
lhotari added a commit that referenced this pull request Jun 24, 2024
@lhotari
Copy link
Member

lhotari commented Jun 24, 2024

I have reverted the changes in branch-3.0 branch-3.2 and branch-3.3 due to regressions referred in #22966

lhotari added a commit to lhotari/pulsar that referenced this pull request Jun 24, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 25, 2024
…gBitmap (apache#22908)

(cherry picked from commit 5b1f653)
(cherry picked from commit f99040d)
merlimat pushed a commit that referenced this pull request Jun 25, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 27, 2024
…y RoaringBitmap (apache#22908)"

This reverts commit f99040d.

(cherry picked from commit 266f98c)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 1, 2024
…gBitmap (apache#22908)

(cherry picked from commit 5b1f653)
(cherry picked from commit f99040d)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 1, 2024
…y RoaringBitmap (apache#22908)"

This reverts commit f99040d.

(cherry picked from commit 266f98c)
@lhotari lhotari added this to the 4.0.0 milestone Oct 14, 2024
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category/performance Performance issues fix or improvements doc-not-needed Your PR changes do not impact docs ready-to-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

High GC pause cases high publish latency

2 participants