Skip to content

Conversation

@lhotari
Copy link
Member

@lhotari lhotari commented Sep 22, 2025

Fixes #24754

Motivation

The contract of Comparable.compareTo:

  • The implementor must ensure signum(x.compareTo(y)) == -signum(y.compareTo(x)) for all x and y. (This implies that x.compareTo(y) must throw an exception if and only if y.compareTo(x) throws an exception.)

  • The implementor must also ensure that the relation is transitive: (x.compareTo(y) > 0 && y.compareTo(z) > 0) implies x.compareTo(z) > 0.

  • Finally, the implementor must ensure that x.compareTo(y)==0 implies that signum(x.compareTo(z)) == signum(y.compareTo(z)), for all z.

Modifications

  • address the compareTo violation by using "resolution" instead of "difference" when comparing values since it's not possible to fulfill the Comparable.compareTo contract by using a difference.
  • add tests for compareTo for NamespaceBundleStats and TimeAverageMessageData
  • Rename the ResourceUnitRanking's compareTo method to compareToOtherRanking and remove the Comparable interface since ResourceUnitRanking cannot fulfill the contract and there isn't a need to do so. Removing the interface and new comments in the compareToOtherRanking method will help prevent possible misuse later. The current usage of ResourceUnitRanking's method is fine, it's just not requiring Comparable.compareTo semantics and contract. That's why it's better to rename the method.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

…sageData

- JVM's default sorting algorithm "TimSort" will fail with exception if compareTo doesn't meet the
  contract of Comparable.compareTo interface
@lhotari lhotari added this to the 4.2.0 milestone Sep 22, 2025
@lhotari lhotari self-assigned this Sep 22, 2025
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 22, 2025
@lhotari lhotari changed the title [fix][misc] Fix compareTo for NamespaceBundleStats and TimeAverageMessageData [fix][misc] Fix compareTo contract violation for NamespaceBundleStats, TimeAverageMessageData and ResourceUnitRanking Sep 22, 2025
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.23%. Comparing base (0c6ba1c) to head (9103b09).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             master   #24772       +/-   ##
=============================================
+ Coverage     39.04%   74.23%   +35.18%     
- Complexity    13345    33248    +19903     
=============================================
  Files          1844     1902       +58     
  Lines        144311   148392     +4081     
  Branches      16730    17201      +471     
=============================================
+ Hits          56353   110158    +53805     
+ Misses        80460    29444    -51016     
- Partials       7498     8790     +1292     
Flag Coverage Δ
inttests 26.40% <50.00%> (-0.37%) ⬇️
systests 22.80% <28.57%> (-0.04%) ⬇️
unittests 73.74% <100.00%> (+38.57%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...broker/loadbalance/impl/SimpleLoadManagerImpl.java 75.32% <100.00%> (+24.27%) ⬆️
...licies/data/loadbalancer/NamespaceBundleStats.java 100.00% <100.00%> (+34.14%) ⬆️
...ava/org/apache/pulsar/common/util/CompareUtil.java 100.00% <100.00%> (ø)
...olicies/data/loadbalancer/ResourceUnitRanking.java 88.00% <ø> (+15.00%) ⬆️
...cies/data/loadbalancer/TimeAverageMessageData.java 100.00% <100.00%> (+18.96%) ⬆️

... and 1391 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@merlimat merlimat merged commit 7504538 into apache:master Sep 22, 2025
102 of 104 checks passed
merlimat pushed a commit that referenced this pull request Sep 22, 2025
…, TimeAverageMessageData and ResourceUnitRanking (#24772)
merlimat pushed a commit that referenced this pull request Sep 22, 2025
…, TimeAverageMessageData and ResourceUnitRanking (#24772)
@lhotari
Copy link
Member Author

lhotari commented Sep 23, 2025

After merging, I noticed that NamespaceBundleStats uses a slightly different implementation for compareDoubleWithResolution. I didn't use org.apache.pulsar.common.util.CompareUtil#compareDoubleWithResolution in that case since the module doesn't currently depend on pulsar-common.
The difference is that NamespaceBundleStats.compareDoubleWithResolution will use rounding to closest integer value Long.compare(Math.round(v1 / resolution), Math.round(v2 / resolution)) and CompareUtil.compareDoubleWithResolution will use just Long.compare((long) (v1 / resolution), (long) (v2 / resolution)). In practice, this doesn't make a difference since both will compare with similar resolution/granularity and the compareTo contract is fulfilled in both cases.
This is a small inconsistency, but I don't see a need to address it.

lhotari added a commit that referenced this pull request Sep 23, 2025
…, TimeAverageMessageData and ResourceUnitRanking (#24772)

(cherry picked from commit 7504538)
manas-ctds pushed a commit to datastax/pulsar that referenced this pull request Sep 26, 2025
…, TimeAverageMessageData and ResourceUnitRanking (apache#24772)

(cherry picked from commit 7504538)
(cherry picked from commit ae9be46)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Sep 26, 2025
…, TimeAverageMessageData and ResourceUnitRanking (apache#24772)

(cherry picked from commit 7504538)
(cherry picked from commit ae9be46)
manas-ctds pushed a commit to datastax/pulsar that referenced this pull request Sep 29, 2025
…, TimeAverageMessageData and ResourceUnitRanking (apache#24772)

(cherry picked from commit d6b00f9)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Sep 29, 2025
…, TimeAverageMessageData and ResourceUnitRanking (apache#24772)

(cherry picked from commit d6b00f9)
walkinggo pushed a commit to walkinggo/pulsar that referenced this pull request Oct 8, 2025
nodece pushed a commit to ascentstream/pulsar that referenced this pull request Oct 30, 2025
…, TimeAverageMessageData and ResourceUnitRanking (apache#24772)

(cherry picked from commit 7504538)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Failure in topK bundle selection due to the transitivity Contract Violation by namespaceBundleStats' & BundleData's compareTo()

3 participants