Skip to content

[fix][broker] Lower log level of DrainingHashesTracker not-found entry to DEBUG#25558

Merged
lhotari merged 1 commit into
apache:masterfrom
singhvishalkr:fix-pulsar-25277-draining-hashes-log-level
Apr 21, 2026
Merged

[fix][broker] Lower log level of DrainingHashesTracker not-found entry to DEBUG#25558
lhotari merged 1 commit into
apache:masterfrom
singhvishalkr:fix-pulsar-25277-draining-hashes-log-level

Conversation

@singhvishalkr

Copy link
Copy Markdown
Contributor

Fixes #25277

Motivation

DrainingHashesTracker.ConsumerDrainingHashesStats#updateConsumerStats currently logs a WARN every time the hash iterator observes an entry that was concurrently removed:

https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/DrainingHashesTracker.java#L188-L196

if (entry == null) {
    log.warn()
            .attr("dispatcher", dispatcherName)
            .attr("hash", hash)
            .attr("consumer", consumer)
            .log("Draining hash not found in the tracker for consumer");
    continue;
}

In the original PIP-379 design the draining-hash stats were strongly consistent with the tracker state, so a missing entry on this read path would indeed have been unexpected. That strong-consistency guarantee was intentionally dropped in #23854 to break a deadlock — the stats read path is now allowed to race with entry removal.

A null entry is therefore a benign, expected outcome during normal broker operation, not an alertable condition. Logging at WARN floods broker logs and drowns out actionable warnings.

The issue author (@lhotari) explicitly notes: "The warning log no longer makes sense and should be changed to debug logging."

Modifications

No behavioural change — the continue and surrounding logic are untouched.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial fix without any test coverage: it only lowers the level of an existing log statement. No functional behaviour changes, so no new tests are required (existing DrainingHashesTracker tests still cover the race path).

Does this pull request potentially affect one of the following parts:

None of the listed areas are affected. Log level only.

Documentation

  • doc-not-needed

No user-facing or config documentation references this log line.

…y to DEBUG

The `log.warn()` call in `DrainingHashesTracker.ConsumerDrainingHashesStats
#updateConsumerStats` fires whenever the hash-iterator observes an entry that
was already removed by a concurrent write path. In the original PIP-379
design the draining-hash stats were strongly consistent with the tracker
state, so a missing entry on the read path was genuinely unexpected.

That strong-consistency guarantee was intentionally dropped in apache#23854 to
resolve a deadlock: the stats path is now allowed to race with entry
removal. A missing entry is therefore a benign, expected outcome during
normal operation, not a signal worth alerting on.

Logging this at `WARN` floods broker logs without a corresponding
actionable condition. Drop it to `DEBUG` and add an inline comment
explaining the race so future readers understand why this is deliberately
quiet.

Fixes apache#25277
@merlimat merlimat requested a review from lhotari April 21, 2026 16:56

@lhotari lhotari left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the contribution @singhvishalkr

@lhotari lhotari merged commit 423a746 into apache:master Apr 21, 2026
44 checks passed
@lhotari lhotari added this to the 4.3.0 milestone Apr 21, 2026
lhotari pushed a commit that referenced this pull request Apr 21, 2026
lhotari pushed a commit that referenced this pull request Apr 21, 2026
lhotari pushed a commit that referenced this pull request Apr 21, 2026
@lhotari

lhotari commented Apr 21, 2026

Copy link
Copy Markdown
Member

@singhvishalkr I've backported the change to maintenance branches. This change will be included in 4.0.10 and 4.2.1 releases which are currently in progress. The current ETA for the release is Apr 30th.

srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 23, 2026
…y to DEBUG (apache#25558)

(cherry picked from commit 423a746)
(cherry picked from commit 4850be8)
nodece pushed a commit to ascentstream/pulsar that referenced this pull request May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Noisy warning log in DrainingHashesTracker.ConsumerDrainingHashesStats#updateConsumerStats

2 participants