Skip to content

MINOR: Gracefully close the consumer in ConsumeBenchWorker#6033

Closed
stanislavkozlovski wants to merge 5 commits into
apache:trunkfrom
stanislavkozlovski:trogdor-gracefully-close-consumer-task
Closed

MINOR: Gracefully close the consumer in ConsumeBenchWorker#6033
stanislavkozlovski wants to merge 5 commits into
apache:trunkfrom
stanislavkozlovski:trogdor-gracefully-close-consumer-task

Conversation

@stanislavkozlovski

@stanislavkozlovski stanislavkozlovski commented Dec 14, 2018

Copy link
Copy Markdown
Contributor

Since WorkerUtils.abort propagates the exception after handling it, any exceptions raised will not close the consumer properly

long curTimeMs = Time.SYSTEM.milliseconds();
log.info("{} Consumed total number of messages={}, bytes={} in {} ms. status: {}",
clientId, messagesConsumed, bytesConsumed, curTimeMs - startTimeMs, statusData);
consumer.close();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we'd want to close the consumer before completing doneFuture right? Given that, it seems like we should do it before calling abort and right after the finally. What do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I suggest renaming abort to abortAndThrow.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we're not done until we close the consumer

@ijuma ijuma left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, LGTM.

@ijuma

ijuma commented Dec 14, 2018

Copy link
Copy Markdown
Member

Quick question: any thoughts on a test we could write for this?

@cmccabe

cmccabe commented Dec 31, 2018

Copy link
Copy Markdown
Contributor

So, the thing to keep in mind here is that when the worker shuts itself down, its stop function will be called shortly thereafter. Since the stop function closes the consumer, there is no actual resource leak here.

One thing I notice that could be fixed is that ThreadSafeConsumer#close does not guard against unsubscribe throwing an exception. It would be good to add this in. Also, the locking is slightly wrong since ThreadSafeConsumer#closed is checked without holding the lock.

        void close() {
            if (closed)
                return;
            this.consumerLock.lock();
            try {
                consumer.unsubscribe();
                Utils.closeQuietly(consumer, "consumer");
                closed = true;
            } finally {
                this.consumerLock.unlock();
            }
        }

@github-actions

Copy link
Copy Markdown

This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

@github-actions github-actions Bot added the stale Stale PRs label Nov 21, 2024
@github-actions

Copy link
Copy Markdown

This PR has been closed since it has not had any activity in 120 days. If you feel like this
was a mistake, or you would like to continue working on it, please feel free to re-open the
PR and ask for a review.

@github-actions github-actions Bot added the closed-stale PRs that were closed due to inactivity label Dec 22, 2024
@github-actions github-actions Bot closed this Dec 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

closed-stale PRs that were closed due to inactivity stale Stale PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants