Skip to content

KAFKA-16189; Extend admin to support ConsumerGroupDescribe API#15253

Merged
dajac merged 16 commits into
apache:trunkfrom
dajac:KAFKA-16189
Feb 1, 2024
Merged

KAFKA-16189; Extend admin to support ConsumerGroupDescribe API#15253
dajac merged 16 commits into
apache:trunkfrom
dajac:KAFKA-16189

Conversation

@dajac

@dajac dajac commented Jan 24, 2024

Copy link
Copy Markdown
Member

This patch extends the Admin client to support describing new consumer groups with the ConsumerGroupDescribe API introduced in KIP-848. Users will continue to use the Admin#describeConsumerGroups API. The admin client does all the magic. Basically, the admin client always tries to describe the requested groups with the ConsumerGroupDescribe API to start with. If all the groups are there, great, the job is done. If there are groups unresolved groups due to a UNSUPPORTED_VERSION or GROUP_ID_NOT_FOUND error, the admin client tries with the DescribeGroups API.

The patch also adds fields to the data structure returned by Admin#describeConsumerGroups as stated in the KIP.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@dajac dajac added the KIP-848 The Next Generation of the Consumer Rebalance Protocol label Jan 24, 2024
@dajac

dajac commented Jan 26, 2024

Copy link
Copy Markdown
Member Author

cc @nizhikov @jolshan FYI - I have a few minor changes related to the consumer group command here. As saw that you’re working on migrating it to Java.

@nizhikov

Copy link
Copy Markdown
Contributor

Hello @dajac

Right now I have #15256 which moves DescribeConsumerGroupTest and ConsumerGroupCommandTest to tools module.
You are modifying those tests in PR.
My PR is ready and I hope will be merged soone.

May be you can wait for it and modify tests in tools?
Or even review it, you may be familiar with the tests right now

@dajac

dajac commented Jan 29, 2024

Copy link
Copy Markdown
Member Author

Thanks for letting me know, @nizhikov. I can definitely adapt my PR when #15256 gets merged. However, I cannot wait too long because this PR is needed for testing KIP-848.

private final boolean isSimpleConsumerGroup;
private final Collection<MemberDescription> members;
private final String partitionAssignor;
private final GroupType type;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a String here in the KIP. It seems better to use an Enum as we have it.

@dajac dajac marked this pull request as ready for review January 30, 2024 12:12
@dajac dajac requested a review from mimaison January 30, 2024 12:13
@dajac

dajac commented Jan 30, 2024

Copy link
Copy Markdown
Member Author

Hey @mimaison! As you have done some work in this area, I wonder if you would be interested by reviewing this PR. It would help us with the KIP-848 effort. We need this as soon as possible for testing purposes. Would you have time?

@cadonna cadonna left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dajac Thanks for the PR!

Here my comments!

@dajac

dajac commented Jan 31, 2024

Copy link
Copy Markdown
Member Author

Thanks @cadonna! I have addressed your comments.

@cadonna cadonna left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates, @dajac !

LGTM!

@dajac dajac merged commit 6c09cc9 into apache:trunk Feb 1, 2024
@dajac dajac deleted the KAFKA-16189 branch February 1, 2024 08:31
@dajac

dajac commented Feb 1, 2024

Copy link
Copy Markdown
Member Author

@nizhikov FYI - I merged this one.

yyu1993 pushed a commit to yyu1993/kafka that referenced this pull request Feb 15, 2024
…e#15253)

This patch extends the Admin client to support describing new consumer groups with the ConsumerGroupDescribe API introduced in KIP-848. Users will continue to use the `Admin#describeConsumerGroups` API. The admin client does all the magic. Basically, the admin client always tries to describe the requested groups with the ConsumerGroupDescribe API to start with. If all the groups are there, great, the job is done. If there are groups unresolved groups due to a UNSUPPORTED_VERSION or GROUP_ID_NOT_FOUND error, the admin client tries with the DescribeGroups API. The patch also adds fields to the data structure returned by `Admin#describeConsumerGroups` as stated in the KIP.

Reviewers: Andrew Schofield <aschofield@confluent.io>, Bruno Cadonna <bruno@confluent.io>
clolov pushed a commit to clolov/kafka that referenced this pull request Apr 5, 2024
…e#15253)

This patch extends the Admin client to support describing new consumer groups with the ConsumerGroupDescribe API introduced in KIP-848. Users will continue to use the `Admin#describeConsumerGroups` API. The admin client does all the magic. Basically, the admin client always tries to describe the requested groups with the ConsumerGroupDescribe API to start with. If all the groups are there, great, the job is done. If there are groups unresolved groups due to a UNSUPPORTED_VERSION or GROUP_ID_NOT_FOUND error, the admin client tries with the DescribeGroups API. The patch also adds fields to the data structure returned by `Admin#describeConsumerGroups` as stated in the KIP.

Reviewers: Andrew Schofield <aschofield@confluent.io>, Bruno Cadonna <bruno@confluent.io>
Phuc-Hong-Tran pushed a commit to Phuc-Hong-Tran/kafka that referenced this pull request Jun 6, 2024
…e#15253)

This patch extends the Admin client to support describing new consumer groups with the ConsumerGroupDescribe API introduced in KIP-848. Users will continue to use the `Admin#describeConsumerGroups` API. The admin client does all the magic. Basically, the admin client always tries to describe the requested groups with the ConsumerGroupDescribe API to start with. If all the groups are there, great, the job is done. If there are groups unresolved groups due to a UNSUPPORTED_VERSION or GROUP_ID_NOT_FOUND error, the admin client tries with the DescribeGroups API. The patch also adds fields to the data structure returned by `Admin#describeConsumerGroups` as stated in the KIP.

Reviewers: Andrew Schofield <aschofield@confluent.io>, Bruno Cadonna <bruno@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

KIP-848 The Next Generation of the Consumer Rebalance Protocol

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants