Reviewers: Ismael Juma <ismael@juma.me.uk>

…he listTopics() API (apache#289)

This is a squash of 2 commits:
* [LI-HOTFIX] Exclude the partitions info in the MetadataResponse for the listTopics() API (apache#183)
* [LI-HOTFIX] Making the ExcludePartitions field in MetadataRequest an optional field (apache#193)

Compared to the original implementation, which uses the topic names directly during `excludePartitions = true`, this new implementation will still visit the metadata cache in order to obtain the topic Id info.

==1st commit message==

[LI-HOTFIX] Exclude the partitions info in the MetadataResponse for the listTopics() API (apache#183)

TICKET = LIKAFKA-37320
LI_DESCRIPTION = Making the listTopics() API more light-weight so that a kafka broker would only return the topic names in the MetadataResponse. The partitions and replicas info would not be included in the MetadataResponse for the listTopics() API.

EXIT_CRITERIA = When this change is merged in upstream and the corresponding change is pulled into a future release.

==2nd commit message==

[LI-HOTFIX] Making the ExcludePartitions field in MetadataRequest an optional field (apache#193)

TICKET = LIKAFKA-37320
LI_DESCRIPTION =
After the change made in b28666d,
an old client version sending a MetadataRequest without the "ExcludePartitions"
field to a new kafka server version expecting the field will result in the exception below.
This PR turns the "ExcludePartitions" field into an optional one introduced in KIP-482.

org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: METADATA, apiVersion: 9, connectionId: 10.154.165.183:16637-10.154.87.109:44876-2848, listenerName: ListenerName(SSL), principal: User:likafka-cruise-control:None:prod-ltx1
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:70) ~[kafka-clients-2.4.1.26.jar:?]
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:89) ~[kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:928) ~[kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.$anonfun$processCompletedReceives$1$adapted(SocketServer.scala:909) ~[kafka_2.12-2.4.1.26.jar:?]
        at scala.collection.Iterator.foreach(Iterator.scala:941) [scala-library-2.12.10.jar:?]
        at scala.collection.Iterator.foreach$(Iterator.scala:941) [scala-library-2.12.10.jar:?]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) [scala-library-2.12.10.jar:?]
        at scala.collection.IterableLike.foreach(IterableLike.scala:74) [scala-library-2.12.10.jar:?]
        at scala.collection.IterableLike.foreach$(IterableLike.scala:73) [scala-library-2.12.10.jar:?]
        at scala.collection.AbstractIterable.foreach(Iterable.scala:56) [scala-library-2.12.10.jar:?]
        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:909) [kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.run(SocketServer.scala:799) [kafka_2.12-2.4.1.26.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field '_tagged_fields': java.nio.BufferUnderflowException
        at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:118) ~[kafka-clients-2.4.1.26.jar:?]
        at org.apache.kafka.common.protocol.ApiKeys.parseRequest(ApiKeys.java:327) ~[kafka-clients-2.4.1.26.jar:?]
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:65) ~[kafka-clients-2.4.1.26.jar:?]
        ... 12 more

EXIT_CRITERIA = When this change is merged in upstream and the corresponding change is pulled into a future release.

Co-authored-by: Lucas Wang <luwang@linkedin.com>

…he listTopics() API (apache#289)

This is a squash of 2 commits:
* [LI-HOTFIX] Exclude the partitions info in the MetadataResponse for the listTopics() API (apache#183)
* [LI-HOTFIX] Making the ExcludePartitions field in MetadataRequest an optional field (apache#193)

Compared to the original implementation, which uses the topic names directly during `excludePartitions = true`, this new implementation will still visit the metadata cache in order to obtain the topic Id info.

==1st commit message==

[LI-HOTFIX] Exclude the partitions info in the MetadataResponse for the listTopics() API (apache#183)

TICKET = LIKAFKA-37320
LI_DESCRIPTION = Making the listTopics() API more light-weight so that a kafka broker would only return the topic names in the MetadataResponse. The partitions and replicas info would not be included in the MetadataResponse for the listTopics() API.

EXIT_CRITERIA = When this change is merged in upstream and the corresponding change is pulled into a future release.

==2nd commit message==

[LI-HOTFIX] Making the ExcludePartitions field in MetadataRequest an optional field (apache#193)

TICKET = LIKAFKA-37320
LI_DESCRIPTION =
After the change made in b28666d,
an old client version sending a MetadataRequest without the "ExcludePartitions"
field to a new kafka server version expecting the field will result in the exception below.
This PR turns the "ExcludePartitions" field into an optional one introduced in KIP-482.

org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: METADATA, apiVersion: 9, connectionId: 10.154.165.183:16637-10.154.87.109:44876-2848, listenerName: ListenerName(SSL), principal: User:likafka-cruise-control:None:prod-ltx1
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:70) ~[kafka-clients-2.4.1.26.jar:?]
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:89) ~[kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:928) ~[kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.$anonfun$processCompletedReceives$1$adapted(SocketServer.scala:909) ~[kafka_2.12-2.4.1.26.jar:?]
        at scala.collection.Iterator.foreach(Iterator.scala:941) [scala-library-2.12.10.jar:?]
        at scala.collection.Iterator.foreach$(Iterator.scala:941) [scala-library-2.12.10.jar:?]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) [scala-library-2.12.10.jar:?]
        at scala.collection.IterableLike.foreach(IterableLike.scala:74) [scala-library-2.12.10.jar:?]
        at scala.collection.IterableLike.foreach$(IterableLike.scala:73) [scala-library-2.12.10.jar:?]
        at scala.collection.AbstractIterable.foreach(Iterable.scala:56) [scala-library-2.12.10.jar:?]
        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:909) [kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.run(SocketServer.scala:799) [kafka_2.12-2.4.1.26.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field '_tagged_fields': java.nio.BufferUnderflowException
        at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:118) ~[kafka-clients-2.4.1.26.jar:?]
        at org.apache.kafka.common.protocol.ApiKeys.parseRequest(ApiKeys.java:327) ~[kafka-clients-2.4.1.26.jar:?]
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:65) ~[kafka-clients-2.4.1.26.jar:?]
        ... 12 more

EXIT_CRITERIA = When this change is merged in upstream and the corresponding change is pulled into a future release.

Co-authored-by: Lucas Wang <luwang@linkedin.com>

…he listTopics() API (apache#289)

This is a squash of 2 commits:
* [LI-HOTFIX] Exclude the partitions info in the MetadataResponse for the listTopics() API (apache#183)
* [LI-HOTFIX] Making the ExcludePartitions field in MetadataRequest an optional field (apache#193)

Compared to the original implementation, which uses the topic names directly during `excludePartitions = true`, this new implementation will still visit the metadata cache in order to obtain the topic Id info.

==1st commit message==

[LI-HOTFIX] Exclude the partitions info in the MetadataResponse for the listTopics() API (apache#183)

TICKET = LIKAFKA-37320
LI_DESCRIPTION = Making the listTopics() API more light-weight so that a kafka broker would only return the topic names in the MetadataResponse. The partitions and replicas info would not be included in the MetadataResponse for the listTopics() API.

EXIT_CRITERIA = When this change is merged in upstream and the corresponding change is pulled into a future release.

==2nd commit message==

[LI-HOTFIX] Making the ExcludePartitions field in MetadataRequest an optional field (apache#193)

TICKET = LIKAFKA-37320
LI_DESCRIPTION =
After the change made in b28666d,
an old client version sending a MetadataRequest without the "ExcludePartitions"
field to a new kafka server version expecting the field will result in the exception below.
This PR turns the "ExcludePartitions" field into an optional one introduced in KIP-482.

org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: METADATA, apiVersion: 9, connectionId: 10.154.165.183:16637-10.154.87.109:44876-2848, listenerName: ListenerName(SSL), principal: User:likafka-cruise-control:None:prod-ltx1
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:70) ~[kafka-clients-2.4.1.26.jar:?]
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:89) ~[kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:928) ~[kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.$anonfun$processCompletedReceives$1$adapted(SocketServer.scala:909) ~[kafka_2.12-2.4.1.26.jar:?]
        at scala.collection.Iterator.foreach(Iterator.scala:941) [scala-library-2.12.10.jar:?]
        at scala.collection.Iterator.foreach$(Iterator.scala:941) [scala-library-2.12.10.jar:?]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) [scala-library-2.12.10.jar:?]
        at scala.collection.IterableLike.foreach(IterableLike.scala:74) [scala-library-2.12.10.jar:?]
        at scala.collection.IterableLike.foreach$(IterableLike.scala:73) [scala-library-2.12.10.jar:?]
        at scala.collection.AbstractIterable.foreach(Iterable.scala:56) [scala-library-2.12.10.jar:?]
        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:909) [kafka_2.12-2.4.1.26.jar:?]
        at kafka.network.Processor.run(SocketServer.scala:799) [kafka_2.12-2.4.1.26.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field '_tagged_fields': java.nio.BufferUnderflowException
        at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:118) ~[kafka-clients-2.4.1.26.jar:?]
        at org.apache.kafka.common.protocol.ApiKeys.parseRequest(ApiKeys.java:327) ~[kafka-clients-2.4.1.26.jar:?]
        at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:65) ~[kafka-clients-2.4.1.26.jar:?]
        ... 12 more

EXIT_CRITERIA = When this change is merged in upstream and the corresponding change is pulled into a future release.

Co-authored-by: Lucas Wang <luwang@linkedin.com>

Single bulk apply of the Language Engine's IntelliJ-style format
profile across the kafka source tree. Pairs with the IntelliJ-real
control branch `intellij-formatting` for side-by-side comparison.

Engine state at apply time includes the following format fixes
landed against the language-engine repo:

  Pre-PR #11:
  - PR apache#275 multi-line // comment indent group
  - PR apache#276 BLANK_LINES_AROUND_CLASS + sibling no-op repair
  - PR apache#277 string-concat chain anchor preservation
  - PR apache#279 forward style through apply routes
  - PR apache#281 Result-tab line-number alignment
  - PR apache#282 spaces after return / throw / yield / instanceof
  - PR apache#283 chain-dot postfix preservation
  - PR apache#284 BlankLines sibling minimum.* conversion
  - PR apache#285 chain-dot single-anchor (N=1) preservation
  - PR apache#286 string-concat chain partial-cascade fix
  - PR apache#287 method-decl param re-align for misaligned source
  - PR apache#288 annotation-in-array-init indent

  Post-original-PR #11 (new in this re-apply):
  - PR apache#289 partial-cascade post-rewrite-anchor (4 shape fixes:
    chain-dot postfix follow-on, MI/NewClass close-paren cascade,
    ternary continuation, lambda body continuation)

Stats: 3101 files written, 0 failures, 2678 already-idempotent
files skipped (no_change). Corpus byte delta vs trunk: -510 bytes
(106275 insertions / 106785 deletions).