MINOR: clarify node grouping of input topics using pattern subscription#7793
Conversation
vvcephei
left a comment
There was a problem hiding this comment.
Thanks for the improvement! Just one small piece of feedback on the wording.
| topics do not match the configured default SerDes. For information about configuring default SerDes, available | ||
| SerDes, and implementing your own custom SerDes see <a class="reference internal" href="datatypes.html#streams-developer-guide-serdes"><span class="std std-ref">Data Types and Serialization</span></a>.</p> | ||
| <p class="last">Several variants of <code class="docutils literal"><span class="pre">stream</span></code> exist, for example to specify a regex pattern for input topics to read from).</p> | ||
| <p class="last">Several variants of <code class="docutils literal"><span class="pre">stream</span></code> exist. For example, you can specify a regex pattern for input topics to read from (note that a single nodegroup will be created for all matching topics, and therefore the number of tasks will scale with the maximum partition count rather than the total number of partitions).</p> |
There was a problem hiding this comment.
Is nodegroup already a defined term by this point? Looking at this and the other changes, this might be kind of a confusing usage of the word "node". Maybe just make up something more intuitive, like "input topic group"?
There was a problem hiding this comment.
Yeah, that's fair. I didn't know what that was when I first saw it in the code. What do you think about using "subtopology" here?
Alternatively, how about: note that all matching topics will be part of the same input topic group, and the work will not be parallelized for different topics if subscribed to in this way
There was a problem hiding this comment.
I think your "alternative" is the best phrasing yet.
There was a problem hiding this comment.
Done. Rephrased the javadocs as well -- thanks!
|
Since this is a docs change only, I'm just going to go ahead and merge it. Just to be sure, I built and tested it locally. |
See also Apache Kafka apache/kafka#7793 Reviewers: John Roesler <vvcephei@apache.org>
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
|
Cherry-picked to 1.0, 1.1, 2.0, 2.1, 2.2, and 2.3 . I'm not doing 2.4 right now because the release is pending. |
…on (#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
|
cherry-picked the PR to 2.4 branch. |
…on (apache#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
A user recently pointed out that we don't document how tasks are generated when using pattern subscription for input topics.
We should document this so users can better understand how their application will scale