changefeedccl: allow topic_name parameter for changefeed kafka sinks#62377
changefeedccl: allow topic_name parameter for changefeed kafka sinks#62377craig[bot] merged 1 commit intocockroachdb:masterfrom
Conversation
|
This change is |
miretskiy
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB and @stevendanna)
pkg/ccl/changefeedccl/changefeedbase/options.go, line 95 at r1 (raw file):
distinguishing messages using metadata.
This patch allows the ?topic_name=foo parameter to be added
to Kafka sink URIs.
Question:
Is this truly only applicable to kafka?
If so, perhaps parameter should be called kafka_topic_name?
|
pkg/ccl/changefeedccl/changefeedbase/options.go, line 95 at r1 (raw file): Previously, miretskiy (Yevgeniy Miretskiy) wrote…
It is currently only applicable to Kafka, as is topic_prefix which is a name we're stuck with. It does seem like it could potentially generalize to other sinks? Not sure. |
miretskiy
left a comment
There was a problem hiding this comment.
Reviewed 1 of 4 files at r1.
Reviewable status:
stevendanna
left a comment
There was a problem hiding this comment.
Reviewable status:
pkg/ccl/changefeedccl/sink.go, line 370 at r1 (raw file):
topics := make(map[descpb.ID]string) useSingleName := false if useSingleName = (name != ""); useSingleName {
[nit] Since useSingleName is used outside this if, might it be more clear to initialize it as:
useSingleName := (name != "")at 369?
stevendanna
left a comment
There was a problem hiding this comment.
Reviewed 4 of 4 files at r1.
Reviewable status:
Previously, changes for a table went to a Kafka topic named for that table, with users only able to specify a prefix. Some users, however, need changes to go to a specific topic, including sometimes the same one for more than one table, distinguishing messages using metadata. This patch allows the `?topic_name=foo` parameter to be added to Kafka sink URIs. This will override the per-table topic generation, so that changes for every table will all go to the specified topic. It may be used in conjunction with topic_prefix, although the distinction is not meaningful. Release note (enterprise change): Kafka sink URIs now accept the "topic_name" parameter to override per-table topic names.
e640e79 to
296223c
Compare
|
pkg/ccl/changefeedccl/sink.go, line 370 at r1 (raw file): Previously, stevendanna (Steven Danna) wrote…
Yup, my way made no sense. Fixed. |
HonoreDB
left a comment
There was a problem hiding this comment.
Reviewable status:
pkg/ccl/changefeedccl/sink.go, line 370 at r1 (raw file):
Previously, HonoreDB (Aaron Zinger) wrote…
Yup, my way made no sense. Fixed.
Done.
|
bors r=[stevendanna, miretskiy] |
|
Build failed (retrying...): |
|
Build failed (retrying...): |
|
Build succeeded: |
Previously, changes for a table went to a Kafka topic
named for that table, with users only able to specify a prefix.
Some users, however, need changes to go to a specific topic,
including sometimes the same one for more than one table,
distinguishing messages using metadata.
This patch allows the
?topic_name=fooparameter to be addedto Kafka sink URIs. This will override the per-table topic
generation, so that changes for every table will all go to
the specified topic. It may be used in conjunction with
topic_prefix, although the distinction is not meaningful.Release note (enterprise change): Kafka sink URIs now accept
the "topic_name" parameter to override per-table topic names.
Closes #59300
Closes #58302