Skip to content

[Feature][kafka-connector] Does the codec in Kafka play the correct role #1353

@liumengkai

Description

@liumengkai

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

Now the kafka sink has a parameter 'codec',It is set by default to 'json',chunjun use JsonDecoder to decode data,and JsonDecoder will return a map that key is 'message'. The data in kafka like this,2 columns value has been splited by ','

{"message":"1344696702,FE3FF851-1409-4621-80C5-8402D40119D0"}

When this parameter is set to 'text' , chunjun will use TextDecoder to decode data, but the TextDecoder will still return a map with 'message' as key. The data in kafka is still like this. It's also a json string

{"message":"1344696702,FE3FF851-1409-4621-80C5-8402D40119D0"}

When we set codec to 'text',I think what we want data in kafka is like this:

1344696702,FE3FF851-1409-4621-80C5-8402D40119D0

Just the value , not a json
How about this? I can make a pr about this change after we discuss.

Use case

When we only need value to sink to kafka not a json

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature-requestthis is a feature requests on the product

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions