[Feature][kafka] kafka support partition by, add partitionStrategy parameter, optional value all-to-zero or hash-by-key, default value all-to-zero #1954
Description
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
According to the key of kafka, hash is distributed to different partitions,
Add parameter : "partitionStrategy" : "hash-by-key"
Use case
{
"job": {
"content": [
{
"reader": {
"parameter": {
"schema": "wujuan",
"username": "zhiqiang",
"password": "lizhiqiang",
"cat": "insert,delete,update",
"jdbcUrl": "jdbc:mysql://localhost:3306/wujuan?useSSL=false",
"host": "localhost",
"port": 3306,
"table": [],
"splitUpdate": true,
"pavingData": true
},
"name": "binlogreader"
},
"writer": {
"parameter": {
"producerSettings": {
"zookeeper.connect": "",
"bootstrap.servers": "localhost:9092"
},
"dataCompelOrder": false,
"partitionStrategy" : "hash-by-key",
"topic": "wujuan4"
},
"name": "kafkawriter",
"type": 26
}
}
],
"setting": {
"restore": {
"isRestore": true,
"isStream": true
},
"errorLimit": {},
"speed": {
"readerChannel": 1,
"writerChannel": 1,
"bytes": -1048576,
"channel": 1
}
}
}
}
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct