-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
When the running memory size of jm and tm is increased through the configuration in confProp, jobmanager.memory.process.size and taskmanager.memory.process.size only support the input of integer parameters, and the unit is bytes. When the transferred value exceeds Integer.MAX_ VALUE (2147483647) will throw the exception;
java.lang.NumberFormatException: For input string: "4294967296"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:583)
at java.lang.Integer.parseInt(Integer.java:615)
What you expected to happen
It is expected that the parameter can have units, such as g, m, k, etc
How to reproduce
flinkx -jobName "test_hdfs2doris_with_parquet"
-mode yarn-per-job
-jobType sync
-partitionKey dt
-partitionValue $[yyyy-MM-dd-1]
-job $pathDir"/HIVE2DORIS_20221212165533.json"
-flinkxDistDir /data/service/flinkx/flinkx-dist
-flinkConfDir /data/service/flink/conf
-flinkLibDir /data/service/flink/lib
-confProp "{"jobmanager.memory.process.size":"2048m","taskmanager.memory.process.size":"2048m"}" \
or
flinkx -jobName "test_hdfs2doris_with_parquet"
-mode yarn-per-job
-jobType sync
-partitionKey dt
-partitionValue $[yyyy-MM-dd-1]
-job $pathDir"/HIVE2DORIS_20221212165533.json"
-flinkxDistDir /data/service/flinkx/flinkx-dist
-flinkConfDir /data/service/flink/conf
-flinkLibDir /data/service/flink/lib
-confProp "{"jobmanager.memory.process.size":4294967296,"taskmanager.memory.process.size":4294967296}" \
Anything else
No response
Version
1.12_release
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct