3.2.2 release cherry-pick cp: Reduce the size of tarball to continue ASF release#16042
3.2.2 release cherry-pick cp: Reduce the size of tarball to continue ASF release#16042wangxj3 merged 20 commits intoapache:3.2.2-preparefrom
Conversation
* Fix rpc channel leak due to concurrent operation * Throw channel create failed exception --------- Co-authored-by: Rick Cheng <rickchengx@gmail.com>
…apache#15540) * Reduce the size of tarball to continue ASF release for more detail you can see https://lists.apache.org/thread/rmp7fghlj0n7h9y2v3p8gkw9f9qbo6qt # Conflicts: # tools/dependencies/known-dependencies.txt
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
…ers (apache#16033) Co-authored-by: Rick Cheng <rickchengx@gmail.com>
…/project parameters data type (apache#15967) * [Improvement][parameter] New data types and type filtering * [Improvement][parameter] Improvement startup parameters/global parameters data type * fix api interfaces compatible * add project parameter data type default value * [Improvement][parameter] New data types and type filtering * [Improvement][parameter] Improvement startup parameters/global parameters data type * fix api interfaces compatible * add project parameter data type default value * improvement project code * remove useless imports * remove method onClearSearchTaskType * add parameter doc * optimisation logic * code conflict resolution * code conflict resolution * [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (apache#15978) * update * test * add monitor enhance ui * update * update * update doc * fix spotless * update * update * Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java Co-authored-by: Wenjun Ruan <wenjun@apache.org> * Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java Co-authored-by: Wenjun Ruan <wenjun@apache.org> * Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java Co-authored-by: Wenjun Ruan <wenjun@apache.org> * Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml Co-authored-by: Wenjun Ruan <wenjun@apache.org> * Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java Co-authored-by: Wenjun Ruan <wenjun@apache.org> * Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml Co-authored-by: Wenjun Ruan <wenjun@apache.org> * update * fix spotless * update --------- Co-authored-by: Wenjun Ruan <wenjun@apache.org> * [Improvement][Monitor] Add UT for montor (apache#15998) * formatting Code
| # This is the version number of the application being deployed. This version number should be | ||
| # incremented each time you make changes to the application. | ||
| appVersion: 3.1.0 | ||
| appVersion: 3.2.2 |
There was a problem hiding this comment.
I don't think this appVersion should be changed.
dolphinscheduler.master.labels quotes appVersion
And dolphinscheduler.master.labels as master's matchLabels, but matchLabels are immutable, which means that, Updating helm charts for an already deployed dolphinscheduler will fail
There was a problem hiding this comment.
If we still want to modify the appVersion, then we should remove the app.kubernetes.io/version: {{ .Chart.AppVersion }} from dolphinscheduler.common.labels first
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpclient</artifactId> | ||
| <version>4.5.13</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpmime</artifactId> | ||
| <version>4.5.13</version> | ||
| </dependency> |
There was a problem hiding this comment.
I don't see any related test cases being added. Why add this?
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-all</artifactId> | ||
| <artifactId>dolphinscheduler-task-all-prune</artifactId> |
There was a problem hiding this comment.
reduce release package size
Co-authored-by: xiangzihao <460888207@qq.com>
Fix the PreparedStatement parameter is TIME, set it to java.sql.Time Co-authored-by: Aaron Wang <wangweirao16@gmail.com> Co-authored-by: Rick Cheng <rickchengx@gmail.com>
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-datax</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-flink</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-flink-stream</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-http</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-mr</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-pigeon</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-procedure</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-python</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-seatunnel</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-shell</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-spark</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-sql</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-sqoop</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-emr</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-zeppelin</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-jupyter</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-k8s</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-mlflow</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-openmldb</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-dvc</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-dinky</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-java</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-sagemaker</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-pytorch</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-hivecli</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-chunjun</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-dms</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-datasync</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-kubeflow</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-linkis</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-datafactory</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.dolphinscheduler</groupId> | ||
| <artifactId>dolphinscheduler-task-remoteshell</artifactId> | ||
| <version>${project.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>*</groupId> | ||
| <artifactId>*</artifactId> | ||
| </exclusion> | ||
| </exclusions> |
There was a problem hiding this comment.
If we do this, please inform the user in the documentation that this is an incompatible change. This may lead to serious production accidents. cc @ruanwenjun @EricGao888 @Radeity
There was a problem hiding this comment.
Yes, this is a sad solution from 3.2.1, since no-one want to refactor the release architecture. Right now, the release package size is 1.x g, large than ASF limit. I create #15960 to do this.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 3.2.2-prepare #16042 +/- ##
===================================================
+ Coverage 40.58% 40.66% +0.07%
- Complexity 5216 5225 +9
===================================================
Files 1380 1380
Lines 46056 46011 -45
Branches 4916 4919 +3
===================================================
+ Hits 18693 18710 +17
+ Misses 25437 25376 -61
+ Partials 1926 1925 -1 ☔ View full report in Codecov by Sentry. |
|
Please retry analysis of this Pull-Request directly on SonarCloud |

Purpose of the pull request
It is prepared for version 3.2.2, mainly to reduce the size of the release package and fix the helm version problem.
Brief change log
Verify this pull request
This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(or)
If your pull request contain incompatible change, you should also add it to
docs/docs/en/guide/upgrede/incompatible.md