Test rebase 1 (#3075) * fix actions/checkout@v2 problem (#2095) * remove incorrect or unnecessary URL tag (#2099) * Add docker internal server check script (#2092) * add environment variables add checkpoint.sh file to docker image 1. add `POSTGRESQL_DATABASE` environment variable 2. add `DOLPHINSCHEDULER_DATA_BASEDIR_PATH` environment variable 3. add `DOLPHINSCHEDULER_DATA_DOWNLOAD_BASEDIR_PATH` environment variable 4. add `DOLPHINSCHEDULER_PROCESS_EXEC_BASEPATH` environment variable 5. add checkpoint.sh file to check server process 6. modify `README.md` and `README_zh_CN.md` * add environment variables add checkpoint.sh file to docker image 1. add `POSTGRESQL_DATABASE` environment variable 2. add `DOLPHINSCHEDULER_DATA_BASEDIR_PATH` environment variable 3. add `DOLPHINSCHEDULER_DATA_DOWNLOAD_BASEDIR_PATH` environment variable 4. add `DOLPHINSCHEDULER_PROCESS_EXEC_BASEPATH` environment variable 5. add checkpoint.sh file to check server process 6. modify `README.md` and `README_zh_CN.md` * add logger port and modify dolphinscheduler_env to dolphinscheduler_env.sh * use stringbuilder to append string (#2108) * fix: remove redundant import class (#2110) * move UT class to reasonable package (#2116) * Adapting partial code(file name start with E) to the sonar cloud rule (#1999) * Adapting partial code(file name start with E) to the sonar cloud rule * remove isEmpty invoke * resolve conflicts * Adapting partial code(file name start with F) to the sonar cloud rule (#2045) * Adapting partial code(file name start with F) to the sonar cloud rule * add more unit test * add License * add includes configuration to maven-surefire-plugin * fix getResourceFilesList incorrect logic * add .asf.yaml file (#2136) Co-authored-by: guirong.ggr * edit README.md modify word from expand to extend (#2138) Co-authored-by: guirong.ggr * fix wrong sonar analysis with checkout V2. (#2148) * docker run failed when docker command was `docker run -d` (#2125) * docker run failed when docker command was `docker run -d` 1. add tini 2. replace `tee` command by `tail` command in startup.sh 3. api-server need to support zookeeper * modify `check.sh` in dockerfile * add init directories and log files * use logger to print exception (#2151) * fix field access modifier (#2150) * fix unequal number of arguments and placeholder (#2149) * Add mail server configuration description information (#2143) * Add mail server configuration description information * Remove mail server configuration note in alert.properties * Limit customization file content to no more than 3000 lines(#2128) (#2141) * Change DOM label * Change name to lowercase * Limit customization file content to no more than 3000 lines * fix: The toolbar buttons of Dag disable bug (#2124) * Adapting partial code(file name start with H) to the sonar cloud rule (#2145) * Change DOM label * add input cache for sqoop and conditions tasks (#2118) Co-authored-by: zhukai * fix bug: data too long for column 'app_link' (#2146) * fix bug: data to long for app_link field * change app_link field length in 1.2.2 ddl * change app_link field type to text * Delete .asf.yaml (#2158) * Adapting partial code(file name start with I) to the sonar cloud rule (#2155) * fix rpmbuild and DS Ambari plugin bug (#2144) * Update quick-start.md * 更新demo的地址 * add rpm plugin * delete the git ignore files * 测试git ignore * 测试ignore * optimize the .gitignore file * add rpm package, delete unuse copy jar action in assembly * delete docs dir * add rpm plugin support * add rpm plugin support * add rpm plugin support * dolphinscheduler-common have not bin dir * dolphinscheduler-common have not bin dir * delete unuse config about assembly * add defineSatement * add rpm plugin support * add rpm plugin support * add rpm plugin support * add rpm support * update install dirname , make front and backend together * update rpm name * update rpm name * update rpm config * add jars excludes * add jars excludes * add rpm plugion * add rpm plugion * add rpm plugion * add auto create /opt/soft * The install dir can not be fixed to apache-dolphinscheduler-incubating-1.2.1 , its version is 1.2.2 * install from rpm will create a link * the ui install thrown rpm only support jetty not nginx * the ui install thrown rpm only support jetty not nginx * sql目录里加上soft_version文件 * env dir need in conf dir * remove no use module * add module * updaate * rpmbuild add dolphinscheduler-service module * Toolbar of the DAG has not prompt (#2161) * Toolbar of the DAG has not prompt * Remove duplication code * use static method in static class JSON (#2156) * Create .asf.yaml (#2166) * Update .asf.yaml (#2167) * Delete .asf.yaml (#2168) * The master and worker server exit exception #2163 (#2176) * fix: #2163 * fix: format * Adapting partial code(file name start with J) to the sonar cloud rule (#2165) * Adapting partial code(file name start with J) to the sonar cloud rule * fix unit failure * fix: NullPointException in the case of task queuing (#2175) * Adapting partial code(file name start with L) to the sonar cloud rule (#2171) * Adapting partial code(file name start with M #1) to the sonar cloud rule (#2173) * Adapting partial code(file name start with M #1) to the sonar cloud rule * remove unused import * Cache HadoopUtils instance with specific days expire time (#2181) * Cache HadoopUtils instance with 7 days expire time * solve sonar issue * add kerberos expire time config * move KERBEROS_EXPIRE_TIME to Constants.java * Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. (#2193) * change static field to instance field (#2190) * fix ResultSet not close and reformat code (#2183) * fix ResultSet not close * fix ResultSet not close * extract code of closing resource to a method * remove redundant if condition * modify e2e timeout * fix: Api server startup fail (#2198) * Adapting partial code(file name start with M #2) to the sonar cloud rule (#2179) * Adapting partial code(file name start with M #2) to the sonar cloud rule * remove unused import * Optimize PropertyUtils instantiation. (#2199) * Optimize PropertyUtils instantiation. * Fix info error. * e2e add the case of deleting user and tenant (#2210) * add e2e * add LoginTest license (#2217) Co-authored-by: chenxingchun <438044805@qq.com> * dd branch flow node verification * Adapting partial code(file name start with N) to the sonar cloud rule (#2215) * add repace time when run history job and batch complement job ,not current time (#2196) * add release notes in 1.2.1 (#1654) * fix Monitor bug (#1656) * fix Monitor bug * fix zk monitor bug * fix api url (#1673) * get root path from zookeeper config * Fixed DAG zoom in and zoom out nodes separated from arrows #1679 (#1685) * fix api url * Fixed DAG zoom in and zoom out nodes separated from arrows #1679 * check license and update (#1725) * Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 (#1734) Co-authored-by: elonlo * upgrade jackson from 2.9.8 to 2.10.1 (#1767) * add out put log when master/worker server start (#1769) * merge hadoop.properties into common.properties * merge hadoop,zookeeper.properties into common.properties remove combined.properties/master.properties/worker.properties * change db user/pwd to test/test * rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh remove some unused in install.sh * add out put log when master/worker server start... * add start log when servers start * add check download resource permission in order to fix issues 1770 (#1788) * merge dev branch front-end code (#1786) * fix #1775 bug，delete process definition when process instance is running (#1790) * fix #1775 bug，delete process definition when process instance is running * revert CONTRIBUTING.md * fix udfs assignment and task instance page jump #1789 (#1791) * merge dev branch front-end code * fix udfs assignment * Fix task instance page jump * fix udfs assignment and task instance page jump #1789 * update method checkDownloadPermission in order to fix issues 1770 (#1794) * add check download resource permission in order to fix issues 1770 * update method checkDownloadPermission in order to fix issues 1770 * update method listUnauthorizedResource in order to fix issues 1770 * update method listUnauthorizedResource in order to fix issues 1770 (#1797) * if login user is admin,it will has permission to query all udf function (#1799) * Fixed space and icon display issues before and after the input box (#1798) * merge dev branch front-end code * fix udfs assignment * Fix task instance page jump * fix udfs assignment and task instance page jump #1789 * Fixed space and icon display issues before and after the input box * add license * add license * donot submit task to queue if sub process (#1793) * dockerfile modify (#1800) * fix #1775 bug，delete process definition when process instance is running * revert CONTRIBUTING.md * dockerfile modify * dockerfile modify * dockerfile modify * Fix data echo, style and popup cannot be closed (#1801) * merge dev branch front-end code * fix udfs assignment * Fix task instance page jump * fix udfs assignment and task instance page jump #1789 * Fixed space and icon display issues before and after the input box * add license * add license * Fix data echo, style and popup cannot be closed * Fix page number loading issue and dag not getting value(#1810) (#1815) * merge dev branch front-end code * fix udfs assignment * Fix task instance page jump * fix udfs assignment and task instance page jump #1789 * Fixed space and icon display issues before and after the input box * add license * add license * Fix data echo, style and popup cannot be closed * Fix page number loading issue and dag not getting value * [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1817) * if login user is admin,it will has permission to query all udf function * check udf and data source in order to fix issue 1770 * check udf and data source in order to fix issue 1770 * check udf and data source in order to fix issue 1770 * check udf and data source in order to fix issue 1770 * check udf and data source in order to fix issue 1770 * revert common.properties * update the test method name * remove useless code and import in unit test * refactor has permission and check permission * #1813 remove "_001" after the master/server register path in zookeepe (#1824) * donot submit task to queue if sub process * [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820) * change master/worker register path. * remove "_" from register path. * remove install.sh server.servlet.context-path replace (#1823) * fix #1775 bug，delete process definition when process instance is running * revert CONTRIBUTING.md * dockerfile modify * dockerfile modify * dockerfile modify * remove install.sh server.servlet.context-path replace * Fix DAG add dependent project value exception # 1816 and keep requesting task interface (#1827) * merge dev branch front-end code * fix udfs assignment * Fix task instance page jump * fix udfs assignment and task instance page jump #1789 * Fixed space and icon display issues before and after the input box * add license * add license * Fix data echo, style and popup cannot be closed * Fix page number loading issue and dag not getting value * Fix DAG add dependent project value exception # 1816 and keep requesting task interface * rpm package add UI (#1846) * [fix #1828] when the executor of process instance is not the owner of udf resouce, the path of the read resource file is incorrect (#1847) * fix issue 1828:get the udf resource path error when create udf function * update grantResources * first verify whether udf resource is bound by udf function * update grantResources * update testListAuthorizedUdfFunc * update getUserInfo in order to run success * add LoggerServer RPC PORT modified. #1848 (#1849) * fix #1775 bug，delete process definition when process instance is running * revert CONTRIBUTING.md * dockerfile modify * dockerfile modify * dockerfile modify * remove install.sh server.servlet.context-path replace * add LoggerServer RPC PORT modified. #1848 * LoggerService Logger RPC PORT get Error modify . #1848 (#1857) * fix #1775 bug，delete process definition when process instance is running * revert CONTRIBUTING.md * dockerfile modify * dockerfile modify * dockerfile modify * remove install.sh server.servlet.context-path replace * add LoggerServer RPC PORT modified. #1848 * LoggerService Logger RPC PORT get Error modify . #1848 * Fix UDF function list delete data without refresh and page data display widened #1851 (#1852) * merge dev branch front-end code * fix udfs assignment * Fix task instance page jump * fix udfs assignment and task instance page jump #1789 * Fixed space and icon display issues before and after the input box * add license * add license * Fix data echo, style and popup cannot be closed * Fix page number loading issue and dag not getting value * Fix DAG add dependent project value exception # 1816 and keep requesting task interface * Fix UDF function list delete data without refresh and page data display widened * [Fix #1828]check whether has permission to download udf file or delete udf function (#1858) * fix issue 1828:get the udf resource path error when create udf function * update grantResources * first verify whether udf resource is bound by udf function * update grantResources * update testListAuthorizedUdfFunc * update getUserInfo in order to run success * check whether has permission to download udf file or delete udf file * update listAuthorizedResourceById in ResourceMapper.xml * add log, don't swallow exception info (#1877) * Added hints in Resource Center (#1891) * update jackson version from 2.9.8 to 2.10.1 (#1892) * update release notes and modify the plugin of rat (#1899) * modify how to build (#1902) * update release notes and modify the plugin of rat * modify how to build * [maven-release-plugin] prepare release 1.2.1 * [maven-release-plugin] prepare for next development iteration * get authorized udf resources need exclude all created by self (#1905) * get authorized udf resources need exclude all created by self * reset pom version to 1.2.1-SNAPSHOT * Update ReleaseNotes.md add detail info * Update ReleaseNotes.md * update notice year form 2019 to 2019-2020 (#1907) * [maven-release-plugin] prepare release 1.2.1 * [maven-release-plugin] prepare for next development iteration * 当手动重跑历史活着补数的时候对shell和sql脚本中的[YYYYmmddd...]变量赋值与传递的日期，而不是当前日期 Co-authored-by: lgcareer <18610854716@163.com> Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com> Co-authored-by: break60 <790061044@qq.com> Co-authored-by: Tboy Co-authored-by: elonlo Co-authored-by: qiaozhanwei Co-authored-by: zhangchunyang <18910529250@163.com> Co-authored-by: dailidong Co-authored-by: 谷雨 * datax * Refactor ConnectionFactory to static inner holder Singleton (#2204) * refactor ConnectionFactory to static inner holder Singleton * remove redundant import * make getSqlSessionFactory() method private * fix sonar issue * remove @Ignore of MailUtilsTest * add MailUtilsTest to pom.xml to test * fix MailUtilsTest path error in pom.xml * modify test method name * add assert and logger to unit test * add log * add log * add AlertDaoTest * move AlertDaoTest to new module * modify test in pom.xml * remove unnecessary log * support custom datax configuration (#2224) * fix #1441 * support custom datax config * support datax custom config * support datax custom config * support datax custom config Co-authored-by: 张世鸣 * Optimizing code (#2240) * updated * optimized the code * Delete ParameterUtils.java * rollback source code * rollback source code * rollback source code Co-authored-by: dailidong * fix blocker problem (#2244) * fix blocker problem * use error level to log exception * feat:optimizing code (#2245) * feat:optimizing code * add licence and format code * Adapting partial code(file name start with R) to the sonar cloud rule (#2243) * fix：Depend check bug (#2229) * fix：#1787 * fix：#1787 * Adapting partial code(file name start with Q) to the sonar cloud rule (#2242) * license and checkoutv2 fix (#2261) * datax add custom * Fix checkoutv2 push trigger on dev (#2263) * push trigger on dev * fix: No operations allowed after statement closed when running sql task (#2265) * sqlTask failed to run * get correct attribute value and logger content Co-authored-by: songqh * add ds logs and server status check with e2e test (#2277) * Change normalize.scss import method and animation.scss license modification * e2e add project and workflow case (#2309) * add LoginTest license * Delete useless packages * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case Co-authored-by: chenxingchun <438044805@qq.com> * Update test code logger class name (#2303) * Adapting partial code(file name start with W) to the sonar cloud rule (#2287) * Decouple DruidDataSource in ConnectionFactory and cache DataSource instance (#2232) * Decouple DruidDataSource in ConnectionFactory and cache DataSource instance * init dataSource first * fix code smell * add unit test * add unit test * add unit test * add unit test * fix unit test * delete useless logger * Adapting partial code(file name start with T) to the sonar cloud rule (#2271) * resolve shell task build command nullpointexception (#2267) Co-authored-by: sunchaohe * Resource tree code merge * Adapting partial code(file name start with S #2) to the sonar cloud rule (#2270) * new feature for #404 add resource tree function (#2323) * add create resource directory * add create resource directory * update the resource test * add upgrade sql in version 1.2.2 * Adding request parameter id to update queryResourceListPaging * set isDirectory value is false default * add full name to update updateResource * remove request parameter isDirectory to update createResource method * update queryResourceListPaging with change get to post * update updateResource method with remove fullName * File management list modification (#1976) * add resource component * add resource tree visitor * return json string * update queryResourceList * upload file need fullName * add method rootNode * Shell task resources and authorization resources (#1989) * File management list modification * Shell task resources and authorization resources * download resource when execute task * download resource when execute task * update authorization type * download resource when execute task * Spark task resource changes (#1990) * File management list modification * Shell task resources and authorization resources * Spark task resource changes * download resource when execute task * update udf function service * add resource type in ResourceComponent * UDF resource tree and change DAG style (#2019) * File management list modification * Shell task resources and authorization resources * Spark task resource changes * UDF resource tree and change DAG style * add deleteIds method in ResourceMapper and ResourceMapperTest * Add comments on class and method * add queryResourceByName method in controller * update verify-name with change name to full name * update queryResource with add parameter pid * update queryResource with add parameter pid * add resource ids in process definition and delete resource need judge whether it is used by any process definition * Breadcrumb development (#2033) * File management list modification * Shell task resources and authorization resources * Spark task resource changes * UDF resource tree and change DAG style * Breadcrumb development * Breadcrumb development * Resource tree bug fix (#2040) * File management list modification * Shell task resources and authorization resources * Spark task resource changes * UDF resource tree and change DAG style * Breadcrumb development * Breadcrumb development * Resource tree bug fix * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * Fix github action rerun failed (#2067) * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * Change crumb position * Change crumb position (#2068) * build resource process definition map * UDF changed to multiple choice * UDF changed to multiple choice (#2077) * Change crumb position * UDF changed to multiple choice * build resource process definition map (#2076) * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * build resource process definition map * update resource name also need update all the children full name * need add queryResource * update resource name also need update all the children full name (#2096) * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * build resource process definition map * update resource name also need update all the children full name * need add queryResource * Limit customization file content to no more than 3000 lines * Limit customization file content to no more than 3000 lines(#2128) * Limit customization file content to no more than 3000 lines(#2128) (#2140) * Change crumb position * UDF changed to multiple choice * Limit customization file content to no more than 3000 lines * Limit customization file content to no more than 3000 lines(#2128) * add queryResourceJarList * add queryResourceJarList * add queryResourceJarList * add queryResourceJarList (#2192) * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * build resource process definition map * update resource name also need update all the children full name * need add queryResource * add queryResourceJarList * add queryResourceJarList * add queryResourceJarList * Modify the main jar package * Modify the main jar package (#2200) * Change crumb position * UDF changed to multiple choice * Limit customization file content to no more than 3000 lines * Limit customization file content to no more than 3000 lines(#2128) * Modify the main jar package * add resource filter in order to get filtered resource * add comments of resource filter * update list children by resource * choose main jar with resource tree (#2220) * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * build resource process definition map * update resource name also need update all the children full name * need add queryResource * add queryResourceJarList * add queryResourceJarList * add queryResourceJarList * add resource filter in order to get filtered resource * add comments of resource filter * update list children by resource * Return null if query resource list is empty * update queryResource method change parameter pid to id * getResouDelete checksum and modify parameter namerceId * revert .env * remove parameter pid * Delete request interface * go back to the last page * jar interface call * Fix issue #2234 and #2228 * change resource name with full name * Fix issue #2234 and #2228 (#2246) * update resource service test * Fix github action rerun failed * add status of PARENT_RESOURCE_NOT_EXIST * build resource process definition map * update resource name also need update all the children full name * need add queryResource * add queryResourceJarList * add queryResourceJarList * add queryResourceJarList * add resource filter in order to get filtered resource * add comments of resource filter * update list children by resource * Return null if query resource list is empty * update queryResource method change parameter pid to id * revert .env * remove parameter pid * Fix issue #2234 and #2228 * change resource name with full name * Fix list query value error * remove unauth-file with authorize-resource-tree * Repair data cannot be echoed * Repair data cannot be echoed * execute mr and spark task need query resource name before * Authorized resource interface replacement * Authorized resource interface replacement * Filter UDF resources * Change parameters * need query all authorized directory children when create task * Change normalize.scss import method and animation.scss license modification * Delete file list update processing * It's fixed that resource not deleted in hdfs when delete it. * add tooltips * add tooltips (#2310) * Echo workflow name and modify udf management name * [new feature]add resource tree function * revert front code in order to be same as dev branch * revert front code in order to be same as dev branch * revert common.properties and application.properties * add super method * update flink parameter test * update flink parameter and unit test * update resource service test * If resource list is empty,need init it * update flink parameter test Co-authored-by: break60 <790061044@qq.com> Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> Co-authored-by: qiaozhanwei * fix：Optimize content returned by interface (#2294) * Fix SqlTask kerberos load scope unreasonable. #2178 (#2321) * Optimize PropertyUtils instantiation. * Fix info error. * Fix SqlTask kerberos load scope unreasonable. #2178 * Support kubernetes deployment (#2153) * Support kubernetes deployment * Support kubernetes deployment * Remove .helmignore file (#2328) * some annotation optimizations (#2333) Co-authored-by: songqh * fix #2334 (#2335) Co-authored-by: dailidong * Delete ReleaseNotes.md (#2338) * Modify ans-ui version and timing management style * Supplemental licenses and repair resources cannot be renamed * Remove invalid code (#2342) Co-authored-by: zhanglong Co-authored-by: dailidong * fix：mail send fail #2345 (#2346) Co-authored-by: dailidong * init full_name in dml of version 1.2.2 (#2341) Co-authored-by: dailidong * Corrections to historical jar packages and resource data * Corrections to historical jar packages and resource data * Fix the problem of data echo in script edit box * fix * Fix the problem of data echo in script edit box * #2282 fix workflow dependent bug (#2329) * fix workflow dependent bug * fix workflow dependent bug 2 * fix workflow dependent bug 2 Co-authored-by: sunchaohe Co-authored-by: dailidong * .gitignore remove taskInstance/index.vue(#2300) and no need to findProcessDefineById 2 times(#2373) (#2374) * some optimization * revert * ProcessDefinitionService#updateProcessDefinition findProcessDefineById 1 time * [optimization]：remove duplicate code (#2377) * sqlTask failed to run * remove duplicate code * remove duplicate code * remove duplicate code * remove duplicate code * remove duplicate code Co-authored-by: songqh * LogClientService modify * java doc error modify (#2004) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * Refactor worker (#2186) * let quartz use the same datasource * move master/worker config from dao.properties to each config add master/worker registry test * Refactor worker (#2248) * let quartz use the same datasource * move master/worker config from dao.properties to each config add master/worker registry test * move mybatis config from application.properties to SpringConnectionFactory * move mybatis-plus config from application.properties to SpringConnectionFactory * refactor TaskCallbackService * add ZookeeperNodeManagerTest * timing online modify (#2296) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * Encapsulate the parameters required by sqltask * 1，Encapsulate the parameters required by sqltask 2，SQLTask optimization * AbstractTask modify * ProcedureTask optimization * MasterSchedulerService modify * TaskUpdateQueueConsumer modify * test * DataxTask process run debug * DataxTask process run debug * add protobuf dependency，MR、Spark task etc need this * TaskUpdateQueueConsumer modify * TaskExecutionContextBuilder set TaskInstance workgroup * WorkerGroupService queryAllGroup modify query available work group * 1，get workergroup from zk modify 2，SpringConnectionFactory repeat load modify * master and worker register ip use OSUtils.getHost() * ProcessInstance host set ip:port format * worker fault tolerance modify * Constants and .env modify * master fault tolerant bug modify * UT add pom.xml * timing online modify Co-authored-by: qiaozhanwei * [Refactor worker] Optimize code , reduce cost time from 60ms to 0.5ms (#2318) * update logback * update log * refactor worker registry (#2107) * Refactor worker (#2115) * refactor worker registry * refactor master server * Modify workgroupid parameter name (#2105) * Delete worker group management page * Modify workgroupid parameter name * Refactor worker (#2121) * refactor worker registry * refactor master server * refactor MasterSchedulerService * cancelTaskInstance set TaskExecutionContext host,logPath,executePath (#2126) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath Co-authored-by: qiaozhanwei * not exist in openjdk,just delete * add master and worker properties * add master and worker properties * add master and worker properties * fix cpu 100% bug * simplify master、 worker、alert、dao properties * add master and worker properties * add master and worker properties * add master and worker properties * Optimize code , reduce cost time from 60ms to 0.5ms * merge code form remote Co-authored-by: Tboy Co-authored-by: break60 <790061044@qq.com> Co-authored-by: qiaozhanwei Co-authored-by: qiaozhanwei * remove ExecuteTaskRequestCommand and ExecuteTaskResponseCommand * Fix document errors (#2382) * Remove invalid code * fix document errors Co-authored-by: zhanglong Co-authored-by: dailidong * Add maven-wrapper support for dolphinscheduler. (#2381) * add maven-wrapper support. * fixes maven build command to maven-wrapper. Co-authored-by: dailidong * Refactor dockerfile (#2384) * Remove .helmignore file * Refactor dockerfile * mapper add Transactional and rollback * modify ci_e2e.yml * postgresql init.sql modify * Exclude .mvn/jvm.config and .mvn/wrapper/maven-wrapper.properties (#2387) * Remove .helmignore file * Exclude .mvn/jvm.config and .mvn/wrapper/maven-wrapper.properties When running rat, exclude .mvn/jvm.config and .mvn/wrapper/maven-wrapper.properties * modify UT * init.sql modify * pom modify * update UT in pom.xml * update UT in pom.xml * update UT in pom.xml * Optimize resource tree * Change the name of the shell node and modify the parameter transmission method of spark, mr, python, and flink nodes * Repair naming * Modify list style * Update worker_group_id to worker_group (#2433) * add LoginTest license * Delete useless packages * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * e2e add project and workflow case * Update worker_group_id to worker_group in init.sql * Update worker_group_id to worker_group in init.sql * Update worker_group_id to worker_group Co-authored-by: chenxingchun <438044805@qq.com> * Online editing is prohibited * [optimization]：prevent repeated database updates (#2396) * sqlTask failed to run * prevent repeated database updates * prevent repeated database updates * prevent repeated database updates Co-authored-by: songqh Co-authored-by: dailidong * fix: Gantt chart bug * Change the background color of the disabled state of the radio button when the workflow goes online * resolve #2474 (#2477) * In the dev branch, in order to the RPM package which is printed under the dist module contain configuration files: master.properties, worker.properties * Restore files deleted by mistake Co-authored-by: zhangchunyang Co-authored-by: qiaozhanwei * send mail error, #2466 bug fix (#2470) * dispatch task fail will set task status failed * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix Co-authored-by: qiaozhanwei * Update HttpUtils.java * delete img * resolve #2475 Ambari plugin development for the dolphin scheduler in version 2.0.0 … (#2488) * Ambari plugin development for the dolphin scheduler in version 2.0.0 is complete * Update the Ambari plugin of DS usage documentation * delete readme whic is not completed Co-authored-by: zhangchunyang * fix #2450 (#2493) * fix #1441 * support custom datax config * support datax custom config * support datax custom config * support datax custom config * fix #2450 Co-authored-by: 张世鸣 Co-authored-by: dailidong * fixed bug #2492 (#2497) Co-authored-by: mzjnumber1@163.com Co-authored-by: dailidong * Add force not to cache and fix jar instance of workflow instance not showing * Add force not to cache and fix jar instance of workflow instance not showing * disabled * fix style * Modify helm config and change directory's name 1.Change the directory's name to kubernetes 2.Add helm directory by version 3.Modify helm config file * Delete 1.2.0 helm config * Modify config * Modify config * Modify README.md file * add license (#2511) Co-authored-by: sunchaohe * fix ui i18n issue * fix showType in sql.vue * Add docker-compose.yml * Modify docker-compose and kubernetes config 1. Move kubernetes to docker directory 2. Change docker-compose directory's name, and move docker-compose to docker directory 3. Modify docker-compose config and kubernetes config 4. Modify startup.sh in dockerfile * #2499 bug fix (#2527) * dispatch task fail will set task status failed * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * #2486 bug fix * host and workergroup compatible * EnterpriseWeChatUtils modify * EnterpriseWeChatUtils modify * EnterpriseWeChatUtils modify * #2499 bug fix * add comment * revert comment * revert comment * #2499 buf fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix Co-authored-by: qiaozhanwei * update version 2.0.0 backup to 1.3.0 (#2531) * Ambari plugin development for the dolphin scheduler in version 2.0.0 is complete * Update the Ambari plugin of DS usage documentation * delete readme whic is not completed * update version 2.0.0 backup to 1.3.0 * update ambari plugin version to 1.3.0 Co-authored-by: zhangchunyang Co-authored-by: zhangchunyang <18910529250@163.com> * Fix page display incomplete * fix The create user is always loading when the queue does not select any value * No master don't create command #2571 (#2575) * dispatch task fail will set task status failed * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * #2486 bug fix * host and workergroup compatible * EnterpriseWeChatUtils modify * EnterpriseWeChatUtils modify * EnterpriseWeChatUtils modify * #2499 bug fix * add comment * revert comment * revert comment * #2499 buf fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * No master don't create command #2571 * No master don't create command #2571 Co-authored-by: qiaozhanwei * Fix condition judgment state value * fix * fix bug "Password display in plain text" * Work example page list supports scrolling * Optimize workflow instance page * 1. add file add README.md2. delete ambari_plugin/common-services/DOLPHIN/2.0.0 (#2546) * Ambari plugin development for the dolphin scheduler in version 2.0.0 is complete * Update the Ambari plugin of DS usage documentation * delete readme whic is not completed * update version 2.0.0 backup to 1.3.0 * update ambari plugin version to 1.3.0 * add readme doc * delete common-services/DOLPHIN/2.0.0 * update the version in doc Co-authored-by: zhangchunyang Co-authored-by: zhangchunyang <18910529250@163.com> Co-authored-by: dailidong * Add a script to check the license (#2552) * Add a script to check the license ### Motivation Check licenses in the distribution package ### Modification 1. Add a script to check licenses, and list known licenses. 1. Remove unused dep license, lombok. 1. Remove unnecessary ignored patterns in .gitignore. 1. Add missing config items to apache-rat plugin, which just checked a small part of files before. 1. Add check to GitHub Actions process ### Result 1. Newly-added dependencies should be checked and confirmed. 1. Closes #1578 * Separate rat and dependency check * Tee dependencies to stdout * Add more debug messages * Check light-weight task first * Zookeeper management interface in monitoring center displays abnormal #2547 (#2595) Co-authored-by: dailidong * add DolphinScheduler slack channel (#2519) * add DolphinScheduler slack channel * Update README.md Co-Authored-By: Jiajie Zhong Co-authored-by: Jiajie Zhong Co-authored-by: qiaozhanwei * No master don't create command #2571 (#2602) * dispatch task fail will set task status failed * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * send mail error, #2466 bug fix * #2486 bug fix * host and workergroup compatible * EnterpriseWeChatUtils modify * EnterpriseWeChatUtils modify * EnterpriseWeChatUtils modify * #2499 bug fix * add comment * revert comment * revert comment * #2499 buf fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * #2499 bug fix * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * no valid worker group，master can kill task directly * No master don't create command #2571 * No master don't create command #2571 * No master don't create command #2571 Co-authored-by: qiaozhanwei * Reorder lint action * Exclude i18n files from code analysis * Fix worker grouping * fix * fix * fix * fix * fix * fix * fix * Fix worker grouping * Fix notes (#2615) Co-authored-by: taoyuxin Co-authored-by: qiaozhanwei Co-authored-by: Tboy * Fix data loading on file management detail page * modify soft_version 1.2.2 to 1.3.0 (#2642) Co-authored-by: chenxingchun <438044805@qq.com> * update filename * fix bug: sub worker flow in complement data would run start date (#2581) (#2656) Co-authored-by: baoliang * Fix conditions node save exception * script variable has "processDefinitionId" is error #2664 (#2666) Co-authored-by: qiaozhanwei * blank in eamil and left font align #2648 (#2673) * script variable has "processDefinitionId" is error #2664 * blank in eamil and left font align #2648 Co-authored-by: qiaozhanwei * delete dockerfile zk * Update known-dependencies.txt * Update known-dependencies.txt (#2748) * fix getWorkers bug on RunConfig (#2747) * fix worker group config no effect * remove codehaus janino jar the license about janino maybe not compatiable with Apache v2 * Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0 # Conflicts: # dolphinscheduler-server/src/main/resources/config/install_config.conf * datasource config * Update datasource.properties * fix RunConfig bug * Fix rename check parameter * update e2e chrome version (#2775) Co-authored-by: chenxingchun <438044805@qq.com> * Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory (#2786) * fixed #2788 resolve the bug #2788 (#2789) * Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory * fixed #2788 * delete unnecessary configuration items (#2803) * Added instructions for ambari to add and remove Dolphin Schdumer components on nodes (#2806) * delete unnecessary configuration items * Added instructions for ambari to add and remove Dolphin Schdumer components on nodes * Added instructions for ambari to add and remove Dolphin Schdumer components on nodes * when init workflow, the tenantId linked by the tenantId of user * fix * add maven Xmx to 5g * add source license full text into source license (#2840) * add source license full text into source license * exclude ScriptRunner.java on rat check * change notice copyright date from 2019 to 2019-2020 (#2847) * add source license full text into source license * exclude ScriptRunner.java on rat check * change notice copyright date from 2019 to 2019-2020 * fix * modify t_ds_task_instance app_link change sql (#2861) * change max length of user name to 39 (#2856) * fix bug: send task result to master until success. * fix bug: send task result to master until success. * add sleep * set null if send result failed * set null if send result failed * change max length of username to 39 Co-authored-by: baoliang * Fix tasks not execute in sub-process when rerun scheduled process. #2903 (#2908) * fix bug: sqoop mysql import params error (#2896) * fix bug: sqoop mysql import params error * fix bug: sqoop mysql import params error * update how to build and remove docs link (#2906) * fix create table sql exception (#2912) fix create table sql exception * skip test for release * Delete the pom files animation.scss, _normalize.scss and _animation.scss under the common folder * add mybatis-3 notice in the top-level notice (#3003) * fix download resource file that under the directory error (#3007) * add mybatis-3 notice in the top-level notice * fix download resource file that under the directory error * Fixed selection of tenant value and defalut changed to lower case * Merge pull request #2898 from 597365581/patch-6 merge method updateuser if and fix bug#2892 * Fix unfiltered jar files in resource directory and UDF upload resource parameters (#3008) * Change email to default * Fix unfiltered jar files in resource directory and UDF upload resource parameters * merge 1.3.0-release branch code * merge 1.3.0-release branch code Co-authored-by: dailidong Co-authored-by: gabry.wu Co-authored-by: liwenhe1993 <32166572+liwenhe1993@users.noreply.github.com> Co-authored-by: tswstarplanet Co-authored-by: Rubik-W <39549317+Rubik-W@users.noreply.github.com> Co-authored-by: AhahaGe Co-authored-by: guirong.ggr Co-authored-by: Jave-Chen Co-authored-by: t1mon <178317391@qq.com> Co-authored-by: break60 <790061044@qq.com> Co-authored-by: zhukai Co-authored-by: zhukai Co-authored-by: Yelli Co-authored-by: DS Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> Co-authored-by: chenxingchun <438044805@qq.com> Co-authored-by: ruson <648380139@qq.com> Co-authored-by: lgcareer <18610854716@163.com> Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com> Co-authored-by: Tboy Co-authored-by: elonlo Co-authored-by: qiaozhanwei Co-authored-by: zhangchunyang <18910529250@163.com> Co-authored-by: 谷雨 Co-authored-by: Simon <3656562@qq.com> Co-authored-by: 张世鸣 Co-authored-by: John Liu <517356906@qq.com> Co-authored-by: jin Co-authored-by: khadgarmage Co-authored-by: songquanhe-gitstudy <1172417734@qq.com> Co-authored-by: songqh Co-authored-by: WeiYe <33245094+WeiYe-Jing@users.noreply.github.com> Co-authored-by: 孙朝和 <649790970@qq.com> Co-authored-by: sunchaohe Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com> Co-authored-by: zhanglong Co-authored-by: itbasketplayer <825621876@qq.com> Co-authored-by: qiaozhanwei Co-authored-by: Zonglei Dong Co-authored-by: Rubik-W Co-authored-by: zhangchunyang Co-authored-by: muzhongjiang Co-authored-by: mzjnumber1@163.com Co-authored-by: liwenhe1993 <411934049@qq.com> Co-authored-by: oceanos Co-authored-by: zhangchunyang Co-authored-by: samz406 Co-authored-by: kezhenxu94 Co-authored-by: tlhhup <137273278@qq.com> Co-authored-by: Jiajie Zhong Co-authored-by: kezhenxu94 Co-authored-by: 743294668 <41327198+743294668@users.noreply.github.com> Co-authored-by: taoyuxin Co-authored-by: baoliang Co-authored-by: xiaochun.liu Co-authored-by: gabry.wu

[BUG] sql task, query type, use column label #2334

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions