-
Notifications
You must be signed in to change notification settings - Fork 5k
[Feature] Add ETL function node #1856
Copy link
Copy link
Closed
Labels
featurenew featurenew feature
Description
I extended the ETL node on DS, Implement the function of executing SQL for data synchronization.
Synchronize data through Datax middleware, DS need not be coupled to Datax.
I can contribute code if needed by the community.
Implementation ideas:
1.Add ETL nodes in DS
2.Generate Datax json file based on configuration parameters
3.Datax python script called by shell script task to start process
Problems with the current implementation:
1.Datax does not support multi-table synchronization, a task can only synchronize one table
2.Low resource utilization, each ETL task needs to start a JVM process
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
featurenew featurenew feature
