Search before asking

• I had searched in the DSIP and found no similar DSIP.

Motivation

Reason: The current data source plugin does not have a class loading isolation mechanism, which makes it impossible to support multiple versions of the data source driver simultaneously
Implementation goal: When users create data sources, they can choose the data source driven version for flexible customization

Design Detail

1. Front end:
   (1) Add data source driver version information when creating, modifying data sources
   (2) Call the backend interface to query the data source version list display

2. Backend:
   (1) Add a new database driver version configuration table, provide an interface to query the data source version list, match and query the driver version configuration table based on the version number to query the corresponding version of the driver package (main version enumeration)

CREATE TABLE t_ds_datasource_driver_version_config (
id int(11) NOT NULL AUTO_INCREMENT COMMENT 'key',
type tinyint(4) NOT NULL COMMENT 'data source type: 0:mysql,1:postgresql,2:hive,3:spark',
version varchar(255) COLLATE utf8_bin DEFAULT NULL COMMENT 'version'
driver_class_name varchar(255) COLLATE utf8_bin DEFAULT NULL COMMENT 'driver_class_name',
driver_jar_name varchar(255) COLLATE utf8_bin DEFAULT NULL COMMENT 'driver_jar_name'
PRIMARY KEY (id),
UNIQUE KEY t_ds_datasource_driver_type_version (type,version)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

(2) Add version field to data source connection parameters

{
"user": "datawings",
"password": "xxx"
"address": "jdbc:mysql://127.0.0.1:50001",
"database": "test",
"jdbcUrl": "jdbc:mysql://127.0.0.1:50001/test",
"driverClassName": "com.mysql.cj.jdbc.Driver",
"version":"8.0.33",
"validationQuery": "select 1"
}

(3) When performing tasks with different data sources and creating database connections, use a custom class loader to dynamically load the corresponding version of the driver package

a. POM configuration: Package multiple versions of driver packages into a specified directory (such as lib/drivers/) to avoid Maven dependency mediation coverage;
b. Dependency isolation: Manually copy multiple versions of drivers to different subdirectories through Maven dependency plugin, instead of relying on Maven for automatic dependency management;
c. Run time loading: By using a custom class loader to load a specified version of the driver package, multiple versions can be isolated for use. Each version of the driver uses an independent URLClassLoader, and the parent loader is set to null to avoid JVM default class loaders loading conflicting classes;

Compatibility, Deprecation, and Migration Plan

Use the default driver package to execute existing tasks

Test Plan

(1) Test connection success when creating different versions of data sources
(2) The same workflow uses different driver versions of data sources, and the workflow executes normally

Code of Conduct

• I agree to follow this project's Code of Conduct