Overview

Relevant source files

• README.md
• bin/install-plugin.cmd
• bin/install-plugin.sh
• config/plugin_config
• docs/en/seatunnel-engine/download-seatunnel.md
• docs/en/start-v2/docker/docker.md
• docs/en/start-v2/kubernetes/kubernetes.mdx
• docs/en/start-v2/locally/deployment.md
• docs/zh/seatunnel-engine/download-seatunnel.md
• docs/zh/start-v2/docker/docker.md
• docs/zh/start-v2/locally/deployment.md
• plugin-mapping.properties
• pom.xml
• seatunnel-connectors-v2/connector-http/pom.xml
• seatunnel-connectors-v2/pom.xml
• seatunnel-dist/pom.xml
• seatunnel-dist/release-docs/LICENSE
• seatunnel-dist/release-docs/NOTICE
• seatunnel-dist/src/main/assembly/assembly-bin-ci.xml
• seatunnel-dist/src/main/assembly/assembly-bin.xml
• seatunnel-dist/src/main/assembly/assembly-src.xml
• seatunnel-dist/src/main/docker/Dockerfile
• seatunnel-e2e/seatunnel-connector-v2-e2e/connector-http-e2e/pom.xml
• seatunnel-e2e/seatunnel-connector-v2-e2e/connector-http-e2e/src/test/resources/http_streaming_json_to_postgresql.conf
• seatunnel-e2e/seatunnel-connector-v2-e2e/pom.xml
• seatunnel-e2e/seatunnel-e2e-common/pom.xml
• seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/util/ContainerUtil.java
• seatunnel-e2e/seatunnel-transforms-v2-e2e/pom.xml
• seatunnel-engine/seatunnel-engine-common/src/main/java/org/apache/seatunnel/engine/common/config/ConfigProvider.java
• tools/dependencies/known-dependencies.txt

Apache SeaTunnel is a multimodal, high-performance, distributed data integration platform designed for synchronizing large volumes of data across diverse systems. It addresses common challenges in data integration including:

• Support for 100+ heterogeneous data sources and sinks
• Multimodal data support (structured, unstructured text, binary files, images, video)
• Complex synchronization patterns (batch, real-time streaming, CDC, full database sync)
• Exactly-once consistency guarantees via distributed snapshot algorithms
• Resource-efficient real-time synchronization with connection multiplexing

SeaTunnel provides a unified Connector API v2 that enables the same connectors to run unchanged across multiple execution engines (SeaTunnel Zeta, Apache Flink, Apache Spark). It is licensed under the Apache License 2.0 and maintained by the Apache Software Foundation.

This document covers SeaTunnel's architecture, component responsibilities, and distribution model. For specific topics:

• For build and deployment details, see Build System and Distribution
• For runtime architecture and job execution, see Architecture and Runtime
• For connector development, see Connector and Plugin System

Sources: README.md30-66 pom.xml32-34 seatunnel-dist/release-docs/LICENSE1-18

---

System Architecture

SeaTunnel is organized into seven major architectural layers, each serving distinct responsibilities in the data integration lifecycle.

Diagram: SeaTunnel System Architecture and Component Organization

Sources: README.md67-71 pom.xml36-56 seatunnel-dist/pom.xml38-72

---

Multi-Module Project Structure

SeaTunnel is organized as a Maven multi-module project with clear separation of concerns. The parent POM pom.xml27-29 defines the project coordinates org.apache.seatunnel:seatunnel and manages dependencies across all modules.

Key Modules:

Module	Location	Purpose
seatunnel-api	pom.xml47	Connector API v2 interfaces: SeaTunnelSource, SeaTunnelSink, SeaTunnelTransform
seatunnel-core	pom.xml44	Engine starters for Flink (1.13/1.15/1.20) and Spark (2.4/3.3)
seatunnel-engine	pom.xml51	Native Zeta engine implementation with master-worker architecture
seatunnel-connectors-v2	pom.xml46	100+ connector implementations organized by type
seatunnel-transforms-v2	pom.xml45	Transform plugins including SQL engine with 100+ functions
seatunnel-translation	pom.xml48	Translation layer converting logical DAGs to engine-specific execution plans
seatunnel-plugin-discovery	pom.xml49	SPI-based plugin discovery with @AutoService support
seatunnel-formats	pom.xml50	Format support: text, CSV, JSON, Parquet, ORC, Avro, Excel, XML
seatunnel-dist	seatunnel-dist/pom.xml29	Distribution assembly and packaging
seatunnel-e2e	pom.xml53	End-to-end test framework using Testcontainers

The project uses Java 8 pom.xml63 and Scala 2.12.15 pom.xml64 as base language versions. It maintains compatibility with multiple engine versions through separate starter modules and careful dependency scoping.

Sources: pom.xml16-56 seatunnel-connectors-v2/pom.xml28-93

---

Multi-Engine Support

SeaTunnel's defining architectural feature is engine independence: the same connector can execute on three different compute engines without modification. This is achieved through the Translation Layer pom.xml48 which converts the unified logical DAG into engine-specific execution plans.

Diagram: Multi-Engine Architecture with Unified Connector API

Engine-Specific Starters:

Each engine has dedicated starter JARs packaged in the distribution seatunnel-dist/src/main/assembly/assembly-bin.xml149-167:

• Zeta Engine: seatunnel-starter.jar in starter/ directory
• Flink: seatunnel-flink-{13,15,20}-starter.jar with version-specific bindings
• Spark: seatunnel-spark-{2,3}-starter.jar for Spark 2.4 and 3.3

The binary distribution includes all starters simultaneously, allowing users to switch engines by changing command-line parameters without reinstalling.

Sources: pom.xml77-82 seatunnel-dist/pom.xml117-154 seatunnel-dist/src/main/assembly/assembly-bin.xml149-167

---

Plugin Ecosystem and Discovery

SeaTunnel's extensibility is built on a plugin architecture where connectors and transforms are discovered at runtime via Java's Service Provider Interface (SPI) mechanism enhanced with Google's @AutoService annotation pom.xml506-510

Diagram: Plugin Architecture with SPI Discovery and Installation Workflow

Plugin Installation Process:

1. Configuration: Users specify required plugins in config/plugin_config22-99:

   --connectors-v2--
   connector-jdbc
   connector-kafka
   connector-fake
   connector-console
   --end--
   

2. Mapping Resolution: plugin-mapping.properties18-157 maps user-friendly names to Maven artifact IDs:

3. Download: bin/install-plugin.sh19-53 reads the config and downloads JARs using Maven wrapper:

4. Discovery: At runtime, ClassLoaderService scans connectors/*.jar for SPI-registered factories implementing TableSourceFactory, TableSinkFactory, or TableTransformFactory.

The system uses @AutoService annotations pom.xml542-546 to automatically generate META-INF/services/ descriptor files during compilation, eliminating manual SPI configuration.

Sources: bin/install-plugin.sh18-53 plugin-mapping.properties18-157 config/plugin_config1-99 seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/util/ContainerUtil.java70-93

---

Distribution Model

SeaTunnel is distributed as a tar.gz archive assembled by Maven's assembly plugin using seatunnel-dist/src/main/assembly/assembly-bin.xml1-225 The distribution follows a minimal core + pluggable connectors model to reduce download size.

Distribution Structure:

apache-seatunnel-${version}-bin/
├── bin/
│   ├── seatunnel.sh                  # Zeta engine CLI
│   ├── seatunnel-cluster.sh          # Zeta cluster mode
│   ├── install-plugin.sh             # Connector installer
│   ├── start-seatunnel-flink-*.sh    # Flink starters
│   └── start-seatunnel-spark-*.sh    # Spark starters
├── config/
│   ├── plugin_config                 # Plugin selection
│   ├── hazelcast.yaml                # Zeta cluster config
│   ├── hazelcast-client.yaml         # Zeta client config
│   └── v2.*.conf.template            # Example jobs
├── connectors/
│   ├── plugin-mapping.properties     # Name-to-artifact mapping
│   ├── connector-fake.jar            # Demo connector
│   └── connector-console.jar         # Demo connector
├── lib/
│   └── seatunnel-transforms-v2.jar   # Transform plugins
├── starter/
│   ├── logging/                      # SLF4J + Log4j2 JARs
│   ├── seatunnel-starter.jar         # Zeta engine entry
│   ├── seatunnel-flink-*-starter.jar # Flink entries
│   └── seatunnel-spark-*-starter.jar # Spark entries
├── LICENSE
├── NOTICE
└── README.md

Assembly Configuration:

The CI distribution seatunnel-dist/src/main/assembly/assembly-bin-ci.xml1-213 includes all connectors for testing, while the release distribution seatunnel-dist/src/main/assembly/assembly-bin.xml1-225 includes only minimal demo connectors (connector-fake, connector-console) to reduce size.

Docker Image:

The project provides an official Docker image built via seatunnel-dist/src/main/docker/Dockerfile1-18:

The image is published to Docker Hub as apache/seatunnel:${version} pom.xml119-121

Sources: seatunnel-dist/src/main/assembly/assembly-bin.xml27-224 seatunnel-dist/src/main/assembly/assembly-bin-ci.xml27-213 seatunnel-dist/src/main/docker/Dockerfile1-18 pom.xml119-124

---

Version and Compatibility

Current Release: SeaTunnel 2.3.13 pom.xml60

Engine Compatibility Matrix:

Engine	Supported Versions	Starter Module
SeaTunnel Zeta	Native (built-in)	seatunnel-starter
Apache Flink	1.13.6, 1.15.3, 1.18.x, 1.20.1	seatunnel-flink-{13,15,20}-starter
Apache Spark	2.4.0, 3.3.0	seatunnel-spark-{2,3}-starter

Runtime Requirements:

• Java 8 or 11 README.md14
• JAVA_HOME environment variable configured

Build Requirements:

• Maven 3.6+ (via included mvnw wrapper)
• Java 8 or 11 pom.xml63

Connector Count: 100+ README.md51 across categories:

• Sources/Sinks: JDBC (30+ databases), File Systems (HDFS, S3, Local, OSS, FTP, SFTP, COS), Message Queues (Kafka, Pulsar, RocketMQ), NoSQL (MongoDB, Redis, Cassandra, Elasticsearch, Neo4j), Cloud (DynamoDB, Firestore, SQS), Data Lakes (Iceberg, Hudi, Paimon)
• CDC: MySQL, Oracle, SQL Server, PostgreSQL, MongoDB, TiDB, OpenGauss
• Transforms: SQL (ZetaSQLEngine with 100+ functions), FieldMapper, Filter, Split, Replace, Copy

All connectors implement the same SeaTunnelSource, SeaTunnelSink, or SeaTunnelTransform interfaces and work identically across all three engines.

Sources: pom.xml60-82 README.md51-63 plugin-mapping.properties23-157

---

Getting Started

SeaTunnel can be deployed in three modes depending on execution requirements:

1. Local Mode (Zeta): Single-process execution for development and testing

   • No cluster deployment required
   • Suitable for small datasets and job validation
   • See Quick Start With SeaTunnel Engine

2. Cluster Mode (Zeta): Distributed execution with master-worker architecture

   • Deploy SeaTunnelServer nodes with Hazelcast clustering
   • Supports high availability and horizontal scaling
   • See Zeta Engine Architecture Overview

3. External Engine Mode: Submit jobs to existing Flink or Spark clusters

   • No SeaTunnel deployment needed beyond CLI tools
   • Leverages existing cluster management and resource allocation
   • See Multi-Engine Architecture

Typical Installation Steps:

1. Download binary: wget https://archive.apache.org/dist/seatunnel/2.3.13/apache-seatunnel-2.3.13-bin.tar.gz
2. Extract: tar -xzvf apache-seatunnel-2.3.13-bin.tar.gz
3. Install connectors: sh bin/install-plugin.sh 2.3.13
4. Configure job: Create .conf file with source, transform, sink definitions
5. Execute:
   • Local: sh bin/seatunnel.sh -m local -c job.conf
   • Cluster: sh bin/seatunnel.sh -c job.conf (defaults to cluster mode)
   • Flink: sh bin/start-seatunnel-flink-13-connector-v2.sh -c job.conf
   • Spark: sh bin/start-seatunnel-spark-3-connector-v2.sh -c job.conf

Docker Quick Start:

Sources: docs/en/start-v2/locally/deployment.md1-91 docs/en/start-v2/docker/docker.md1-144 README.md79-91

---

Project Governance and Licensing

Apache SeaTunnel is a top-level project of the Apache Software Foundation README.md1 All code is licensed under Apache License 2.0 README.md137 seatunnel-dist/release-docs/LICENSE1-201

Community Resources:

• Source Code: https://github.com/apache/seatunnel
• Documentation: https://seatunnel.apache.org/docs
• Downloads: https://seatunnel.apache.org/download
• Mailing List: dev@seatunnel.apache.org
• Slack: https://s.apache.org/seatunnel-slack

Third-Party Dependencies:

SeaTunnel includes components with separate licenses documented in seatunnel-dist/release-docs/NOTICE1-600 including:

• Hazelcast (Apache 2.0)
• Apache Avro (Apache 2.0)
• Jackson (Apache 2.0)
• Guava (Apache 2.0)
• Log4j2 (Apache 2.0)
• SLF4J (MIT)
• Netty (Apache 2.0)
• Protobuf (BSD)

All dependencies are tracked in tools/dependencies/known-dependencies.txt1-131 for license compliance verification.

Sources: README.md1-166 seatunnel-dist/release-docs/LICENSE1-201 seatunnel-dist/release-docs/NOTICE1-600 tools/dependencies/known-dependencies.txt1-131