How to Choose Apache SeaTunnel Zeta, Flink, or Spark?

.

   tar -zxvf apache-seatunnel-*.tar.gz

   cd apache-seatunnel-*

7.3 Install Connector Plugins (Important!)

This is the step most beginners tend to overlook.

The default distribution does not include all connectors. You need to run the script to automatically download them.

# Automatically install all plugins defined in plugin_config

sh bin/install-plugin.sh

7.4 Run Your First Job Quickly

Create a simple configuration file config/quick_start.conf to generate data from a Fake source and print it to the console:

env {

  execution.parallelism = 1

  job.mode = "BATCH"

}



source {

  FakeSource {

    result_table_name = "fake"

    row.num = 100

    schema = {

      fields {

        name = "string"

        age = "int"

      }

    }

  }

}



transform {

  # Simple SQL processing

  Sql {

    source_table_name = "fake"

    result_table_name = "sql_result"

    query = "select name, age from fake where age > 50"

  }

}



sink {

  Console {

    source_table_name = "sql_result"

  }

}

Run the job (Local mode):

./bin/seatunnel.sh --config ./config/quick_start.conf -e local

If you see tabular data printed in the console, congratulations — you have successfully mastered the basic usage of SeaTunnel!

8. Deep Learning Path for the Zeta Engine Internals

If you want to gain a deeper understanding of how the Zeta engine works internally, or plan to contribute to the community, you can follow the learning path below to read and debug the source code.

8.1 Core Module Overview

The Zeta engine code is mainly located under the seatunnel-engine module:

• 
  seatunnel-engine-core: Defines core data structures (such as Job and Task) and communication protocols.


• 
  seatunnel-engine-server: Contains the concrete implementations of the Coordinator and Worker.


• 
  seatunnel-engine-client: Handles client-side job submission logic.

8.2 Recommended Source Code Reading Path

1. Job Submission and Parsing (Coordinator Side)

Start from the JobMaster class to understand how jobs are received and initialized.

• 
  Entry point: org.apache.seatunnel.engine.server.master.JobMaster
  


• 
  Key logic: Focus on the init and run methods to understand the transformation from LogicalDag to PhysicalPlan.

2. Task Execution (Worker Side)

Understand how Tasks are scheduled and executed.

• 
  

  Service entry:
  
  

  
  
  
  
  
  • Focus on the triggerCheckpoint method to understand how barriers are distributed.
  
  
  
  
  
  



• 
  
  

  Planning:
  
  CheckpointPlan.java

  
  
  
  
  
  • Understand how the scope of tasks involved in a checkpoint is calculated.
  
  
  
  
  
  

8.3 Debugging Tips

1. Adjust log level:
   
   In config/log4j2.properties, set the log level of org.apache.seatunnel to DEBUG to observe detailed RPC communication and state transition logs.




2. Local debugging:
   
   Run the org.apache.seatunnel.core.starter.seatunnel.SeaTunnelStarter class directly in your IDE, passing the parameters
   
   -c config/your_job.conf -e local,
   
   to set breakpoints and debug the entire execution flow.