Skip to content

Commit a0bcab4

Browse files
authored
Merge branch 'apache:dev' into dev-doris-redirect
2 parents a8a514c + df6ca72 commit a0bcab4

76 files changed

Lines changed: 247 additions & 184 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/en/architecture/api-design/catalog-table.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -663,15 +663,15 @@ sink {
663663

664664
- [Source Architecture](source-architecture.md)
665665
- [Sink Architecture](sink-architecture.md)
666-
- [Schema Evolution](../../introduction/concepts/schema-evolution.md)
666+
- [Schema Evolution](../../introduction/configuration/schema-evolution.md)
667667
- [Schema Feature](../../introduction/concepts/schema-feature.md)
668668

669669
## 11. References
670670

671671
### Key Source Files
672672

673-
- [CatalogTable.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/CatalogTable.java)
674-
- [TableSchema.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java)
675-
- [Column.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/Column.java)
676-
- [SeaTunnelDataType.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelDataType.java)
677-
- [SchemaChangeEvent.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/event/SchemaChangeEvent.java)
673+
- [CatalogTable.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/CatalogTable.java)
674+
- [TableSchema.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java)
675+
- [Column.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/Column.java)
676+
- [SeaTunnelDataType.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelDataType.java)
677+
- [SchemaChangeEvent.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/schema/event/SchemaChangeEvent.java)

docs/en/architecture/api-design/sink-architecture.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -607,10 +607,10 @@ public class HiveAggregatedCommitter
607607
### 3.4 Code References
608608

609609
**API Interfaces**:
610-
- [SeaTunnelSink.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SeaTunnelSink.java)
611-
- [SinkWriter.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkWriter.java)
612-
- [SinkCommitter.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkCommitter.java)
613-
- [SinkAggregatedCommitter.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkAggregatedCommitter.java)
610+
- [SeaTunnelSink.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SeaTunnelSink.java)
611+
- [SinkWriter.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkWriter.java)
612+
- [SinkCommitter.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkCommitter.java)
613+
- [SinkAggregatedCommitter.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkAggregatedCommitter.java)
614614

615615
**Example Implementations**:
616616
- JDBC Sink: `seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/`

docs/en/architecture/api-design/source-architecture.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -516,10 +516,10 @@ public class ConfigChangeEvent implements SourceEvent {
516516
### 3.4 Code References
517517

518518
**API Interfaces**:
519-
- [SeaTunnelSource.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SeaTunnelSource.java)
520-
- [SourceSplitEnumerator.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplitEnumerator.java)
521-
- [SourceReader.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceReader.java)
522-
- [SourceSplit.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplit.java)
519+
- [SeaTunnelSource.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SeaTunnelSource.java)
520+
- [SourceSplitEnumerator.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplitEnumerator.java)
521+
- [SourceReader.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceReader.java)
522+
- [SourceSplit.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplit.java)
523523

524524
**Example Implementations**:
525525
- JDBC Source: `seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/`

docs/en/architecture/engine/dag-execution.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -745,11 +745,11 @@ Parallelism = min(
745745

746746
### Key Source Files
747747

748-
- [LogicalDag.java](../../../seatunnel-engine/seatunnel-engine-core/src/main/java/org/apache/seatunnel/engine/core/dag/logical/LogicalDag.java)
749-
- [PhysicalPlan.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/dag/physical/PhysicalPlan.java)
750-
- [SubPlan.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/dag/physical/SubPlan.java)
751-
- [PhysicalVertex.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/dag/physical/PhysicalVertex.java)
752-
- [TaskGroupDefaultImpl.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/group/TaskGroupDefaultImpl.java)
748+
- [LogicalDag.java](../../../../seatunnel-engine/seatunnel-engine-core/src/main/java/org/apache/seatunnel/engine/core/dag/logical/LogicalDag.java)
749+
- [PhysicalPlan.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/dag/physical/PhysicalPlan.java)
750+
- [SubPlan.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/dag/physical/SubPlan.java)
751+
- [PhysicalVertex.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/dag/physical/PhysicalVertex.java)
752+
- [TaskGroupDefaultImpl.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/execution/TaskGroupDefaultImpl.java)
753753

754754
### Further Reading
755755

docs/en/architecture/engine/engine-architecture.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,7 @@ IMap<Long, JobInfo> completedJobInfoIMap;
135135
```
136136

137137
**Code Reference**:
138-
- [CoordinatorService.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java)
138+
- [CoordinatorService.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java)
139139

140140
#### JobMaster
141141

@@ -160,7 +160,7 @@ Created → Initialized → Scheduled → Running → Finished/Failed/Canceled
160160
3. `handleFailure()`: Restart failed tasks, restore from checkpoint
161161

162162
**Code Reference**:
163-
- [JobMaster.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java)
163+
- [JobMaster.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java)
164164

165165
#### ResourceManager
166166

@@ -181,8 +181,8 @@ Manages worker resources and slot allocation.
181181
```
182182

183183
**Code Reference**:
184-
- [ResourceManager.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/ResourceManager.java)
185-
- [AbstractResourceManager.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/AbstractResourceManager.java)
184+
- [ResourceManager.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/ResourceManager.java)
185+
- [AbstractResourceManager.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/AbstractResourceManager.java)
186186

187187
## 3. DAG Execution Model
188188

@@ -496,7 +496,7 @@ public class CheckpointCoordinator {
496496
7. Create CompletedCheckpoint, persist to storage
497497

498498
**Code Reference**:
499-
- [CheckpointCoordinator.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java)
499+
- [CheckpointCoordinator.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java)
500500

501501
### 5.2 Checkpoint Barrier
502502

docs/en/architecture/engine/resource-management.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -522,10 +522,10 @@ env {
522522

523523
### Key Source Files
524524

525-
- [ResourceManager.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/ResourceManager.java)
526-
- [AbstractResourceManager.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/AbstractResourceManager.java)
527-
- [SlotProfile.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/resource/SlotProfile.java)
528-
- [WorkerProfile.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/worker/WorkerProfile.java)
525+
- [ResourceManager.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/ResourceManager.java)
526+
- [AbstractResourceManager.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/AbstractResourceManager.java)
527+
- [SlotProfile.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/resource/SlotProfile.java)
528+
- [WorkerProfile.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/resourcemanager/worker/WorkerProfile.java)
529529

530530
### Further Reading
531531

docs/en/architecture/fault-tolerance/checkpoint-mechanism.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -744,9 +744,9 @@ class GoodSourceReader {
744744
745745
### Key Source Files
746746
747-
- [CheckpointCoordinator.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java)
748-
- [PendingCheckpoint.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/PendingCheckpoint.java)
749-
- [CheckpointStorage.java](../../../seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-api/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/api/CheckpointStorage.java)
747+
- [CheckpointCoordinator.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java)
748+
- [PendingCheckpoint.java](../../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/PendingCheckpoint.java)
749+
- [CheckpointStorage.java](../../../../seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-api/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/api/CheckpointStorage.java)
750750
751751
### Academic Papers
752752

docs/en/architecture/features/multi-table.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -741,15 +741,15 @@ if (table.getWriteRate() > threshold) {
741741
- [CatalogTable and Metadata](../api-design/catalog-table.md)
742742
- [Sink Architecture](../api-design/sink-architecture.md)
743743
- [DAG Execution](../engine/dag-execution.md)
744-
- [Schema Evolution](../../introduction/concepts/schema-evolution.md)
744+
- [Schema Evolution](../../introduction/configuration/schema-evolution.md)
745745

746746
## 14. References
747747

748748
### Key Source Files
749749

750-
- [MultiTableSink.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/MultiTableSink.java)
751-
- [SinkIdentifier.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkIdentifier.java)
752-
- [TablePath.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TablePath.java)
750+
- [MultiTableSink.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/multitablesink/MultiTableSink.java)
751+
- [SinkIdentifier.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/multitablesink/SinkIdentifier.java)
752+
- [TablePath.java](../../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TablePath.java)
753753

754754
### Example Implementations
755755

docs/en/architecture/overview.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -98,8 +98,8 @@ The API layer provides engine-independent abstractions:
9898
**Key Design**: Separation of coordination (Enumerator) and execution (Reader) enables efficient parallel processing and fault tolerance.
9999

100100
**Code Reference**:
101-
- [seatunnel-api/.../SeaTunnelSource.java](../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SeaTunnelSource.java)
102-
- [seatunnel-api/.../SourceSplitEnumerator.java](../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplitEnumerator.java)
101+
- [seatunnel-api/.../SeaTunnelSource.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SeaTunnelSource.java)
102+
- [seatunnel-api/.../SourceSplitEnumerator.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplitEnumerator.java)
103103

104104
#### Sink API
105105
- **SeaTunnelSink**: Factory interface for creating writers and committers
@@ -110,24 +110,24 @@ The API layer provides engine-independent abstractions:
110110
**Key Design**: Two-phase commit protocol (prepareCommit → commit) ensures exactly-once semantics.
111111

112112
**Code Reference**:
113-
- [seatunnel-api/.../SeaTunnelSink.java](../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SeaTunnelSink.java)
114-
- [seatunnel-api/.../SinkWriter.java](../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkWriter.java)
113+
- [seatunnel-api/.../SeaTunnelSink.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SeaTunnelSink.java)
114+
- [seatunnel-api/.../SinkWriter.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkWriter.java)
115115

116116
#### Transform API
117117
- **SeaTunnelTransform**: Data transformation interface
118118
- **SeaTunnelMapTransform**: 1:1 transformation
119119
- **SeaTunnelFlatMapTransform**: 1:N transformation
120120

121121
**Code Reference**:
122-
- [seatunnel-api/.../SeaTunnelTransform.java](../../seatunnel-api/src/main/java/org/apache/seatunnel/api/transform/SeaTunnelTransform.java)
122+
- [seatunnel-api/.../SeaTunnelTransform.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/transform/SeaTunnelTransform.java)
123123

124124
#### Table API
125125
- **CatalogTable**: Complete table metadata (schema, partition keys, options)
126126
- **TableSchema**: Schema definition (columns, primary key, constraints)
127127
- **SchemaChangeEvent**: Represents DDL changes for schema evolution
128128

129129
**Code Reference**:
130-
- [seatunnel-api/.../CatalogTable.java](../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/CatalogTable.java)
130+
- [seatunnel-api/.../CatalogTable.java](../../../seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/CatalogTable.java)
131131

132132
### 3.2 SeaTunnel Engine (Zeta)
133133

@@ -150,8 +150,8 @@ LogicalDag → PhysicalPlan → SubPlan (Pipeline) → PhysicalVertex → TaskGr
150150
```
151151

152152
**Code Reference**:
153-
- [seatunnel-engine/.../server/CoordinatorService.java](../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java)
154-
- [seatunnel-engine/.../server/master/JobMaster.java](../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java)
153+
- [seatunnel-engine/.../server/CoordinatorService.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java)
154+
- [seatunnel-engine/.../server/master/JobMaster.java](../../../seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java)
155155

156156
### 3.3 Translation Layer
157157

@@ -163,7 +163,7 @@ Enables engine portability through adapter pattern:
163163
- **Serialization Adapters**: Bridges SeaTunnel and engine serialization mechanisms
164164

165165
**Code Reference**:
166-
- [seatunnel-translation/.../flink/source/FlinkSource.java](../../seatunnel-translation/seatunnel-translation-flink/seatunnel-translation-flink-common/src/main/java/org/apache/seatunnel/translation/flink/source/FlinkSource.java)
166+
- [seatunnel-translation/.../flink/source/FlinkSource.java](../../../seatunnel-translation/seatunnel-translation-flink/seatunnel-translation-flink-common/src/main/java/org/apache/seatunnel/translation/flink/source/FlinkSource.java)
167167

168168
### 3.4 Connector Ecosystem
169169

docs/en/connectors/sink/DuckDB.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ semantics (using XA transaction guarantee).
8181
| auto_commit | Boolean | No | true | Automatic transaction commit is enabled by default |
8282
| field_ide | String | No | - | Identify whether the field needs to be converted when synchronizing from the source to the sink. `ORIGINAL` indicates no conversion is needed; `UPPERCASE` indicates conversion to uppercase; `LOWERCASE` indicates conversion to lowercase. |
8383
| properties | Map | No | - | Additional connection configuration parameters, when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in DuckDB, properties take precedence over the URL. |
84-
| common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](../sink-common-options.md) for details |
84+
| common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](../common-options/sink-common-options.md) for details |
8585
| schema_save_mode | Enum | No | CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side. |
8686
| data_save_mode | Enum | No | APPEND_DATA | Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side. |
8787
| custom_sql | String | No | - | When data_save_mode selects CUSTOM_PROCESSING, you should fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that can be executed. SQL will be executed before synchronization tasks. |
@@ -201,4 +201,4 @@ sink {
201201

202202
## Changelog
203203

204-
<ChangeLog />
204+
<ChangeLog />

0 commit comments

Comments
 (0)