Merge branch 'dev' into dev-jdbc-improve-load-metadata

Author: hailin0

.github/workflows/documents.yml

@@ -45,11 +45,12 @@ jobs:
 
       - uses: actions/setup-node@v2
         with:
-          node-version: 14
+          node-version: 16.19.0
 
       - name: Run docusaurus build
         run: |
           cd seatunnel-website
+          npm set strict-ssl false
           npm install
           npm run build
 

docs/en/command/connector-check.md

@@ -0,0 +1,35 @@
+# Connector check command usage
+
+## Command Entrypoint
+
+```shell
+bin/seatunnel-connector.sh
+```
+
+## Options
+
+```text
+Usage: seatunnel-connector.sh [options]
+  Options:
+    -h, --help         Show the usage message
+    -l, --list         List all supported plugins(sources, sinks, transforms) 
+                       (default: false)
+    -o, --option-rule  Get option rule of the plugin by the plugin 
+                       identifier(connector name or transform name)
+    -pt, --plugin-type SeaTunnel plugin type, support [source, sink, 
+                       transform] 
+```
+
+## Example
+
+```shell
+# List all supported connectors(sources and sinks) and transforms
+bin/seatunnel-connector.sh -l
+# List all supported sinks
+bin/seatunnel-connector.sh -l -pt sink
+# Get option rule of the connector or transform by the name
+bin/seatunnel-connector.sh -o Paimon
+# Get option rule of paimon sink
+bin/seatunnel-connector.sh -o Paimon -pt sink
+```
+

docs/en/concept/JobEnvConfig.md

@@ -1,4 +1,4 @@
-# JobEnvConfig
+# Job Env Config
 
 This document describes env configuration information, the common parameters can be used in all engines. In order to better distinguish between engine parameters, the additional parameters of other engine need to carry a prefix.
 In flink engine, we use `flink.` as the prefix. In the spark engine, we do not use any prefixes to modify parameters, because the official spark parameters themselves start with `spark.`
@@ -29,6 +29,10 @@ In `STREAMING` mode, checkpoints is required, if you do not set it, it will be o
 
 This parameter configures the parallelism of source and sink.
 
+### job.retry.times
+
+Used to control the default retry times when a job fails. The default value is 3, and it only works in the Zeta engine.
+
 ### shade.identifier
 
 Specify the method of encryption, if you didn't have the requirement for encrypting or decrypting config files, this option can be ignored.

docs/en/concept/config.md

@@ -12,6 +12,8 @@ configure the Config file.
 The main format of the Config file is `hocon`, for more details of this format type you can refer to [HOCON-GUIDE](https://github.com/lightbend/config/blob/main/HOCON.md),
 BTW, we also support the `json` format, but you should know that the name of the config file should end with `.json`
 
+We also support the `SQL` format, for details, please refer to the [SQL configuration](sql-config.md) file.
+
 ## Example
 
 Before you read on, you can find config file
@@ -64,6 +66,19 @@ sink {
 }
 ```
 
+#### multi-line support
+
+In `hocon`, multiline strings are supported, which allows you to include extended passages of text without worrying about newline characters or special formatting. This is achieved by enclosing the text within triple quotes **`"""`** . For example:
+
+```
+var = """
+Apache SeaTunnel is a
+next-generation high-performance,
+distributed, massive data integration tool.
+"""
+sql = """ select * from "table" """
+```
+
 ### json
 
 ```json
@@ -193,6 +208,116 @@ configured with these two parameters, because in SeaTunnel, there is a default c
 parameters are not configured, then the generated data from the last module of the previous node will be used.
 This is much more convenient when there is only one source.
 
+## Config variable substitution
+
+In config file we can define some variables and replace it in run time. **This is only support `hocon` format file**.
+
+```hocon
+env {
+  job.mode = "BATCH"
+  job.name = ${jobName}
+  parallelism = 2
+}
+
+source {
+  FakeSource {
+    result_table_name = ${resName}
+    row.num = ${rowNum}
+    string.template = ${strTemplate}
+    int.template = [20, 21]
+    schema = {
+      fields {
+        name = ${nameType}
+        age = "int"
+      }
+    }
+  }
+}
+
+transform {
+    sql {
+      source_table_name = "fake"
+      result_table_name = "sql"
+      query = "select * from "${resName}" where name = '"${nameVal}"' "
+    }
+
+}
+
+sink {
+  Console {
+     source_table_name = "sql"
+     username = ${username}
+     password = ${password}
+     blankSpace = ${blankSpace}
+  }
+}
+
+```
+
+In the above config, we define some variables, like `${rowNum}`, `${resName}`.
+We can replace those parameters with this shell command:
+
+```shell
+./bin/seatunnel.sh -c <this_config_file> 
+-i jobName='st var job' 
+-i resName=fake 
+-i rowNum=10 
+-i strTemplate=['abc','d~f','h i'] 
+-i nameType=string 
+-i nameVal=abc 
+-i username=seatunnel=2.3.1 
+-i password='$a^b%c.d~e0*9(' 
+-i blankSpace='2023-12-26 11:30:00' 
+-e local
+```
+
+Then the final submitted config is:
+
+```hocon
+env {
+  job.mode = "BATCH"
+  job.name = "st var job"
+  parallelism = 2
+}
+
+source {
+  FakeSource {
+    result_table_name = "fake"
+    row.num = 10
+    string.template = ["abc","d~f","h i"]
+    int.template = [20, 21]
+    schema = {
+      fields {
+        name = string
+        age = "int"
+      }
+    }
+  }
+}
+
+transform {
+    sql {
+      source_table_name = "fake"
+      result_table_name = "sql"
+      query = "select * from fake where name = 'abc' "
+    }
+
+}
+
+sink {
+  Console {
+     source_table_name = "sql"
+     username = "seatunnel=2.3.1"
+     password = "$a^b%c.d~e0*9("
+     blankSpace = "2023-12-26 11:30:00"
+  }
+}
+```
+
+Some Notes:
+- quota with `'` if the value has space ` ` or special character (like `(`)
+- if the replacement variables is in `"` or `'`, like `resName` and `nameVal`, you need add `"`
+
 ## What's More
 
 If you want to know the details of this format configuration, Please