[Doc]Adjust Seatunnel Doc Structure (#10395)

Author: misi1987107

docs/en/developer/coding-guide.md

@@ -0,0 +1,111 @@
+# Coding Guide
+
+This guide documents an overview of the current Apache SeaTunnel modules and best practices on how to submit a high quality pull request to Apache SeaTunnel.
+
+## Modules Overview
+
+| Module Name                            | Introduction                                                                                                                                       |
+|----------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
+| seatunnel-api                          | SeaTunnel connector V2 API module                                                                                                                  |
+| seatunnel-common                       | SeaTunnel common module                                                                                                                            |
+| seatunnel-connectors-v2                | SeaTunnel connector V2 module, currently connector V2 is under development and the community will focus on it                                      |
+| seatunnel-core/seatunnel-spark-starter | SeaTunnel core starter module of connector V2 on Spark engine                                                                                      |
+| seatunnel-core/seatunnel-flink-starter | SeaTunnel core starter module of connector V2 on Flink engine                                                                                      |
+| seatunnel-core/seatunnel-starter       | SeaTunnel core starter module of connector V2 on SeaTunnel engine                                                                                  |
+| seatunnel-e2e                          | SeaTunnel end-to-end test module                                                                                                                   |
+| seatunnel-examples                     | SeaTunnel local examples module, developer can use it to do unit test and integration test                                                         |
+| seatunnel-engine                       | SeaTunnel engine module, seatunnel-engine is a new computational engine developed by the SeaTunnel Community that focuses on data synchronization. |
+| seatunnel-formats                      | SeaTunnel formats module, used to offer the ability of formatting data                                                                             |
+| seatunnel-plugin-discovery             | SeaTunnel plugin discovery module, used to offer the ability of loading SPI plugins from classpath                                                 |
+| seatunnel-transforms-v2                | SeaTunnel transform V2 module, currently transform V2 is under development and the community will focus on it                                      |
+| seatunnel-translation                  | SeaTunnel translation module, used to adapt Connector V2 and other computing engines such as Spark, Flink etc...                                   |
+
+## How To Submit A High Quality Pull Request
+
+1. Create entity classes using annotations in the `lombok` plugin (`@Data` `@Getter` `@Setter` `@NonNull` etc...) to reduce the amount of code. It's a good practice to prioritize the use of lombok plugins in your coding process.
+
+2. If you need to use log4j to print logs in a class, preferably use the annotation `@Slf4j` in the `lombok` plugin.
+
+3. SeaTunnel uses issue to track logical issues, including bugs and improvements, and uses Github's pull requests to manage the review and merge of specific code changes. So making a clear issue or pull request helps the community better understand the developer's intent. The best practice of creating issue or pull request is as the following shown:
+
+   > [purpose] [module name] [sub-module name] Description
+
+   1. Pull request purpose includes: `Hotfix`, `Feature`, `Improve`, `Docs`, `WIP`. Note that if your pull request's purpose is `WIP`, then you need to use github's draft pull request
+   2. Issue purpose includes: `Feature`, `Bug`, `Docs`, `Discuss`
+   3. Module name: the current pull request or issue involves the name of the module, for example: `Core`, `Connector-V2`, `Connector-V1`, etc.
+   4. Sub-module name: the current pull request or issue involves the name of the sub-module, for example:`File` `Redis` `Hbase` etc.
+   5. Description: provide a brief, clear summary of the current pull request and issue's main goals and aim for a title that conveys the core purpose at a glance.
+
+   Tips:**For more details, you can refer to [Issue Guide](https://seatunnel.apache.org/community/contribution_guide/contribute#issue) and [Pull Request Guide](https://seatunnel.apache.org/community/contribution_guide/contribute#pull-request)**
+
+4. Code segments are never repeated. If a code segment is used multiple times, define it multiple times is not a good option, make it a public segment for other modules to use is a best practice.
+
+5. When throwing an exception, throw it along with a hint message and the exception should be smaller in scope. Throwing overly broad exceptions promotes complex error handling code that is more likely to contain security vulnerabilities. For example, if your connector encounters an `IOException` while reading data, a reasonable approach would be to the following:
+
+   ```java
+   try {
+       // read logic
+   } catch (IOException e) {
+       throw SeaTunnelORCFormatException("This orc file is corrupted, please check it", e);
+   }
+   ```
+
+6. The Apache project has very strict licensing requirements, so every file in an Apache project should contain a license statement. Check that each new file you add contains the `Apache License Header` before submitting pull request:
+
+   ```java
+   /*
+    * Licensed to the Apache Software Foundation (ASF) under one or more
+    * contributor license agreements.  See the NOTICE file distributed with
+    * this work for additional information regarding copyright ownership.
+    * The ASF licenses this file to You under the Apache License, Version 2.0
+    * (the "License"); you may not use this file except in compliance with
+    * the License.  You may obtain a copy of the License at
+    *
+    *    http://www.apache.org/licenses/LICENSE-2.0
+    *
+    * Unless required by applicable law or agreed to in writing, software
+    * distributed under the License is distributed on an "AS IS" BASIS,
+    * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    * See the License for the specific language governing permissions and
+    * limitations under the License.
+    */
+   ```
+
+7. Apache SeaTunnel uses `Spotless` for code style and formatting checks. You could run the following command and `Spotless` will automatically fix the code style and formatting errors for you:
+
+   ```shell
+   ./mvnw spotless:apply
+   ```
+
+8. Before you submit your pull request, make sure the project will compile properly after adding your code, you can use the following commands to package the whole project:
+
+   ```shell
+   # multi threads compile
+   ./mvnw -T 1C clean package
+   ```
+
+   ```shell
+   # single thread compile
+   ./mvnw clean package
+   ```
+
+9. Before submitting pull request, do a full unit test and integration test locally can better verify the functionality of your code, best practice is to use the `seatunnel-examples` module's ability to self-test to ensure that the multi-engine is running properly and the results are correct.
+
+10. If you submit a pull request with a feature that requires updated documentation, always remember to update the documentation.
+
+11. Submit the pull request of connector type can write e2e test to ensure the robustness and robustness of the code, e2e test should include the full data type, and e2e test as little as possible to initialize the docker image, write the test cases of sink and source together to reduce the loss of resources, while using asynchronous features to ensure the stability of the test. A good example can be found at: [MongodbIT.java](https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-mongodb-e2e/src/test/java/org/apache/seatunnel/e2e/connector/v2/mongodb/MongodbIT.java)
+
+12. The priority of property permission in the class is set to `private`, and mutability is set to `final`, which can be changed reasonably if special circumstances are encountered.
+
+13. The properties in the class and method parameters prefer to use the base type(int boolean double float...), not recommended to use the wrapper type(Integer Boolean Double Float...), if encounter special circumstances reasonable change.
+
+14. When developing a sink connector you need to be aware that the sink will be serialized, and if some properties cannot be serialized, encapsulate the properties into classes and use the singleton pattern.
+
+15. If there are multiple `if` process judgments in the code flow, try to simplify the flow to multiple ifs instead of if-else-if.
+
+16. Pull request has the characteristic of single responsibility, not allowed to include irrelevant code of the feature in pull request, once this situation deal with their own branch before submitting pull request, otherwise the Apache SeaTunnel community will actively close pull request.
+
+17. Contributors should be responsible for their own pull request. If your pull request contains new features or modifies old features, add test cases or e2e tests to prove the reasonableness and functional integrity of your pull request is a good practice.
+
+18. If you think which part of the community's current code is unreasonable (especially the core `core` module and the `api` module), the function needs to be updated or modified, the first thing to do is to propose a `discuss issue` or `email` with the community to discuss the need to modify this part of the function, if the community agrees to submit pull request again, do not submit the issue and pull request directly without discussion, so the community will directly consider this pull request is useless, and will be closed down.
+

docs/en/developer/contribute-plugin.md

@@ -0,0 +1,5 @@
+# Contribute Connector-V2 Plugins
+
+If you want to contribute Connector-V2, please click the Connector-V2 Contribution Guide below for reference. It can help you enter development more quickly.
+
+[Connector-v2 Contribution Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.md)

docs/en/developer/contribute-transform-v2-guide.md

@@ -0,0 +1,5 @@
+# Contribute Transform-V2 Plugins
+
+If you want to contribute Transform-V2, please click the Transform-V2 Contribution Guide below for reference. It can help you enter development more quickly.
+
+[Connector-v2 Contribution Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-transforms-v2/README.md)

docs/en/developer/docs-format-specification.md

@@ -0,0 +1,28 @@
+# Docs Format Specification
+## Admonitions
+
+We have special admonitions syntax by wrapping text with a set of 3 colons, followed by a label denoting its type. When you want to emphasize the content, it is recommended to use admonitions.
+
+In use, the following specifications need to be followed:
+
+- Tip: mainly used for operational  tips and tricks.
+
+- Note: used for more details and explanations.
+
+- Caution: used for warnings and precautions.
+
+You may also specify an optional title. Here are the examples of admonitions syntax:
+
+```Markdown
+:::tip Tip
+Some content with tips
+:::
+
+:::info Note
+Some content with explanations
+:::
+
+:::caution Warning
+Some content with precuations and warnings
+:::
+```

docs/en/developer/how-to-create-your-connector.md

@@ -0,0 +1,3 @@
+# Develop Your Own Connector
+
+If you want to develop your own connector for the new SeaTunnel connector API (Connector V2), please check [here](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.md).

docs/en/developer/new-license.md

@@ -0,0 +1,53 @@
+# How To Add New License
+
+### ASF 3RD PARTY LICENSE POLICY
+
+You have to pay attention to the following open-source software protocols which Apache projects support when you intend to add a new feature to the SeaTunnel (or other Apache projects), which functions refers to other open-source software references.
+
+[ASF 3RD PARTY LICENSE POLICY](https://apache.org/legal/resolved.html)
+
+If the 3rd party software is not present at the above policy, we wouldn't accept your code.
+
+### How to Legally Use 3rd Party Open-source Software In The SeaTunnel
+
+Moreover, when we intend to refer a new software ( not limited to 3rd party jar, text, CSS, js, pics, icons, audios etc and modifications based on 3rd party files) to our project, we need to use them legally in addition to the permission of ASF. Refer to the following article:
+
+* [COMMUNITY-LED DEVELOPMENT "THE APACHE WAY"](https://apache.org/dev/licensing-howto.html)
+
+For example, we should contain the NOTICE file (most of open-source project has NOTICE file, generally under root directory) of ZooKeeper in our project when we are using ZooKeeper. As the Apache explains, "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work.
+
+We are not going to dive into every 3rd party open-source license policy in here, you may look up them if interested.
+
+### SeaTunnel-License Check Rules
+
+In general, we would have our License-check scripts to our project. SeaTunnel-License-Check is provided by [SkyWalking](https://github.com/apache/skywalking) which differ a bit from other open-source projects. All in all, we are trying to make sure avoiding the license issues at the first time.
+
+We need to follow the following steps when we need to add new jars or external resources:
+
+* Add the name and the version of the jar file in the known-dependencies.txt
+* Add relevant maven repository address under 'seatunnel-dist/release-docs/LICENSE' directory
+* Append relevant NOTICE files under 'seatunnel-dist/release-docs/NOTICE' directory and make sure they are no different to the original repository
+* Add relevant source code protocols under 'seatunnel-dist/release-docs/licenses' directory and the file name should be named as license+filename.txt. e.g.: license-zk.txt
+* check dependency license fail
+
+```
+--- /dev/fd/63 2020-12-03 03:08:57.191579482 +0000
++++ /dev/fd/62 2020-12-03 03:08:57.191579482 +0000
+@@ -1,0 +2 @@
++HikariCP-java6-2.3.13.jar
+@@ -16,0 +18 @@
++c3p0-0.9.5.2.jar
+@@ -149,0 +152 @@
++mchange-commons-java-0.2.11.jar
+
+- commons-lang-2.1.3.jar
+Error: Process completed with exit code 1.
+```
+
+Generally speaking, the work of adding a jar is often not so easy to end, because it often depends on various other jars, and we also need to add corresponding licenses for these jars. In this case, we will get the error message of check dependency license fail in check. As above, we are missing the license declaration of `HikariCP-java6-2.3.13`, `c3p0`, etc. (`+` means new, `-` means need to delete ), follow the steps to add jar to add
+
+### References
+
+* [COMMUNITY-LED DEVELOPMENT "THE APACHE WAY"](https://apache.org/dev/licensing-howto.html)
+* [ASF 3RD PARTY LICENSE POLICY](https://apache.org/legal/resolved.html)
+