|
| 1 | +# 配置文件加密和解密 |
| 2 | + |
| 3 | +## 介绍 |
| 4 | + |
| 5 | +在大多数生产环境中,需要对敏感的配置项(如密码)进行加密,不能以明文形式存储。SeaTunnel 为此提供了一个方便的一站式解决方案。 |
| 6 | + |
| 7 | +## 如何使用 |
| 8 | + |
| 9 | +SeaTunnel 具备Base64编码和解码的功能,但不建议在生产环境中使用,SeaTunnel 建议用户根据自身需求,实现个性化的加密和解密逻辑。您可以参考本章节[如何实现用户自定义的加密和解密](#如何实现用户自定义的加密和解密)以获取更多相关细节。 |
| 10 | + |
| 11 | +Base64编码支持加密以下参数: |
| 12 | +- username |
| 13 | +- password |
| 14 | +- auth |
| 15 | + |
| 16 | +接下来,将展示如何快速使用 SeaTunnel 自带的 `base64` 加密功能: |
| 17 | + |
| 18 | +1. 在配置文件的环境变量(env)部分新增了一个选项 `shade.identifier`。此选项用于表示您想要使用的加密方法。 |
| 19 | +2. 在这个示例中,我们在配置文件中添加了 `shade.identifier = base64`,如下所示: |
| 20 | + |
| 21 | + ```hocon |
| 22 | + # |
| 23 | + # Licensed to the Apache Software Foundation (ASF) under one or more |
| 24 | + # contributor license agreements. See the NOTICE file distributed with |
| 25 | + # this work for additional information regarding copyright ownership. |
| 26 | + # The ASF licenses this file to You under the Apache License, Version 2.0 |
| 27 | + # (the "License"); you may not use this file except in compliance with |
| 28 | + # the License. You may obtain a copy of the License at |
| 29 | + # |
| 30 | + # http://www.apache.org/licenses/LICENSE-2.0 |
| 31 | + # |
| 32 | + # Unless required by applicable law or agreed to in writing, software |
| 33 | + # distributed under the License is distributed on an "AS IS" BASIS, |
| 34 | + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 35 | + # See the License for the specific language governing permissions and |
| 36 | + # limitations under the License. |
| 37 | + # |
| 38 | +
|
| 39 | + env { |
| 40 | + parallelism = 1 |
| 41 | + shade.identifier = "base64" |
| 42 | + } |
| 43 | +
|
| 44 | + source { |
| 45 | + MySQL-CDC { |
| 46 | + result_table_name = "fake" |
| 47 | + parallelism = 1 |
| 48 | + server-id = 5656 |
| 49 | + port = 56725 |
| 50 | + hostname = "127.0.0.1" |
| 51 | + username = "seatunnel" |
| 52 | + password = "seatunnel_password" |
| 53 | + database-name = "inventory_vwyw0n" |
| 54 | + table-name = "products" |
| 55 | + base-url = "jdbc:mysql://localhost:56725" |
| 56 | + } |
| 57 | + } |
| 58 | +
|
| 59 | + transform { |
| 60 | + } |
| 61 | +
|
| 62 | + sink { |
| 63 | + # 将数据输出到 Clickhouse。 |
| 64 | + Clickhouse { |
| 65 | + host = "localhost:8123" |
| 66 | + database = "default" |
| 67 | + table = "fake_all" |
| 68 | + username = "seatunnel" |
| 69 | + password = "seatunnel_password" |
| 70 | +
|
| 71 | + # cdc options |
| 72 | + primary_key = "id" |
| 73 | + support_upsert = true |
| 74 | + } |
| 75 | + } |
| 76 | + ``` |
| 77 | +3. 通过Shell脚本调用不同的计算引擎来对配置文件进行加密操作。在本示例中,我们使用 Zeta 引擎对配置文件进行加密。 |
| 78 | + |
| 79 | + ```shell |
| 80 | + ${SEATUNNEL_HOME}/bin/seatunnel.sh --config config/v2.batch.template --encrypt |
| 81 | + ``` |
| 82 | + |
| 83 | + 然后,您可以在终端中看到加密后的配置文件。 |
| 84 | + |
| 85 | + ```log |
| 86 | + 2023-02-20 17:50:58,319 INFO org.apache.seatunnel.core.starter.command.ConfEncryptCommand - Encrypt config: |
| 87 | + { |
| 88 | + "env" : { |
| 89 | + "parallelism" : 1, |
| 90 | + "shade.identifier" : "base64" |
| 91 | + }, |
| 92 | + "source" : [ |
| 93 | + { |
| 94 | + "base-url" : "jdbc:mysql://localhost:56725", |
| 95 | + "hostname" : "127.0.0.1", |
| 96 | + "password" : "c2VhdHVubmVsX3Bhc3N3b3Jk", |
| 97 | + "port" : 56725, |
| 98 | + "database-name" : "inventory_vwyw0n", |
| 99 | + "parallelism" : 1, |
| 100 | + "result_table_name" : "fake", |
| 101 | + "table-name" : "products", |
| 102 | + "plugin_name" : "MySQL-CDC", |
| 103 | + "server-id" : 5656, |
| 104 | + "username" : "c2VhdHVubmVs" |
| 105 | + } |
| 106 | + ], |
| 107 | + "transform" : [], |
| 108 | + "sink" : [ |
| 109 | + { |
| 110 | + "database" : "default", |
| 111 | + "password" : "c2VhdHVubmVsX3Bhc3N3b3Jk", |
| 112 | + "support_upsert" : true, |
| 113 | + "host" : "localhost:8123", |
| 114 | + "plugin_name" : "Clickhouse", |
| 115 | + "primary_key" : "id", |
| 116 | + "table" : "fake_all", |
| 117 | + "username" : "c2VhdHVubmVs" |
| 118 | + } |
| 119 | + ] |
| 120 | + } |
| 121 | + ``` |
| 122 | +4. 当然,不仅支持加密配置文件,还支持对配置文件的解密。如果用户想要查看解密后的配置文件,可以执行以下命令: |
| 123 | + |
| 124 | + ```shell |
| 125 | + ${SEATUNNEL_HOME}/bin/seatunnel.sh --config config/v2.batch.template --decrypt |
| 126 | + ``` |
| 127 | + |
| 128 | +## 如何实现用户自定义的加密和解密 |
| 129 | + |
| 130 | +如果您希望自定义加密方法和加密配置,本章节将帮助您解决问题。 |
| 131 | + |
| 132 | +1. 创建一个 java maven 项目 |
| 133 | + |
| 134 | +2. 在 maven 依赖中添加 `seatunnel-api` 模块,如下所示: |
| 135 | + |
| 136 | + ```xml |
| 137 | + <dependency> |
| 138 | + <groupId>org.apache.seatunnel</groupId> |
| 139 | + <artifactId>seatunnel-api</artifactId> |
| 140 | + <version>${seatunnel.version}</version> |
| 141 | + </dependency> |
| 142 | + ``` |
| 143 | +3. 创建一个 java 类并实现 `ConfigShade` 接口,该接口包含以下方法: |
| 144 | + |
| 145 | + ```java |
| 146 | + /** |
| 147 | + * The interface that provides the ability to encrypt and decrypt {@link |
| 148 | + * org.apache.seatunnel.shade.com.typesafe.config.Config} |
| 149 | + */ |
| 150 | + public interface ConfigShade { |
| 151 | + |
| 152 | + /** |
| 153 | + * The unique identifier of the current interface, used it to select the correct {@link |
| 154 | + * ConfigShade} |
| 155 | + */ |
| 156 | + String getIdentifier(); |
| 157 | + |
| 158 | + /** |
| 159 | + * Encrypt the content |
| 160 | + * |
| 161 | + * @param content The content to encrypt |
| 162 | + */ |
| 163 | + String encrypt(String content); |
| 164 | + |
| 165 | + /** |
| 166 | + * Decrypt the content |
| 167 | + * |
| 168 | + * @param content The content to decrypt |
| 169 | + */ |
| 170 | + String decrypt(String content); |
| 171 | + |
| 172 | + /** To expand the options that user want to encrypt */ |
| 173 | + default String[] sensitiveOptions() { |
| 174 | + return new String[0]; |
| 175 | + } |
| 176 | + } |
| 177 | + ``` |
| 178 | +4. 在 `resources/META-INF/services` 目录下添加 `org.apache.seatunnel.api.configuration.ConfigShade` |
| 179 | +5. 将其打成 jar 包, 并添加到 `${SEATUNNEL_HOME}/lib` 目录下。 |
| 180 | +6. 将选项 `shade.identifier` 的值更改为上面定义在配置文件中的 `ConfigShade#getIdentifier` 的值。 |
| 181 | + |
0 commit comments