Skip to content

Commit 87e9355

Browse files
duzhendiduzhendi
authored
[Doc][Improve] support chinese [docs/zh/connector-v2/sink/IoTDB.md] (#8722)
1 parent f67d933 commit 87e9355

File tree

1 file changed

+221
-0
lines changed

1 file changed

+221
-0
lines changed

‎docs/zh/connector-v2/sink/IoTDB.md

+221
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# IoTDB
2+
3+
> IoTDB数据接收器
4+
5+
## 支持引擎
6+
7+
> Spark<br/>
8+
> Flink<br/>
9+
> SeaTunnel Zeta<br/>
10+
11+
## 描述
12+
13+
将数据写入IoTDB。
14+
15+
## Using Dependency
16+
17+
### 适用于Spark/Flink引擎
18+
19+
> 1. 您需要确保[jdbc驱动程序jar包](https://mvnrepository.com/artifact/org.apache.iotdb/iotdb-jdbc)已放置在目录`${SEATUNNEL_HOME}/plugins/`中。
20+
21+
### 适用于SeaTunnelZeta引擎
22+
23+
> 1. 您需要确保[jdbc驱动程序jar包](https://mvnrepository.com/artifact/org.apache.iotdb/iotdb-jdbc)已放置在目录“${SEATUNNEL_HOME}/lib/”中。
24+
25+
## 主要特性
26+
27+
- [x] [exactly-once](../../concept/connector-v2-features.md)
28+
29+
IoTDB通过幂等写支持 `exactly-once` 功能。如果两条数据
30+
如果使用相同的`key`和`timestamp`，新数据将覆盖旧数据。
31+
32+
:::提示
33+
34+
IoTDB和Spark之间存在节俭版本冲突。因此，您需要执行`rm -f $SPARK_HOME/jars/libthrift*` 和 `cp $IOTDB_HOME/lib/libthrift* $SPARK_HOME/jars/`来解决这个问题。
35+
36+
:::
37+
38+
## 支持的数据源信息
39+
40+
| 数据源 | Supported 版本 | 地址 |
41+
|------------|--------------------|----------------|
42+
| IoTDB | `>= 0.13.0` | localhost:6667 |
43+
44+
## 数据类型映射
45+
46+
| IotDB 数据类型 | SeaTunnel 数据类型 |
47+
|-----------------|---------------------|
48+
| BOOLEAN | BOOLEAN |
49+
| INT32 | TINYINT |
50+
| INT32 | SMALLINT |
51+
| INT32 | INT |
52+
| INT64 | BIGINT |
53+
| FLOAT | FLOAT |
54+
| DOUBLE | DOUBLE |
55+
| TEXT | STRING |
56+
57+
## Sink 选项
58+
59+
| 名称 | 类型 | 是否必传 | 默认值 | 描述 |
60+
|-----------------------------|---------|----------|--------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
61+
| node_urls | String | 是 | - | `IoTDB` 集群地址，格式为 `"host1:port"` 或 `"host1:port,host2:port"` |
62+
| username | String | 是 | - | `IoTDB` 用户的用户名 |
63+
| password | String | 是 | - | `IoTDB` 用户的密码 |
64+
| key_device | String | 是 | - | 在SeaTunnelRow中指定`IoTDB`设备ID的字段名 |
65+
| key_timestamp | String | 否 | processing time | 在SeaTunnelRow中指定`IoTDB`时间戳的字段名。如果未指定，则使用处理时间作为时间戳 |
66+
| key_measurement_fields | Array | 否 | exclude `device` & `timestamp` | 在SeaTunnelRow中指定`IoTDB`测量列表的字段名称。如果未指定，则包括所有字段，但排除 `device` & `timestamp` |
67+
| storage_group | Array | 否 | - | 指定设备存储组（路径前缀） <br/> 例如: deviceId = ${storage_group} + "." + ${key_device} |
68+
| batch_size | Integer | 否 | 1024 | 对于批写入，当缓冲区的数量达到`batch_size`的数量或时间达到`batch_interval_ms`时，数据将被刷新到IoTDB中 |
69+
| max_retries | Integer | 否 | - | 刷新的重试次数 failed |
70+
| retry_backoff_multiplier_ms | Integer | 否 | - | 用作生成下一个退避延迟的乘数 |
71+
| max_retry_backoff_ms | Integer | 否 | - | 尝试重试对`IoTDB`的请求之前等待的时间量 |
72+
| default_thrift_buffer_size | Integer | 否 | - | 在`IoTDB`客户端中节省初始化缓冲区大小 |
73+
| max_thrift_frame_size | Integer | 否 | - | 在`IoTDB`客户端中节约最大帧大小 |
74+
| zone_id | string | 否 | - | `IoTDB` java.time.ZoneId client |
75+
| enable_rpc_compression | Boolean | 否 | - | 在`IoTDB`客户端中启用rpc压缩 |
76+
| connection_timeout_in_ms | Integer | 否 | - | 连接到`IoTDB`时等待的最长时间（毫秒） |
77+
| common-options | | 否 | - | Sink插件常用参数，详见[Sink common Options]（../Sink common Options.md） |
78+
79+
## 示例
80+
81+
```hocon
82+
env {
83+
parallelism = 2
84+
job.mode = "BATCH"
85+
}
86+
87+
source {
88+
FakeSource {
89+
row.num = 16
90+
bigint.template = [1664035200001]
91+
schema = {
92+
fields {
93+
device_name = "string"
94+
temperature = "float"
95+
moisture = "int"
96+
event_ts = "bigint"
97+
c_string = "string"
98+
c_boolean = "boolean"
99+
c_tinyint = "tinyint"
100+
c_smallint = "smallint"
101+
c_int = "int"
102+
c_bigint = "bigint"
103+
c_float = "float"
104+
c_double = "double"
105+
}
106+
}
107+
}
108+
}
109+
```
110+
111+
上游SeaTunnelRow数据格式如下:
112+
113+
| device_name | temperature | moisture | event_ts | c_string | c_boolean | c_tinyint | c_smallint | c_int | c_bigint | c_float | c_double |
114+
|--------------------------|-------------|----------|---------------|----------|-----------|-----------|------------|-------|------------|---------|----------|
115+
| root.test_group.device_a | 36.1 | 100 | 1664035200001 | abc1 | true | 1 | 1 | 1 | 2147483648 | 1.0 | 1.0 |
116+
| root.test_group.device_b | 36.2 | 101 | 1664035200001 | abc2 | false | 2 | 2 | 2 | 2147483649 | 2.0 | 2.0 |
117+
| root.test_group.device_c | 36.3 | 102 | 1664035200001 | abc3 | false | 3 | 3 | 3 | 2147483649 | 3.0 | 3.0 |
118+
119+
### 案例1
120+
121+
只填写所需的配置。
122+
使用当前处理时间作为时间戳。并包括所有字段，但不包括`device` & `timestamp`作为测量字段
123+
124+
```hocon
125+
sink {
126+
IoTDB {
127+
node_urls = "localhost:6667"
128+
username = "root"
129+
password = "root"
130+
key_device = "device_name" # specify the `deviceId` use device_name field
131+
}
132+
}
133+
```
134+
135+
"IoTDB"数据格式的输出如下:
136+
137+
```shell
138+
IoTDB> SELECT * FROM root.test_group.* align by device;
139+
+------------------------+------------------------+--------------+-----------+--------------+---------+----------+----------+-----------+------+-----------+--------+---------+
140+
| Time| Device| temperature| moisture| event_ts| c_string| c_boolean| c_tinyint| c_smallint| c_int| c_bigint| c_float| c_double|
141+
+------------------------+------------------------+--------------+-----------+--------------+---------+----------+----------+-----------+------+-----------+--------+---------+
142+
|2023-09-01T00:00:00.001Z|root.test_group.device_a| 36.1| 100| 1664035200001| abc1| true| 1| 1| 1| 2147483648| 1.0| 1.0|
143+
|2023-09-01T00:00:00.001Z|root.test_group.device_b| 36.2| 101| 1664035200001| abc2| false| 2| 2| 2| 2147483649| 2.0| 2.0|
144+
|2023-09-01T00:00:00.001Z|root.test_group.device_c| 36.3| 102| 1664035200001| abc2| false| 3| 3| 3| 2147483649| 3.0| 3.0|
145+
+------------------------+------------------------+--------------+-----------+--------------+---------+---------+-----------+-----------+------+-----------+--------+---------+
146+
```
147+
148+
### 案例2
149+
150+
使用源事件的时间
151+
152+
```hocon
153+
sink {
154+
IoTDB {
155+
node_urls = "localhost:6667"
156+
username = "root"
157+
password = "root"
158+
key_device = "device_name" # specify the `deviceId` use device_name field
159+
key_timestamp = "event_ts" # specify the `timestamp` use event_ts field
160+
}
161+
}
162+
```
163+
164+
"IoTDB"数据格式的输出如下:
165+
166+
```shell
167+
IoTDB> SELECT * FROM root.test_group.* align by device;
168+
+------------------------+------------------------+--------------+-----------+--------------+---------+----------+----------+-----------+------+-----------+--------+---------+
169+
| Time| Device| temperature| moisture| event_ts| c_string| c_boolean| c_tinyint| c_smallint| c_int| c_bigint| c_float| c_double|
170+
+------------------------+------------------------+--------------+-----------+--------------+---------+----------+----------+-----------+------+-----------+--------+---------+
171+
|2022-09-25T00:00:00.001Z|root.test_group.device_a| 36.1| 100| 1664035200001| abc1| true| 1| 1| 1| 2147483648| 1.0| 1.0|
172+
|2022-09-25T00:00:00.001Z|root.test_group.device_b| 36.2| 101| 1664035200001| abc2| false| 2| 2| 2| 2147483649| 2.0| 2.0|
173+
|2022-09-25T00:00:00.001Z|root.test_group.device_c| 36.3| 102| 1664035200001| abc2| false| 3| 3| 3| 2147483649| 3.0| 3.0|
174+
+------------------------+------------------------+--------------+-----------+--------------+---------+---------+-----------+-----------+------+-----------+--------+---------+
175+
```
176+
177+
### 案例3
178+
179+
使用源事件的时间和限制度量字段
180+
181+
```hocon
182+
sink {
183+
IoTDB {
184+
node_urls = "localhost:6667"
185+
username = "root"
186+
password = "root"
187+
key_device = "device_name"
188+
key_timestamp = "event_ts"
189+
key_measurement_fields = ["temperature", "moisture"]
190+
}
191+
}
192+
```
193+
194+
"IoTDB"数据格式的输出如下:
195+
196+
```shell
197+
IoTDB> SELECT * FROM root.test_group.* align by device;
198+
+------------------------+------------------------+--------------+-----------+
199+
| Time| Device| temperature| moisture|
200+
+------------------------+------------------------+--------------+-----------+
201+
|2022-09-25T00:00:00.001Z|root.test_group.device_a| 36.1| 100|
202+
|2022-09-25T00:00:00.001Z|root.test_group.device_b| 36.2| 101|
203+
|2022-09-25T00:00:00.001Z|root.test_group.device_c| 36.3| 102|
204+
+------------------------+------------------------+--------------+-----------+
205+
```
206+
207+
## 更改日志
208+
209+
### 2.2.0-beta 2022-09-26
210+
211+
- 添加IoTDB数据接收器
212+
213+
### 2.3.0-beta 2022-10-20
214+
215+
- [Improve] 改进IoTDB数据接收器 ([2917](https://github.com/apache/seatunnel/pull/2917))
216+
- 支持sql语法对齐
217+
- 支持sql拆分忽略案例
218+
- 支持将拆分偏移量恢复到至少一次
219+
- 支持从RowRecord读取时间戳
220+
- [BugFix] 固定IoTDB连接器写入NPE ([3080](https://github.com/apache/seatunnel/pull/3080))
221+

0 commit comments

Comments
 (0)