Skip to content

Commit 0f0f304

Browse files
michaelkoepfmichaelkoepf
authored
[docs] Adapt docs for bind.listeners config option (#759)
1 parent 409ae64 commit 0f0f304

File tree

4 files changed

+93
-70
lines changed

4 files changed

+93
-70
lines changed

‎website/docs/install-deploy/deploying-distributed-cluster.md‎

Lines changed: 66 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -30,33 +30,37 @@ This page provides instructions on how to deploy a *distributed cluster* for Flu
3030

3131
Fluss runs on all *UNIX-like environments*, e.g. **Linux**, **Mac OS X**.
3232
To build a distributed cluster, you need to have at least two nodes.
33-
This doc provides a simple example of how to deploy a distributed cluster on three nodes.
33+
This doc provides a simple example of how to deploy a distributed cluster on four nodes.
3434

3535
### Software Requirements
3636

37-
Before you start to set up the system, make sure you have the following software installed **on each node**:
38-
- **Java 17** or higher (Java 8 and Java 11 are not recommended)
39-
- **Zookeeper 3.6.0** or higher (It is not recommended to use zookeeper versions below 3.6.0)
37+
Before you start to set up the system, make sure you have installed **Java 17** or higher **on each node** in your cluster.
38+
Java 8 and Java 11 are not recommended.
4039

41-
If your cluster does not fulfill these software requirements you will need to install/upgrade it.
40+
Additionally, you need a running **ZooKeeper** cluster with version 3.6.0 or higher.
41+
We do not recommend to use ZooKeeper versions below 3.6.0.
42+
For further information how to deploy a distributed ZooKeeper cluster, see [Running Replicated ZooKeeper](https://zookeeper.apache.org/doc/r3.6.0/zookeeperStarted.html#sc_RunningReplicatedZooKeeper).
43+
44+
If your cluster does not fulfill these software requirements, you will need to install/upgrade them.
4245

4346
### `JAVA_HOME` Configuration
4447

4548
Flink requires the `JAVA_HOME` environment variable to be set on all nodes and point to the directory of your Java installation.
4649

4750
## Fluss Setup
4851

49-
This part will describe how to set up Fluss cluster consisting of one coordinator server and multiple tablet servers
50-
across three machines. Suppose you have three nodes have ip address:
51-
- Node1: `192.168.10.1`
52-
- Node2: `192.168.10.2`
53-
- Node3: `192.168.10.3`
52+
This part will describe how to set up a Fluss cluster consisting of one CoordinatorServer and multiple TabletServers
53+
across four machines. Suppose you have four nodes in a `192.168.10/24` subnet with the following IP address assignment:
54+
- Node0: `192.168.10.100`
55+
- Node1: `192.168.10.101`
56+
- Node2: `192.168.10.102`
57+
- Node3: `192.168.10.103`
5458

55-
Node1 will deploy the CoordinatorServer and one TabletServer, Node2 and Node3 will deploy one TabletServer.
59+
Node0 will deploy a CoordinatorServer instance. Node1, Node2 and Node3 will deploy one TabletServer instance, respectively.
5660

5761
### Preparation
5862

59-
1. Make sure ZooKeeper has been deployed, and assuming the ZooKeeper address is `192.168.10.99:2181`. see [Running zookeeper cluster](https://zookeeper.apache.org/doc/r3.6.0/zookeeperStarted.html#sc_RunningReplicatedZooKeeper) to deploy a distributed ZooKeeper.
63+
1. Make sure ZooKeeper has been deployed. We assume that ZooKeeper listens on `192.168.10.199:2181`.
6064

6165
2. Download Fluss
6266

@@ -70,62 +74,85 @@ cd fluss-$FLUSS_VERSION$/
7074

7175
### Configuring Fluss
7276

73-
After having extracted the archived files, you need to configure Fluss for the cluster by editing `conf/server.yaml`
77+
After having extracted the archived files, you need to configure Fluss for a distributed deployment.
78+
We will use the _default config file_ (`conf/server.yaml`) to configure Fluss.
79+
Adapt the `server.yaml` on each node as follows.
80+
81+
**Node0**
7482

75-
For **Node1**, the config is as follows:
76-
```yaml
77-
coordinator.host: 192.168.10.1
78-
coordinator.port: 9123
79-
zookeeper.address: 192.168.10.99:2181
83+
```yaml title="server.yaml"
84+
# coordinator server
85+
bind.listeners: FLUSS://192.168.10.100:9123
86+
87+
zookeeper.address: 192.168.10.199:2181
8088
zookeeper.path.root: /fluss
8189

82-
tablet-server.host: 192.168.10.1
83-
tablet-server.id: 1
90+
remote.data.dir: /tmp/fluss-remote-data
8491
```
8592
86-
For **Node2**, the config is as follows:
87-
```yaml
88-
zookeeper.address: 192.168.10.99:2181
93+
**Node1**
94+
95+
```yaml title="server.yaml"
96+
# tablet server
97+
bind.listeners: FLUSS://192.168.10.101:9123 # alternatively, setting the port to 0 assigns a random port
98+
tablet-server.id: 1
99+
100+
zookeeper.address: 192.168.10.199:2181
89101
zookeeper.path.root: /fluss
90102

91-
tablet-server.host: 192.168.10.2
92-
tablet-server.id: 2
103+
remote.data.dir: /tmp/fluss-remote-data
93104
```
94105
95-
For **Node3**, the config is as follows:
96-
```yaml
97-
zookeeper.address: 192.168.10.99:2181
106+
**Node2**
107+
108+
```yaml title="server.yaml"
109+
# tablet server
110+
bind.listeners: FLUSS://192.168.10.102:9123 # alternatively, setting the port to 0 assigns a random port
111+
tablet-server.id: 2
112+
113+
zookeeper.address: 192.168.10.199:2181
98114
zookeeper.path.root: /fluss
99115

100-
tablet-server.host: 192.168.10.3
116+
remote.data.dir: /tmp/fluss-remote-data
117+
```
118+
119+
**Node3**
120+
```yaml title="server.yaml"
121+
# tablet server
122+
bind.listeners: FLUSS://192.168.10.103:9123 # alternatively, setting the port to 0 assigns a random port
101123
tablet-server.id: 3
124+
125+
zookeeper.address: 192.168.10.199:2181
126+
zookeeper.path.root: /fluss
127+
128+
remote.data.dir: /tmp/fluss-remote-data
102129
```
103130
104131
:::note
105-
- `tablet-server.id` is the unique id of the TabletServer, if you have multiple TabletServers, you should set different id for each TabletServer.
106-
- In this example, we only set the properties that must be configured, and for some other properties, you can refer to [Configuration](maintenance/configuration.md) for more details.
132+
- `tablet-server.id` is the unique id of the TabletServer. If you have multiple TabletServers, you should set a different id for each TabletServer.
133+
- In this example, we only set the mandatory properties. For additional properties, you can refer to [Configuration](maintenance/configuration.md) for more details.
107134
:::
108135

109136
### Starting Fluss
110137

111-
To start Fluss, you should first to start a CoordinatorServer in **node1** and
112-
then start each TabletServer in **node1**, **node2**, **node3**. The command is as follows:
138+
To deploy a distributed Fluss cluster, you should first start a CoordinatorServer instance on **Node0**.
139+
Then, start a TabletServer instance on **Node1**, **Node2**, and **Node3**, respectively.
113140

114-
#### Starting CoordinatorServer
141+
**CoordinatorServer**
115142

116-
In **node1**, starting a CoordinatorServer:
143+
On **Node0**, start a CoordinatorServer as follows.
117144
```shell
118145
./bin/coordinator-server.sh start
119146
```
120147

121-
#### Starting TabletServer
148+
**TabletServer**
122149

123-
In **node1**, **node2**, **node3**, starting a TabletServer is as follows:
150+
On **Node1**, **Node2** and **Node3**, start a TabletServer as follows.
124151
```shell
125152
./bin/tablet-server.sh start
126153
```
127154

128-
After that, the Fluss distributed cluster is started.
155+
After that, you have successfully deployed a distributed Fluss cluster.
129156

130157
## Interacting with Fluss
131158

@@ -148,7 +175,7 @@ In Flink SQL client, a catalog is created and named by executing the following q
148175
```sql title="Flink SQL"
149176
CREATE CATALOG fluss_catalog WITH (
150177
'type' = 'fluss',
151-
'bootstrap.servers' = '192.168.10.1:9123'
178+
'bootstrap.servers' = '192.168.10.100:9123'
152179
);
153180
```
154181

‎website/docs/install-deploy/deploying-with-docker.md‎

Lines changed: 20 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ docker run \
8181
--name coordinator-server \
8282
--network=fluss-demo \
8383
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
84-
coordinator.host: coordinator-server" \
84+
bind.listeners: FLUSS://coordinator-server:9123" \
8585
-p 9123:9123 \
8686
-d fluss/fluss:$FLUSS_VERSION$ coordinatorServer
8787
```
@@ -100,9 +100,8 @@ docker run \
100100
--name tablet-server \
101101
--network=fluss-demo \
102102
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
103-
tablet-server.host: tablet-server
103+
bind.listeners: FLUSS://tablet-server:9124
104104
tablet-server.id: 0
105-
tablet-server.port: 9124
106105
data.dir: /tmp/fluss/data
107106
remote.data.dir: /tmp/fluss/remote-data" \
108107
-p 9124:9124 \
@@ -115,47 +114,44 @@ remote.data.dir: /tmp/fluss/remote-data" \
115114
In a production environment, you need to start multiple Fluss TabletServer nodes.
116115
Here we start 3 Fluss TabletServer nodes in daemon and connect to Zookeeper. The command is as follows:
117116

118-
1. start tablet-server-0
117+
1. Start tablet-server-0
119118
```bash
120119
docker run \
121120
--name tablet-server-0 \
122121
--network=fluss-demo \
123122
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
124-
tablet-server.host: tablet-server-0
123+
bind.listeners: FLUSS://tablet-server-0:9124
125124
tablet-server.id: 0
126-
tablet-server.port: 9124
127125
data.dir: /tmp/fluss/data/tablet-server-0
128126
remote.data.dir: /tmp/fluss/remote-data" \
129127
-p 9124:9124 \
130128
--volume shared-tmpfs:/tmp/fluss \
131129
-d fluss/fluss:$FLUSS_VERSION$ tabletServer
132130
```
133131

134-
2. start tablet-server-1
132+
2. Start tablet-server-1
135133
```bash
136134
docker run \
137135
--name tablet-server-1 \
138136
--network=fluss-demo \
139137
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
140-
tablet-server.host: tablet-server-1
138+
bind.listeners: FLUSS://tablet-server-1:9125
141139
tablet-server.id: 1
142-
tablet-server.port: 9125
143140
data.dir: /tmp/fluss/data/tablet-server-1
144141
remote.data.dir: /tmp/fluss/remote-data" \
145142
-p 9125:9125 \
146143
--volume shared-tmpfs:/tmp/fluss \
147144
-d fluss/fluss:$FLUSS_VERSION$ tabletServer
148145
```
149146

150-
3. start tablet-server-2
147+
3. Start tablet-server-2
151148
```bash
152149
docker run \
153150
--name tablet-server-2 \
154151
--network=fluss-demo \
155152
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
156-
tablet-server.host: tablet-server-2
153+
bind.listeners: FLUSS://tablet-server-2:9126
157154
tablet-server.id: 2
158-
tablet-server.port: 9126
159155
data.dir: /tmp/fluss/data/tablet-server-2
160156
remote.data.dir: /tmp/fluss/remote-data" \
161157
-p 9126:9126 \
@@ -179,7 +175,7 @@ to interact with Fluss.
179175

180176
#### Start Flink Cluster
181177

182-
1. start jobManager
178+
1. Start jobManager
183179

184180
```bash
185181
docker run \
@@ -191,7 +187,7 @@ docker run \
191187
-d fluss/quickstart-flink:1.20-$FLUSS_VERSION_SHORT$ jobmanager
192188
```
193189

194-
2. start taskManager
190+
2. Start taskManager
195191

196192
```bash
197193
docker run \
@@ -253,7 +249,7 @@ services:
253249
- |
254250
FLUSS_PROPERTIES=
255251
zookeeper.address: zookeeper:2181
256-
coordinator.host: coordinator-server
252+
bind.listeners: FLUSS://coordinator-server:9123
257253
remote.data.dir: /tmp/fluss/remote-data
258254
tablet-server:
259255
image: fluss/fluss:$FLUSS_VERSION$
@@ -264,7 +260,7 @@ services:
264260
- |
265261
FLUSS_PROPERTIES=
266262
zookeeper.address: zookeeper:2181
267-
tablet-server.host: tablet-server
263+
bind.listeners: FLUSS://tablet-server:9123
268264
tablet-server.id: 0
269265
kv.snapshot.interval: 0s
270266
data.dir: /tmp/fluss/data
@@ -298,7 +294,7 @@ services:
298294
- |
299295
FLUSS_PROPERTIES=
300296
zookeeper.address: zookeeper:2181
301-
coordinator.host: coordinator-server
297+
bind.listeners: FLUSS://coordinator-server:9123
302298
remote.data.dir: /tmp/fluss/remote-data
303299
tablet-server-0:
304300
image: fluss/fluss:$FLUSS_VERSION$
@@ -309,7 +305,7 @@ services:
309305
- |
310306
FLUSS_PROPERTIES=
311307
zookeeper.address: zookeeper:2181
312-
tablet-server.host: tablet-server-0
308+
bind.listeners: FLUSS://tablet-server-0:9123
313309
tablet-server.id: 0
314310
kv.snapshot.interval: 0s
315311
data.dir: /tmp/fluss/data/tablet-server-0
@@ -325,7 +321,7 @@ services:
325321
- |
326322
FLUSS_PROPERTIES=
327323
zookeeper.address: zookeeper:2181
328-
tablet-server.host: tablet-server-1
324+
bind.listeners: FLUSS://tablet-server-1:9123
329325
tablet-server.id: 1
330326
kv.snapshot.interval: 0s
331327
data.dir: /tmp/fluss/data/tablet-server-1
@@ -341,7 +337,7 @@ services:
341337
- |
342338
FLUSS_PROPERTIES=
343339
zookeeper.address: zookeeper:2181
344-
tablet-server.host: tablet-server-2
340+
bind.listeners: FLUSS://tablet-server-2:9123
345341
tablet-server.id: 2
346342
kv.snapshot.interval: 0s
347343
data.dir: /tmp/fluss/data/tablet-server-2
@@ -387,7 +383,7 @@ services:
387383
- |
388384
FLUSS_PROPERTIES=
389385
zookeeper.address: zookeeper:2181
390-
coordinator.host: coordinator-server
386+
bind.listeners: FLUSS://coordinator-server:9123
391387
remote.data.dir: /tmp/fluss/remote-data
392388
tablet-server-0:
393389
image: fluss/fluss:$FLUSS_VERSION$
@@ -398,7 +394,7 @@ services:
398394
- |
399395
FLUSS_PROPERTIES=
400396
zookeeper.address: zookeeper:2181
401-
tablet-server.host: tablet-server-0
397+
bind.listeners: FLUSS://tablet-server-0:9123
402398
tablet-server.id: 0
403399
kv.snapshot.interval: 0s
404400
data.dir: /tmp/fluss/data/tablet-server-0
@@ -414,7 +410,7 @@ services:
414410
- |
415411
FLUSS_PROPERTIES=
416412
zookeeper.address: zookeeper:2181
417-
tablet-server.host: tablet-server-1
413+
bind.listeners: FLUSS://tablet-server-1:9123
418414
tablet-server.id: 1
419415
kv.snapshot.interval: 0s
420416
data.dir: /tmp/fluss/data/tablet-server-1
@@ -430,7 +426,7 @@ services:
430426
- |
431427
FLUSS_PROPERTIES=
432428
zookeeper.address: zookeeper:2181
433-
tablet-server.host: tablet-server-2
429+
bind.listeners: FLUSS://tablet-server-2:9123
434430
tablet-server.id: 2
435431
kv.snapshot.interval: 0s
436432
data.dir: /tmp/fluss/data/tablet-server-2

‎website/docs/maintenance/observability/quickstart.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ services:
108108
- |
109109
FLUSS_PROPERTIES=
110110
zookeeper.address: zookeeper:2181
111-
coordinator.host: coordinator-server
111+
bind.listeners: FLUSS://coordinator-server:9123
112112
remote.data.dir: /tmp/fluss/remote-data
113113
datalake.format: paimon
114114
datalake.paimon.metastore: filesystem
@@ -130,7 +130,7 @@ services:
130130
- |
131131
FLUSS_PROPERTIES=
132132
zookeeper.address: zookeeper:2181
133-
tablet-server.host: tablet-server
133+
bind.listeners: FLUSS://tablet-server:9123
134134
data.dir: /tmp/fluss/data
135135
remote.data.dir: /tmp/fluss/remote-data
136136
kv.snapshot.interval: 0s
@@ -223,7 +223,7 @@ This recreates `shared-tmpfs` and all data is lost (created tables, running jobs
223223
Make sure that the modified and added containers are up and running using
224224

225225
```shell
226-
docker ps
226+
docker container ls -a
227227
```
228228

229229
4. Now you are all set! You can visit

0 commit comments

Comments
 (0)