Skip to content

Commit 4e2989c

Browse files
committed
[docs]Refactor the document of deploying-with-docker to use flink outside of docker
1 parent 30224fc commit 4e2989c

File tree

1 file changed

+45
-153
lines changed

1 file changed

+45
-153
lines changed

website/docs/install-deploy/deploying-with-docker.md

Lines changed: 45 additions & 153 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,9 @@ docker run \
8181
--name coordinator-server \
8282
--network=fluss-demo \
8383
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
84-
bind.listeners: FLUSS://coordinator-server:9123" \
84+
bind.listeners: FLUSS://coordinator-server:9123
85+
advertised.listeners: FLUSS://localhost:9123
86+
" \
8587
-p 9123:9123 \
8688
-d fluss/fluss:$FLUSS_VERSION$ coordinatorServer
8789
```
@@ -101,6 +103,7 @@ docker run \
101103
--network=fluss-demo \
102104
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
103105
bind.listeners: FLUSS://tablet-server:9124
106+
advertised.listeners: FLUSS://localhost:9124
104107
tablet-server.id: 0
105108
data.dir: /tmp/fluss/data
106109
remote.data.dir: /tmp/fluss/remote-data" \
@@ -121,6 +124,7 @@ docker run \
121124
--network=fluss-demo \
122125
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
123126
bind.listeners: FLUSS://tablet-server-0:9124
127+
advertised.listeners: FLUSS://localhost:9124
124128
tablet-server.id: 0
125129
data.dir: /tmp/fluss/data/tablet-server-0
126130
remote.data.dir: /tmp/fluss/remote-data" \
@@ -136,6 +140,7 @@ docker run \
136140
--network=fluss-demo \
137141
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
138142
bind.listeners: FLUSS://tablet-server-1:9125
143+
advertised.listeners: FLUSS://localhost:9125
139144
tablet-server.id: 1
140145
data.dir: /tmp/fluss/data/tablet-server-1
141146
remote.data.dir: /tmp/fluss/remote-data" \
@@ -151,6 +156,7 @@ docker run \
151156
--network=fluss-demo \
152157
--env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
153158
bind.listeners: FLUSS://tablet-server-2:9126
159+
advertised.listeners: FLUSS://localhost:9126
154160
tablet-server.id: 2
155161
data.dir: /tmp/fluss/data/tablet-server-2
156162
remote.data.dir: /tmp/fluss/remote-data" \
@@ -174,38 +180,17 @@ The following subsections will show you how to use 'Docker' to build a Flink clu
174180
to interact with Fluss.
175181

176182
#### Start Flink Cluster
183+
You can start a Flink standalone cluster refer to [Flink Environment Preparation](engine-flink/getting-started.md#preparation-when-using-flink-sql-client)
177184

178-
1. Start jobManager
179-
180-
```bash
181-
docker run \
182-
--name jobmanager \
183-
--network=fluss-demo \
184-
--env FLINK_PROPERTIES=" jobmanager.rpc.address: jobmanager" \
185-
-p 8083:8081 \
186-
--volume shared-tmpfs:/tmp/fluss \
187-
-d fluss/quickstart-flink:1.20-$FLUSS_VERSION_SHORT$ jobmanager
185+
**Note**: Make sure the [Fluss connector jar](/downloads/) already has copied to the `lib` directory of your Flink home.
186+
```shell title="<FLINK_HOME>"
187+
bin/start-cluster.sh
188188
```
189189

190-
2. Start taskManager
191-
192-
```bash
193-
docker run \
194-
--name taskmanager \
195-
--network=fluss-demo \
196-
--env FLINK_PROPERTIES=" jobmanager.rpc.address: jobmanager" \
197-
--volume shared-tmpfs:/tmp/fluss \
198-
-d fluss/quickstart-flink:1.20-$FLUSS_VERSION_SHORT$ taskmanager
199-
```
200190

201191
#### Enter into SQL-Client
202-
First, use the following command to enter pod:
203-
```shell
204-
docker exec -it jobmanager /bin/bash
205-
```
206-
207-
Then, use the following command to enter the Flink SQL CLI Container:
208-
```shell
192+
Use the following command to enter the Flink SQL CLI Container:
193+
```shell title="<FLINK_HOME>"
209194
./sql-client
210195
```
211196

@@ -215,7 +200,7 @@ Use the following SQL to create a Fluss catalog:
215200
```sql title="Flink SQL"
216201
CREATE CATALOG fluss_catalog WITH (
217202
'type' = 'fluss',
218-
'bootstrap.servers' = 'coordinator-server:9123'
203+
'bootstrap.servers' = 'localhost:9123'
219204
);
220205
```
221206

@@ -249,8 +234,12 @@ services:
249234
- |
250235
FLUSS_PROPERTIES=
251236
zookeeper.address: zookeeper:2181
252-
bind.listeners: FLUSS://coordinator-server:9123
237+
bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
238+
advertised.listeners: CLIENT://localhost:9123
239+
internal.listener.name: INTERNAL
253240
remote.data.dir: /tmp/fluss/remote-data
241+
ports:
242+
- "9123:9123"
254243
tablet-server:
255244
image: fluss/fluss:$FLUSS_VERSION$
256245
command: tabletServer
@@ -260,11 +249,15 @@ services:
260249
- |
261250
FLUSS_PROPERTIES=
262251
zookeeper.address: zookeeper:2181
263-
bind.listeners: FLUSS://tablet-server:9123
252+
bind.listeners: INTERNAL://tablet-server:0, CLIENT://tablet-server:9123
253+
advertised.listeners: CLIENT://localhost:9124
254+
internal.listener.name: INTERNAL
264255
tablet-server.id: 0
265256
kv.snapshot.interval: 0s
266257
data.dir: /tmp/fluss/data
267258
remote.data.dir: /tmp/fluss/remote-data
259+
ports:
260+
- "9124:9123"
268261
volumes:
269262
- shared-tmpfs:/tmp/fluss
270263
zookeeper:
@@ -294,8 +287,12 @@ services:
294287
- |
295288
FLUSS_PROPERTIES=
296289
zookeeper.address: zookeeper:2181
297-
bind.listeners: FLUSS://coordinator-server:9123
290+
bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
291+
advertised.listeners: CLIENT://localhost:9123
292+
internal.listener.name: INTERNAL
298293
remote.data.dir: /tmp/fluss/remote-data
294+
ports:
295+
- "9123:9123"
299296
tablet-server-0:
300297
image: fluss/fluss:$FLUSS_VERSION$
301298
command: tabletServer
@@ -305,11 +302,15 @@ services:
305302
- |
306303
FLUSS_PROPERTIES=
307304
zookeeper.address: zookeeper:2181
308-
bind.listeners: FLUSS://tablet-server-0:9123
305+
bind.listeners: INTERNAL://tablet-server-0:0, CLIENT://tablet-server-0:9123
306+
advertised.listeners: CLIENT://localhost:9124
307+
internal.listener.name: INTERNAL
309308
tablet-server.id: 0
310309
kv.snapshot.interval: 0s
311310
data.dir: /tmp/fluss/data/tablet-server-0
312311
remote.data.dir: /tmp/fluss/remote-data
312+
ports:
313+
- "9124:9123"
313314
volumes:
314315
- shared-tmpfs:/tmp/fluss
315316
tablet-server-1:
@@ -321,11 +322,15 @@ services:
321322
- |
322323
FLUSS_PROPERTIES=
323324
zookeeper.address: zookeeper:2181
324-
bind.listeners: FLUSS://tablet-server-1:9123
325+
bind.listeners: INTERNAL://tablet-server-1:0, CLIENT://tablet-server-1:9123
326+
advertised.listeners: CLIENT://localhost:9125
327+
internal.listener.name: INTERNAL
325328
tablet-server.id: 1
326329
kv.snapshot.interval: 0s
327330
data.dir: /tmp/fluss/data/tablet-server-1
328331
remote.data.dir: /tmp/fluss/remote-data
332+
ports:
333+
- "9125:9123"
329334
volumes:
330335
- shared-tmpfs:/tmp/fluss
331336
tablet-server-2:
@@ -337,11 +342,15 @@ services:
337342
- |
338343
FLUSS_PROPERTIES=
339344
zookeeper.address: zookeeper:2181
340-
bind.listeners: FLUSS://tablet-server-2:9123
345+
bind.listeners: INTERNAL://tablet-server-2:0, CLIENT://tablet-server-2:9123
346+
advertised.listeners: CLIENT://localhost:9126
347+
internal.listener.name: INTERNAL
341348
tablet-server.id: 2
342349
kv.snapshot.interval: 0s
343350
data.dir: /tmp/fluss/data/tablet-server-2
344351
remote.data.dir: /tmp/fluss/remote-data
352+
ports:
353+
- "9126:9123"
345354
volumes:
346355
- shared-tmpfs:/tmp/fluss
347356
zookeeper:
@@ -368,124 +377,7 @@ docker container ls -a
368377
```
369378

370379
### Interacting with Fluss
371-
372-
If you want to interact with this Fluss cluster, you can change the `docker-compose.yml` file to add a Flink cluster.
373-
The changed `docker-compose.yml` file is as follows:
374-
375-
```yaml
376-
services:
377-
coordinator-server:
378-
image: fluss/fluss:$FLUSS_VERSION$
379-
command: coordinatorServer
380-
depends_on:
381-
- zookeeper
382-
environment:
383-
- |
384-
FLUSS_PROPERTIES=
385-
zookeeper.address: zookeeper:2181
386-
bind.listeners: FLUSS://coordinator-server:9123
387-
remote.data.dir: /tmp/fluss/remote-data
388-
tablet-server-0:
389-
image: fluss/fluss:$FLUSS_VERSION$
390-
command: tabletServer
391-
depends_on:
392-
- coordinator-server
393-
environment:
394-
- |
395-
FLUSS_PROPERTIES=
396-
zookeeper.address: zookeeper:2181
397-
bind.listeners: FLUSS://tablet-server-0:9123
398-
tablet-server.id: 0
399-
kv.snapshot.interval: 0s
400-
data.dir: /tmp/fluss/data/tablet-server-0
401-
remote.data.dir: /tmp/fluss/remote-data
402-
volumes:
403-
- shared-tmpfs:/tmp/fluss
404-
tablet-server-1:
405-
image: fluss/fluss:$FLUSS_VERSION$
406-
command: tabletServer
407-
depends_on:
408-
- coordinator-server
409-
environment:
410-
- |
411-
FLUSS_PROPERTIES=
412-
zookeeper.address: zookeeper:2181
413-
bind.listeners: FLUSS://tablet-server-1:9123
414-
tablet-server.id: 1
415-
kv.snapshot.interval: 0s
416-
data.dir: /tmp/fluss/data/tablet-server-1
417-
remote.data.dir: /tmp/fluss/remote-data
418-
volumes:
419-
- shared-tmpfs:/tmp/fluss
420-
tablet-server-2:
421-
image: fluss/fluss:$FLUSS_VERSION$
422-
command: tabletServer
423-
depends_on:
424-
- coordinator-server
425-
environment:
426-
- |
427-
FLUSS_PROPERTIES=
428-
zookeeper.address: zookeeper:2181
429-
bind.listeners: FLUSS://tablet-server-2:9123
430-
tablet-server.id: 2
431-
kv.snapshot.interval: 0s
432-
data.dir: /tmp/fluss/data/tablet-server-2
433-
remote.data.dir: /tmp/fluss/remote-data
434-
volumes:
435-
- shared-tmpfs:/tmp/fluss
436-
zookeeper:
437-
restart: always
438-
image: zookeeper:3.9.2
439-
jobmanager:
440-
image: fluss/quickstart-flink:1.20-$FLUSS_VERSION_SHORT$
441-
ports:
442-
- "8083:8081"
443-
command: jobmanager
444-
environment:
445-
- |
446-
FLINK_PROPERTIES=
447-
jobmanager.rpc.address: jobmanager
448-
volumes:
449-
- shared-tmpfs:/tmp/fluss
450-
taskmanager:
451-
image: fluss/quickstart-flink:1.20-$FLUSS_VERSION_SHORT$
452-
depends_on:
453-
- jobmanager
454-
command: taskmanager
455-
environment:
456-
- |
457-
FLINK_PROPERTIES=
458-
jobmanager.rpc.address: jobmanager
459-
volumes:
460-
- shared-tmpfs:/tmp/fluss
461-
462-
volumes:
463-
shared-tmpfs:
464-
driver: local
465-
driver_opts:
466-
type: "tmpfs"
467-
device: "tmpfs"
468-
```
469-
Save the `docker-compose.yaml` script and execute the `docker-compose up -d` command in the same directory to create the cluster.
470-
471-
#### Enter into SQL-Client
472-
First, use the following command to enter the Flink SQL CLI Container:
473-
```shell
474-
docker compose exec jobmanager ./sql-client
475-
```
476-
477-
#### Create Fluss Catalog
478-
Use the following SQL to create a Fluss catalog:
479-
```sql title="Flink SQL"
480-
CREATE CATALOG fluss_catalog WITH (
481-
'type' = 'fluss',
482-
'bootstrap.servers' = 'coordinator-server:9123'
483-
);
484-
```
485-
486-
```sql title="Flink SQL"
487-
USE CATALOG fluss_catalog;
488-
```
380+
Same as [Interacting with Fluss](deploying-with-docker.md#interacting-with-fluss)
489381

490382
#### Do more with Fluss
491383

0 commit comments

Comments
 (0)