Skip to content

Commit 94a07be

Browse files
committed
Add the njet load balancing module
1 parent 508f99b commit 94a07be

File tree

8 files changed

+2018
-10
lines changed

8 files changed

+2018
-10
lines changed

docs/admin-manual/cluster-management/load-balancing.md

Lines changed: 224 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,229 @@ mysql> show databases;
121121
2 rows in set (0.00 sec)
122122
```
123123

124-
### 03 HAProxy
124+
### 03 NJet
125+
126+
You can use [NJet](https://docs.njet.org.cn/) to keep an Nginx-like experience while gaining enhanced features such as dynamic management and health checks. The following example also uses `192.168.1.100` as the proxy node and `192.168.1.101/102/103`.
127+
128+
#### Install NJet
129+
130+
Please refer to the official NJet installation guide: https://docs.njet.org.cn/docs/v4.0.0/guide/install/index.html. Below are example installation steps for Docker and Ubuntu.
131+
132+
1. Docker
133+
134+
```shell
135+
## Pull NJET image
136+
docker pull tmlake/njet:latest
137+
138+
## Run NJET
139+
docker run -d --rm --privileged tmlake/njet:latest
140+
```
141+
142+
2. Ubuntu
143+
144+
```shell
145+
## Prepare repo
146+
sudo apt-get update
147+
sudo apt-get install ca-certificates curl gnupg
148+
sudo install -m 0755 -d /etc/apt/keyrings
149+
curl -fsSL https://njet.org.cn/download/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/njet.gpg
150+
sudo chmod a+r /etc/apt/keyrings/njet.gpg
151+
152+
## Add APT source
153+
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/njet.gpg] https://njet.org.cn/download/linux/ubuntu $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" | sudo tee /etc/apt/sources.list.d/njet.list > /dev/null
154+
sudo apt-get update
155+
156+
## Install and start
157+
sudo apt-get install njet
158+
sudo systemctl start njet
159+
```
160+
161+
#### Initial NJet configuration
162+
163+
`njet.conf` example:
164+
165+
```text
166+
events {
167+
worker_connections 1024;
168+
}
169+
stream {
170+
upstream mysqld {
171+
hash $remote_addr consistent;
172+
server 192.168.1.101:9030 weight=1 max_fails=2 fail_timeout=60s;
173+
server 192.168.1.102:9030 weight=1 max_fails=2 fail_timeout=60s;
174+
server 192.168.1.103:9030 weight=1 max_fails=2 fail_timeout=60s;
175+
}
176+
server {
177+
# Proxy port
178+
listen 6030;
179+
proxy_connect_timeout 300s;
180+
proxy_timeout 300s;
181+
proxy_pass mysqld;
182+
}
183+
}
184+
```
185+
186+
#### Dynamic upstream member management
187+
188+
OpenNJet provides HTTP APIs to dynamically add/remove upstream members. To enable this you need to enable related modules on both data plane and control plane.
189+
190+
Data plane `njet.conf` example:
191+
192+
```text
193+
helper broker modules/njt_helper_broker_module.so conf/mqtt.conf;
194+
helper ctrl modules/njt_helper_ctrl_module.so conf/ctrl.conf;
195+
load_module modules/njt_http_upstream_member_module.so;
196+
197+
stream {
198+
upstream mysqld {
199+
zone mysqld 16k; # shared memory required
200+
hash $remote_addr consistent;
201+
server 192.168.1.101:9030 weight=1 max_fails=2 fail_timeout=60s;
202+
server 192.168.1.102:9030 weight=1 max_fails=2 fail_timeout=60s;
203+
server 192.168.1.103:9030 weight=1 max_fails=2 fail_timeout=60s;
204+
}
205+
server {
206+
listen 6030;
207+
proxy_connect_timeout 300s;
208+
proxy_timeout 300s;
209+
proxy_pass mysqld;
210+
}
211+
}
212+
```
213+
214+
Control plane `njet_ctrl.conf` example:
215+
216+
```text
217+
load_module modules/njt_http_sendmsg_module.so;
218+
load_module modules/njt_http_upstream_api_module.so;
219+
load_module modules/njt_helper_health_check_module.so;
220+
221+
http {
222+
server {
223+
listen 8081;
224+
location /api {
225+
dyn_module_api;
226+
}
227+
}
228+
}
229+
```
230+
231+
##### Add an upstream member
232+
233+
When you add a new FE node (for example `192.168.1.104:9030`), you can add it via API. OpenNJet will assign an ID to the newly added server which can be used for later queries, removal, or update.
234+
235+
POST URL: `POST http://{ip}:8081/api/v1/upstream_api/stream/upstreams/{upstream_name}/servers/`
236+
237+
```bash
238+
curl -X POST http://127.0.0.1:8081/api/v1/upstream_api/stream/upstreams/mysqld/servers/ -d '{
239+
"server": "192.168.1.104:9030",
240+
"weight": 2,
241+
"max_conns": 2,
242+
"max_fails": 1,
243+
"fail_timeout": "5s",
244+
"slow_start": "5s",
245+
"route": "",
246+
"backup": false,
247+
"down": false
248+
}'
249+
```
250+
251+
##### Remove an upstream member
252+
253+
DELETE URL: `DELETE http://{ip}:8081/api/v1/upstream_api/stream/upstreams/{upstream_name}/servers/{id}`
254+
255+
```bash
256+
curl -X DELETE http://127.0.0.1:8081/api/v1/upstream_api/stream/upstreams/mysqld/servers/3
257+
```
258+
259+
#### Configure active health checks
260+
261+
Active health checks are enabled by default; you can configure them via `http://{ip}:8081/api/v1/hc/smysql/{upstream}`:
262+
263+
```bash
264+
curl -X 'POST' \
265+
'http://127.0.0.1:8081/api/v1/hc/smysql/mysqld' \
266+
-H 'accept: application/json' \
267+
-H 'Content-Type: application/json' \
268+
-d '{
269+
"interval": "5s",
270+
"jitter": "1s",
271+
"timeout": "5s",
272+
"passes": 1,
273+
"fails": 1,
274+
"sql": {
275+
"select": "select 1",
276+
"useSsl": true,
277+
"user": "root",
278+
"password": "123456",
279+
"db": "db"
280+
}
281+
}'
282+
```
283+
284+
Field notes:
285+
286+
- `interval`: required, health check frequency.
287+
- `visit_interval`: optional, if the server has been accessed by clients within this interval, skip the health check; `interval` should be greater than `visit_interval`.
288+
- `jitter`: required, maximum timer jitter for the health check to avoid synchronized checks.
289+
- `timeout`: required, timeout for the health check.
290+
- `passes`: required, number of consecutive successful checks to mark server healthy.
291+
- `fails`: required, number of consecutive failures to mark server unhealthy.
292+
- `port`: optional, override the port used for health checks; if omitted, the upstream server port is used.
293+
- `sql`: contains `select/useSsl/user/password/db` fields; `db` is required.
294+
295+
#### JSON configuration support
296+
297+
NJet supports JSON configuration which is helpful for automation. Convert `njet.conf` to `njet.json` and load it with `njet -c conf/njet.json`.
298+
299+
```json
300+
[
301+
{
302+
"cmd": "events",
303+
"args": [],
304+
"block": [
305+
{
306+
"cmd": "worker_connections",
307+
"args": ["1024"]
308+
}
309+
]
310+
},
311+
{
312+
"cmd": "stream",
313+
"args": [],
314+
"block": [
315+
{
316+
"cmd": "upstream",
317+
"args": ["mysqld"],
318+
"block": [
319+
{"cmd": "hash", "args": ["$remote_addr", "consistent"]},
320+
{"cmd": "server", "args": ["192.168.1.101:9030", "weight=1", "max_fails=2", "fail_timeout=60s"]},
321+
{"cmd": "server", "args": ["192.168.1.102:9030", "weight=1", "max_fails=2", "fail_timeout=60s"]},
322+
{"cmd": "server", "args": ["192.168.1.103:9030", "weight=1", "max_fails=2", "fail_timeout=60s"]}
323+
]
324+
},
325+
{
326+
"cmd": "server",
327+
"args": [],
328+
"block": [
329+
{"cmd": "listen", "args": ["6030"]},
330+
{"cmd": "proxy_connect_timeout", "args": ["300s"]},
331+
{"cmd": "proxy_timeout", "args": ["300s"]},
332+
{"cmd": "proxy_pass", "args": ["mysqld"]}
333+
]
334+
}
335+
]
336+
}
337+
]
338+
```
339+
340+
More references:
341+
342+
- OpenNJet installation: https://docs.njet.org.cn/docs/v4.0.0/guide/install/index.html
343+
- Health check details: https://docs.njet.org.cn/docs/v4.0.0/reference/upstream/health_check/mysql_health_check/index.html
344+
- Upstream API: https://docs.njet.org.cn/docs/v4.0.0/reference/upstream/upstream_api/index.html
345+
346+
### 04 HAProxy
125347

126348
[HAProxy](https://www.haproxy.org/) is a high-performance TCP/HTTP load balancer written in C language.
127349

@@ -244,7 +466,7 @@ mysql> show databases;
244466

245467
`mysql -h 192.168.1.100 -uroot -P6030 -p`
246468

247-
### 04 ProxySQL
469+
### 05 ProxySQL
248470

249471
[ProxySQL](https://proxysql.com/) is an open-source MySQL database proxy software written in C language. It can implement connection management, read-write splitting, load balancing, failover, and other functions. It has advantages such as high performance, configurability, and dynamic management, and is commonly used in Web services, big data platforms, cloud databases, and other scenarios.
250472

0 commit comments

Comments
 (0)