Skip to content

Commit ca423cf

Browse files
EricZequanEricZequan
authored
ci: Add cluster test for vector search (#61009)
close #61017
1 parent 1b1c0c0 commit ca423cf

17 files changed

+2259
-0
lines changed

‎tests/clusterintegrationtest/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
/tidb-server
2+
/gobin/
3+
/.venv/

‎tests/clusterintegrationtest/README.md

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Cluster Integration Test
2+
3+
Before running the tests, please install tiup and build tidb binary:
4+
```shell
5+
curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
6+
source .bash_profile
7+
tiup --version
8+
9+
# cd tidb
10+
make
11+
```
12+
13+
## Guide: Run tests
14+
15+
```shell
16+
# cd clusterintegrationtest
17+
./run_mysql_tester.sh # mysql-tester test
18+
19+
# vector python testers
20+
python3 -m pip install uv
21+
uv init --python python3.9
22+
uv venv
23+
uv pip install -r requirements.txt
24+
# prepare datasets
25+
cd datasets
26+
wget https://ann-benchmarks.com/fashion-mnist-784-euclidean.hdf5
27+
wget https://ann-benchmarks.com/mnist-784-euclidean.hdf5
28+
cd ..
29+
./run_python_testers.sh
30+
31+
./run_upgrade_test.sh # upgrade cluster test
32+
```
33+
34+
## Guide: Update `r`
35+
36+
After changing `t` or changing optimizer plans, `r` need to be updated.
37+
38+
1. Start an empty cluster and expose TiDB as :4000
39+
40+
```shell
41+
# cd clusterintegrationtest
42+
./cluster.sh
43+
```
44+
45+
Note: You may need to wait about 30s for TiFlash to be ready.
46+
47+
2. Run following commands
48+
49+
```shell
50+
# cd clusterintegrationtest
51+
GOBIN=$(realpath .)/gobin go install github.com/pingcap/mysql-tester/src@314107b26aa8fce86beb0dd48e75827fb269b365
52+
./gobin/src -retry-connection-count 5 -record
53+
```
54+
55+
## Guide: Develop python_testers
56+
57+
1. Prepare local environment
58+
59+
```shell
60+
# cd clusterintegrationtest
61+
python3 -m pip install uv
62+
uv init --python python3.9
63+
uv venv
64+
uv pip install -r requirements.txt
65+
```
66+
67+
2. Download datasets
68+
69+
```shell
70+
# cd clusterintegrationtest
71+
cd datasets
72+
wget https://ann-benchmarks.com/fashion-mnist-784-euclidean.hdf5
73+
wget https://ann-benchmarks.com/mnist-784-euclidean.hdf5
74+
cd ..
75+
```
76+
77+
3. Start a CSE cluster and expose TiDB as :4000
78+
79+
```shell
80+
# cd clusterintegrationtest
81+
./cluster.sh
82+
```
83+
84+
Note: You may need to wait about 30s for TiFlash to be ready.
85+
86+
4. Run, edit and debug tests
87+
88+
```shell
89+
# cd clusterintegrationtest
90+
uv run python_testers/vector_recall.py
91+
```
92+
93+
## Note:
94+
If your contribution involves tidb and tiflash and will affect the test cases of this test, please submit your contribution to tiflash first and wait 2 hours after the merge before executing this test.
95+
96+
97+
1. In tidb, we will download the binary of the tiflash master branch as a component for cluster testing. Please make sure that your tiflash code is submitted to the master branch.
98+
2. In tiflash, we will also download the binary of the tidb master branch as a component for cluster testing. Please make sure that your tidb code is submitted to the master branch.
99+
100+
If you have other questions about this test, please contact @EricZequan.

‎tests/clusterintegrationtest/_include.sh

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
#!/bin/bash
2+
#
3+
# Copyright 2025 PingCAP, Inc.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
set -euo pipefail
18+
19+
function start_tidb() {
20+
export VERSION_SOURCE="nightly"
21+
22+
if [ ! -f "../../bin/tidb-server" ]; then
23+
cd ../../ || exit 1
24+
echo "building tidb-server..."
25+
make
26+
echo "build successfully"
27+
cd - || exit 1
28+
fi
29+
30+
echo "Starting TiUP Playground in the background..."
31+
if [ -f "../../bin/tikv-server" ] && [ -f "../../bin/pd-server" ] && [ -f "../../bin/tiflash" ]; then
32+
tiup playground nightly --mode=tidb \
33+
--db.binpath=../../bin/tidb-server \
34+
--db.config=./config.toml \
35+
--kv.binpath=../../bin/tikv-server \
36+
--pd.binpath=../../bin/pd-server \
37+
--tiflash.binpath=../../bin/tiflash &
38+
else
39+
tiup playground nightly --db=1 --kv=1 --tiflash=1 --db.binpath=../../bin/tidb-server --db.config=./config.toml &
40+
fi
41+
}
42+
43+
function check_and_prepare_datasets() {
44+
if [ -f "./fashion-mnist-784-euclidean.hdf5" ] && [ -f "./mnist-784-euclidean.hdf5" ]; then
45+
echo "Datasets already exist, skip"
46+
return
47+
fi
48+
49+
if [ -d "${ASSETS_DIR}" ]; then
50+
if [ -f "${ASSETS_DIR}/fashion-mnist-784-euclidean.hdf5" ]; then
51+
echo "Moving fashion-mnist dataset from ASSETS_DIR..."
52+
mv "${ASSETS_DIR}/fashion-mnist-784-euclidean.hdf5" .
53+
else
54+
echo "Downloading fashion-mnist dataset..."
55+
wget https://ann-benchmarks.com/fashion-mnist-784-euclidean.hdf5
56+
fi
57+
58+
if [ -f "${ASSETS_DIR}/mnist-784-euclidean.hdf5" ]; then
59+
echo "Moving mnist dataset from ASSETS_DIR..."
60+
mv "${ASSETS_DIR}/mnist-784-euclidean.hdf5" .
61+
else
62+
echo "Downloading mnist dataset..."
63+
wget https://ann-benchmarks.com/mnist-784-euclidean.hdf5
64+
fi
65+
else
66+
echo "Downloading fashion-mnist dataset..."
67+
wget https://ann-benchmarks.com/fashion-mnist-784-euclidean.hdf5
68+
69+
echo "Downloading mnist dataset..."
70+
wget https://ann-benchmarks.com/mnist-784-euclidean.hdf5
71+
72+
fi
73+
}
74+
75+
function start_tidb_fixed_version() {
76+
export VERSION_SOURCE="v8.5.1"
77+
78+
echo "Starting TiUP Playground in the background..."
79+
tiup playground v8.5.1 --db=1 --kv=1 --tiflash=1 --db.config=./config.toml &
80+
}
81+
82+
function build_mysql_tester() {
83+
echo "+ Installing mysql-tester"
84+
GOBIN=$PWD go install github.com/pingcap/mysql-tester/src@0d83955ea569706e5296cd3e2f54efb7f1206d0b
85+
mv src mysql-tester
86+
}
87+
88+
function wait_for_tidb() {
89+
echo
90+
echo "+ Waiting TiDB start up"
91+
92+
for i in {1..30}; do
93+
if mysql -e 'show databases' -u root -h 127.0.0.1 --port 4000; then
94+
echo " - TiDB startup successfully"
95+
return
96+
fi
97+
sleep 3
98+
done
99+
echo "* Fail to start TiDB cluster in 900s"
100+
exit 1
101+
}
102+
103+
function wait_for_tiflash() {
104+
echo
105+
echo "+ Waiting TiFlash start up (30s)"
106+
sleep 30
107+
}
108+
109+
function stop_tiup() {
110+
echo "+ Stopping TiUP"
111+
TIUP_PID=$(pgrep -f "tiup-playground")
112+
if [ -n "$TIUP_PID" ]; then
113+
echo " - Sending SIGTERM to PID=$TIUP_PID"
114+
kill $TIUP_PID
115+
fi
116+
117+
for i in {1..60}; do
118+
if ! pgrep -f "tiup-playground" > /dev/null; then
119+
echo " - TiUP stopped successfully"
120+
return
121+
fi
122+
sleep 1
123+
done
124+
125+
echo "* Fail to stop TiUP in 60s"
126+
exit 1
127+
}
128+
129+
function print_versions() {
130+
# Print versions
131+
if [ "$VERSION_SOURCE" = "nightly" ]; then
132+
echo "+ TiDB Version"
133+
../../bin/tidb-server -V
134+
echo
135+
if [ -f "../../bin/tikv-server" ] && [ -f "../../bin/pd-server" ] && [ -f "../../bin/tiflash" ]; then
136+
echo "+ TiKV Version"
137+
../../bin/tikv-server --version
138+
echo
139+
echo "+ TiFlash Version"
140+
../../bin/tiflash --version
141+
echo
142+
else
143+
echo "+ TiKV Version"
144+
tiup tikv:nightly --version
145+
echo
146+
echo "+ TiFlash Version"
147+
tiup tiflash:nightly --version
148+
echo
149+
fi
150+
else
151+
echo "+ TiDB Version"
152+
tiup tidb:v8.5.1 -V
153+
echo
154+
echo "+ TiKV Version"
155+
tiup tikv:v8.5.1 --version
156+
echo
157+
echo "+ TiFlash Version"
158+
tiup tiflash:v8.5.1 --version
159+
echo
160+
fi
161+
162+
echo "+ TiUP Version"
163+
~/.tiup/bin/tiup playground -v
164+
165+
}

‎tests/clusterintegrationtest/cluster.sh

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/bin/bash
2+
#
3+
# Copyright 2025 PingCAP, Inc.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
cd ../../ || exit 1
18+
echo "building tidb-server..."
19+
make
20+
echo "build successfully"
21+
22+
cd - || exit 1
23+
24+
echo "Starting TiUP Playground in the background..."
25+
tiup playground nightly --db=1 --kv=1 --tiflash=1 --db.binpath=../../bin/tidb-server --db.config=./config.toml

‎tests/clusterintegrationtest/config.toml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Copyright 2025 PingCAP, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
lease = "0"
16+
host = "127.0.0.1"
17+
new_collations_enabled_on_first_bootstrap = true
18+
enable-table-lock = true
19+
20+
[status]
21+
status-host = "127.0.0.1"
22+
23+
[performance]
24+
stats-lease = "0"
25+
26+
[tikv-client.async-commit]
27+
safe-window = 0
28+
allowed-clock-drift = 0
29+
30+
[experimental]
31+
enable-new-charset = true
32+
allow-expression-index = true

‎tests/clusterintegrationtest/datasets/.gitkeep

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright 2025 PingCAP, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

0 commit comments

Comments
 (0)