Skip to content

Commit 7aca11c

Browse files
maksymsurmaksymsur
committed
feat(kms-connector): db insertion tool
1 parent 9f9e0b1 commit 7aca11c

File tree

20 files changed

+1851
-92
lines changed

20 files changed

+1851
-92
lines changed

‎test-suite/README.md‎

Lines changed: 115 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -30,29 +30,131 @@ KMS can be configured to two modes:
3030
The test suite offers a unified CLI for all operations:
3131

3232
```sh
33-
3433
cd test-suite/fhevm
34+
3535
# Deploy the entire stack
3636
./fhevm-cli deploy
37+
# WIP: Build images locally (when private registry not available)
38+
./fhevm-cli deploy --build
3739

38-
# Run specific tests
39-
./fhevm-cli test input-proof
40-
# Trivial
41-
./fhevm-cli test user-decryption
42-
# Trivial
43-
./fhevm-cli test public-decryption
44-
./fhevm-cli test erc20
45-
46-
# Upgrade a specific service
40+
# Run blockchain integration tests
41+
./fhevm-cli test input-proof # Test input proofs
42+
./fhevm-cli test user-decryption # Test user decryptions
43+
./fhevm-cli test public-decryption # Test public decryptions
44+
./fhevm-cli test erc20 # Test ERC20 operations
45+
./fhevm-cli test debug # Debug mode testing
46+
47+
# Database connector testing (runs gateway-stress via fhevm-cli wrapper)
48+
./fhevm-cli db-test --track-responses # Test with all DB URLs from config
49+
./fhevm-cli db-test --duration 60s -t mixed # Run for 60s with mixed requests
50+
./fhevm-cli db-test -b 100 --clear-db # Batch size 100, clear DB first
51+
52+
# Upgrade specific services
53+
./fhevm-cli upgrade host
54+
./fhevm-cli upgrade gateway
55+
./fhevm-cli upgrade connector
4756
./fhevm-cli upgrade coprocessor
57+
./fhevm-cli upgrade relayer
58+
./fhevm-cli upgrade test-suite
4859

49-
# View logs
50-
./fhevm-cli logs relayer
60+
# View logs for any service
61+
./fhevm-cli logs [SERVICE]
5162

52-
# Clean up
63+
# Clean up all containers and volumes
5364
./fhevm-cli clean
5465
```
5566

67+
### Database Connector Testing
68+
69+
The `fhevm-cli db-test` command provides database-level stress testing:
70+
71+
```sh
72+
# Implemented options:
73+
-n, --num-connectors NUM # Number of DB URLs to use from config
74+
-t, --type TYPE # Request type: public, user, or mixed
75+
-b, --batch-size SIZE # Requests per batch
76+
--duration TIME # Test duration (e.g., 30s, 5m, 1h)
77+
-i, --interval TIME # Batch interval (e.g., 1s, 500ms, 2s)
78+
-c, --config FILE # Path to custom config file
79+
--track-responses # Enable response tracking
80+
--clear-db # Clear DB tables before test
81+
82+
# Examples:
83+
./fhevm-cli db-test --clear-db --track-responses # Clear DB, then test with tracking
84+
./fhevm-cli db-test -n 2 -t mixed # Test 2 DBs with mixed requests
85+
./fhevm-cli db-test --duration 60s -i 500ms # 60s test, 500ms intervals
86+
./fhevm-cli db-test -c custom.toml -b 100 --clear-db # Custom config, clear DB, batch 100
87+
88+
# Default config: test-suite/gateway-stress/config/config.toml
89+
```
90+
91+
### Advanced Gateway Stress Testing
92+
93+
For more control, use the standalone `gateway-stress` tool directly (in `test-suite/gateway-stress`):
94+
95+
1. **Blockchain-based testing** - Sends actual transactions through the blockchain
96+
2. **Database-level testing** - Directly inserts requests into PostgreSQL for focused DB testing
97+
98+
#### Blockchain-based Testing
99+
Sends decryption requests through the blockchain (requires deployed contracts):
100+
101+
```sh
102+
cd test-suite/gateway-stress
103+
104+
# Build the tool first
105+
cargo build --release
106+
107+
# Send public decryption transactions
108+
./target/release/gateway-stress public
109+
110+
# Send user decryption transactions
111+
./target/release/gateway-stress user
112+
113+
# Note: 'mixed' mode is not yet implemented
114+
```
115+
116+
#### Database-level Testing
117+
Bypasses blockchain and directly inserts into PostgreSQL databases:
118+
119+
```sh
120+
cd test-suite/gateway-stress
121+
122+
# Build the tool
123+
cargo build --release
124+
125+
# Basic database test with default settings from config
126+
./target/release/gateway-stress db-connector
127+
128+
# Override test duration and request type
129+
./target/release/gateway-stress db-connector --duration 60s --request-type public
130+
131+
# Enable response tracking to verify sync across databases
132+
./target/release/gateway-stress db-connector --track-responses --batch-size 1000
133+
134+
# Use custom configuration
135+
./target/release/gateway-stress --config custom-config.toml db-connector
136+
```
137+
138+
#### Prerequisites
139+
- Rust toolchain (for building gateway-stress)
140+
- For blockchain testing: Deployed FHEVM contracts and configured ct_handles
141+
- For database testing: Running PostgreSQL with credentials in `config/config.toml`
142+
143+
#### DB Connector Options
144+
- `--request-type <TYPE>` - Request type: `public`, `user`, or `mixed`
145+
- `--duration <TIME>` - Test duration (e.g., `30s`, `5m`, `1h`)
146+
- `--batch-size <NUM>` - Requests per batch (for load control)
147+
- `--track-responses` - Monitor response processing and sync status
148+
149+
#### Configuration File
150+
The tool reads from a TOML configuration file (default: `config/config.toml`):
151+
- Database connection strings
152+
- Batch intervals and sizes
153+
- Connection pool settings
154+
- Request generation parameters
155+
156+
See `gateway-stress/README.md` for detailed configuration examples.
157+
56158
### WIP - Forcing Local Builds (`--build`)
57159

58160
⚠️ **IMPORTANT: THIS FEATURE IS STILL A WORK IN PROGRESS!** ⚠️

‎test-suite/fhevm/docker-compose/test-suite-docker-compose.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,28 @@ services:
1414
- ../env/staging/.env.test-suite.local
1515
command:
1616
- tail -f /dev/null
17+
18+
# Gateway stress test with db-connector subcommand
19+
gateway-stress-db:
20+
container_name: fhevm-gateway-stress-db
21+
image: ghcr.io/zama-ai/fhevm/gateway-stress:${TEST_SUITE_VERSION}
22+
build:
23+
context: ../../..
24+
dockerfile: test-suite/gateway-stress/Dockerfile
25+
env_file:
26+
- ../env/staging/.env.connector.local # Reuse connector env for DB connection
27+
environment:
28+
RUST_LOG: ${RUST_LOG:-gateway_stress=info}
29+
command:
30+
- db-connector
31+
- --connectors
32+
- "${DB_TEST_CONNECTORS:-1}"
33+
- --track-responses
34+
- --duration
35+
- "${DB_TEST_DURATION:-30s}"
36+
- --batch-size
37+
- "${DB_TEST_BATCH:-10}"
38+
- --request-type
39+
- "${DB_TEST_TYPE:-mixed}"
40+
profiles:
41+
- db-test

‎test-suite/fhevm/fhevm-cli‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ function usage {
5151
echo -e "${BOLD}${LIGHT_BLUE}Commands:${RESET}"
5252
echo -e " ${YELLOW}deploy${RESET} ${CYAN}[--build]${RESET} WIP: Deploy the full fhevm stack (optionally rebuild images)"
5353
echo -e " ${YELLOW}test${RESET} ${CYAN}[TYPE]${RESET} Run tests (input-proof|user-decryption|public-decryption|erc20|debug)"
54+
echo -e " ${YELLOW}db-test${RESET} ${CYAN}[OPTIONS]${RESET} Test N database connectors with direct insertions"
5455
echo -e " ${YELLOW}upgrade${RESET} ${CYAN}[SERVICE]${RESET} Upgrade specific service (host|gateway|connector|coprocessor|relayer|test-suite)"
5556
echo -e " ${YELLOW}clean${RESET} Remove all containers and volumes"
5657
echo -e " ${YELLOW}logs${RESET} ${CYAN}[SERVICE]${RESET} View logs for a specific service"
@@ -69,6 +70,7 @@ function usage {
6970
echo -e " ${PURPLE}./fhevm-cli test user-decryption ${RESET}"
7071
echo -e " ${PURPLE}./fhevm-cli test public-decryption -n staging${RESET}"
7172
echo -e " ${PURPLE}./fhevm-cli test erc20${RESET}"
73+
echo -e " ${PURPLE}./fhevm-cli db-test -n 3 --track-responses${RESET}"
7274
echo -e " ${PURPLE}./fhevm-cli upgrade coprocessor${RESET}"
7375
echo -e "${BLUE}============================================================${RESET}"
7476
}
@@ -209,6 +211,140 @@ case $COMMAND in
209211
fi
210212
;;
211213

214+
db-test)
215+
print_logo
216+
echo -e "${LIGHT_BLUE}${BOLD}[DB-TEST] Running DB Connector Test...${RESET}"
217+
218+
# Check if running with docker-compose or locally
219+
USE_DOCKER=false
220+
USE_LOCAL=false
221+
222+
# Parse first argument to determine mode
223+
if [[ "$1" == "--docker" ]] || [[ "$1" == "-d" ]]; then
224+
USE_DOCKER=true
225+
shift
226+
elif [[ "$1" == "--local" ]] || [[ "$1" == "-l" ]]; then
227+
USE_LOCAL=true
228+
shift
229+
else
230+
# Default to docker if stack is running
231+
if docker ps | grep -q "kms-connector-db"; then
232+
USE_DOCKER=true
233+
echo -e "${YELLOW}[INFO] Detected running docker stack, using docker mode${RESET}"
234+
else
235+
USE_LOCAL=true
236+
echo -e "${YELLOW}[INFO] No docker stack detected, using local mode${RESET}"
237+
fi
238+
fi
239+
240+
if [ "$USE_DOCKER" = true ]; then
241+
# Run using docker-compose
242+
echo -e "${BLUE}[INFO] Running DB test in Docker environment${RESET}"
243+
244+
# Build environment variables for docker-compose
245+
COMPOSE_ENV=""
246+
while (( "$#" )); do
247+
case "$1" in
248+
-n|--connectors)
249+
COMPOSE_ENV="$COMPOSE_ENV DB_TEST_CONNECTORS=$2"
250+
shift 2
251+
;;
252+
--duration)
253+
COMPOSE_ENV="$COMPOSE_ENV DB_TEST_DURATION=$2"
254+
shift 2
255+
;;
256+
-b|--batch-size)
257+
COMPOSE_ENV="$COMPOSE_ENV DB_TEST_BATCH=$2"
258+
shift 2
259+
;;
260+
-t|--type)
261+
COMPOSE_ENV="$COMPOSE_ENV DB_TEST_TYPE=$2"
262+
shift 2
263+
;;
264+
*)
265+
shift
266+
;;
267+
esac
268+
done
269+
270+
# Run the test service using existing test-suite docker-compose
271+
eval "$COMPOSE_ENV docker compose -p fhevm -f ${SCRIPT_DIR}/docker-compose/test-suite-docker-compose.yml --profile db-test run --rm gateway-stress-db"
272+
else
273+
# Run locally with cargo
274+
GATEWAY_STRESS_DIR="${SCRIPT_DIR}/../gateway-stress"
275+
276+
if [ ! -d "$GATEWAY_STRESS_DIR" ]; then
277+
echo -e "${RED}[ERROR]${RESET} ${BOLD}gateway-stress directory not found at: $GATEWAY_STRESS_DIR${RESET}"
278+
exit 1
279+
fi
280+
281+
cd "$GATEWAY_STRESS_DIR"
282+
CMD="cargo run -- db-connector"
283+
284+
# Parse arguments for db-test
285+
while (( "$#" )); do
286+
case "$1" in
287+
-n|--num-connectors)
288+
CMD="$CMD --num-connectors $2"
289+
shift 2
290+
;;
291+
-t|--type)
292+
CMD="$CMD --request-type $2"
293+
shift 2
294+
;;
295+
-b|--batch-size)
296+
CMD="$CMD --batch-size $2"
297+
shift 2
298+
;;
299+
--duration)
300+
CMD="$CMD --duration $2"
301+
shift 2
302+
;;
303+
-i|--interval)
304+
CMD="$CMD --interval $2"
305+
shift 2
306+
;;
307+
-c|--config)
308+
CMD="$CMD --config $2"
309+
shift 2
310+
;;
311+
--track-responses)
312+
CMD="$CMD --track-responses"
313+
shift
314+
;;
315+
--clear-db)
316+
CMD="$CMD --clear-db"
317+
shift
318+
;;
319+
*)
320+
echo -e "${RED}[ERROR]${RESET} ${BOLD}Unknown option: $1${RESET}"
321+
echo ""
322+
echo -e "${BOLD}DB-Test Options:${RESET}"
323+
echo -e " ${CYAN}-n, --num-connectors NUM${RESET} Number of DB URLs to use"
324+
echo -e " ${CYAN}-t, --type TYPE${RESET} Request type: public, user, or mixed"
325+
echo -e " ${CYAN}-b, --batch-size SIZE${RESET} Requests per batch"
326+
echo -e " ${CYAN}--duration TIME${RESET} Test duration (e.g., 30s, 5m)"
327+
echo -e " ${CYAN}-i, --interval TIME${RESET} Batch interval (e.g., 1s, 500ms)"
328+
echo -e " ${CYAN}-c, --config FILE${RESET} Path to config file"
329+
echo -e " ${CYAN}--track-responses${RESET} Enable response tracking"
330+
echo -e " ${CYAN}--clear-db${RESET} Clear DB tables before test"
331+
exit 1
332+
;;
333+
esac
334+
done
335+
336+
echo -e "${YELLOW}Executing: $CMD${RESET}"
337+
eval $CMD
338+
339+
if [ $? -eq 0 ]; then
340+
echo -e "${GREEN}${BOLD}[SUCCESS] DB Connector Test completed successfully!${RESET}"
341+
else
342+
echo -e "${RED}${BOLD}[FAILURE] DB Connector Test failed${RESET}"
343+
exit 1
344+
fi
345+
fi
346+
;;
347+
212348
help|-h|--help)
213349
usage
214350
exit 0

0 commit comments

Comments
 (0)