Skip to content

Commit 52b70fb

Browse files
PXB-3862 : TEMPORARY: bundle the validation benchmark kit
https://perconadev.atlassian.net/browse/PXB-3862 Ships pxb-bench (the team's page-tracking benchmark tool, with numeric gap modes enabled) and a from-bare-machine runbook inside the source tree, so validating the measured bridge budget on a new machine needs exactly one git clone. Runs the same 5..100% sweep as the original PXB-3862 report plus a pin-10 column, and collects the calibration and refusal log lines that explain every grouping decision. NOT FOR MERGE: this commit is a distribution vehicle for the multi- machine validation campaign and will be dropped from the branch before the pull request.
1 parent d74d6c6 commit 52b70fb

2 files changed

Lines changed: 150 additions & 0 deletions

File tree

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# PXB-3862 bridge-budget validation — from a bare machine
2+
3+
Everything needed is in this checkout: the xtrabackup source (this
4+
repo, branch PXB-3862-8.4) and the benchmark tool (pxb-test.tar.gz in
5+
this directory — jinyou's pxb-bench with numeric gap modes enabled;
6+
17 unit tests included). This file is the complete runbook.
7+
8+
Path variables used below — set them to taste:
9+
10+
SRC=~/WORK/pxb-3862-8.4 # this repo checkout
11+
SB=~/sandboxes/msb_ps8_4_x # the sandbox (created in step 3)
12+
13+
## 0. Packages (once, ~5 min)
14+
15+
```bash
16+
# Debian/Ubuntu (adjust for RHEL):
17+
sudo apt install -y build-essential cmake bison libssl-dev \
18+
libncurses-dev libaio-dev libcurl4-openssl-dev libudev-dev \
19+
libgcrypt-dev libev-dev libproc2-dev \
20+
python3 python3-pip sysbench wget
21+
```
22+
23+
dbdeployer (single static Go binary; assets are named
24+
dbdeployer-<ver>.linux.tar.gz):
25+
26+
```bash
27+
VER=1.76.0 # or check https://github.com/datacharmer/dbdeployer/releases
28+
mkdir -p ~/bin
29+
wget -q https://github.com/datacharmer/dbdeployer/releases/download/v$VER/dbdeployer-$VER.linux.tar.gz
30+
tar xzf dbdeployer-$VER.linux.tar.gz
31+
mv dbdeployer-$VER.linux ~/bin/dbdeployer && chmod +x ~/bin/dbdeployer
32+
export PATH=$PATH:~/bin # add to ~/.bashrc for permanence
33+
34+
# one-time init: creates ~/opt/mysql (binaries), ~/sandboxes, and
35+
# fetches the tarball download index
36+
dbdeployer init
37+
```
38+
39+
## 1. Source (one clone brings everything)
40+
41+
```bash
42+
git clone -b PXB-3862-8.4 https://github.com/satya-bodapati/percona-xtrabackup.git $SRC
43+
```
44+
45+
## 2. Build xtrabackup (~15 min)
46+
47+
```bash
48+
cd $SRC
49+
cmake -S . -B build -DBUILD_CONFIG=xtrabackup_release -DWITH_MAN_PAGES=OFF
50+
cmake --build build --target xtrabackup -j"$(nproc)"
51+
# sanity - MUST print the option:
52+
build/runtime_output_directory/xtrabackup --help | grep page-tracking-max-gap
53+
```
54+
55+
## 3. Server sandbox (~5 min)
56+
57+
```bash
58+
# download a Percona Server 8.4 tarball via dbdeployer's index
59+
# (or wget one from percona.com/downloads and skip this line):
60+
dbdeployer downloads get-by-version 8.4 --flavor percona --newest
61+
62+
# make it deployable, then deploy one sandbox:
63+
dbdeployer unpack Percona-Server-8.4*.tar.gz --prefix ps
64+
dbdeployer deploy single ps8.4.<x> # <x> from: dbdeployer versions
65+
SB=~/sandboxes/msb_ps8_4_<x> # dbdeployer prints this path;
66+
# port: grep ^port $SB/my.sandbox.cnf
67+
# recommended for large tables:
68+
printf 'innodb_buffer_pool_size=4G\ninnodb_redo_log_capacity=4G\n' >> $SB/my.sandbox.cnf
69+
$SB/restart
70+
```
71+
72+
## 4. Load the table (100M rows ~ 24GB, ~1h; scale down freely)
73+
74+
```bash
75+
PORT=$(awk -F= '/^port/{gsub(/ /,"");print $2;exit}' $SB/my.sandbox.cnf)
76+
$SB/use -e "CREATE DATABASE IF NOT EXISTS test"
77+
sysbench oltp_insert --tables=1 --table-size=100000000 \
78+
--create_secondary=off --mysql-db=test --db-driver=mysql \
79+
--mysql-host=127.0.0.1 --mysql-port=$PORT \
80+
--mysql-user=msandbox --mysql-password=msandbox prepare
81+
```
82+
83+
## 5. Install the benchmark tool (~2 min)
84+
85+
```bash
86+
tar xzf $SRC/storage/innobase/xtrabackup/test/benchmarks/pxb-test.tar.gz -C $SB
87+
cd $SB/pxb-test && python3 -m pip install --user -e .
88+
```
89+
90+
## 6. config.toml — edit ONLY what has no CLI flag (~1 min)
91+
92+
In `$SB/pxb-test/config.toml` (marked CHANGE-ME):
93+
94+
```toml
95+
[sandbox] port = <the $PORT from step 4>
96+
[xtrabackup] full_binary = "<absolute $SRC>/build/runtime_output_directory/xtrabackup"
97+
incremental_binary = "<absolute $SRC>/build/runtime_output_directory/xtrabackup"
98+
[workload] total_rows = <rows loaded in step 4>
99+
```
100+
101+
Sweep parameters (percents/gaps/workloads) are NEVER edited here — they
102+
are command-line flags.
103+
104+
## 7. Run the sweep (overnight at 100M rows)
105+
106+
```bash
107+
pxb-bench --workdir $SB run \
108+
--percents 5:105:5 --gaps none,0,auto,10 --workloads scattered
109+
```
110+
111+
Same densities as the original PXB-3862 chart, so cells overlay it
112+
directly; the `10` column measures what bridging buys wherever auto
113+
declines.
114+
115+
SMALL-TABLE caveat: if the table fits in the buffer pool, dirty pages
116+
never flush by eviction and the tracked set stays empty. Then drive the
117+
individual steps with a restart per density instead of `run`:
118+
119+
```bash
120+
pxb-bench --workdir $SB full-backup
121+
for pct in 5 10 15 20; do
122+
pxb-bench --workdir $SB change-data $pct --workload scattered
123+
pxb-bench --workdir $SB restart
124+
for gap in none 0 auto 10; do
125+
pxb-bench --workdir $SB inc-backup $pct $gap --workload scattered
126+
done
127+
done
128+
```
129+
130+
## 8. Send back one tarball
131+
132+
```bash
133+
cd $SB
134+
grep -h "calibrated storage\|typical gap\|request reduction" \
135+
pxb-test/logs/scattered_*_gap_auto.log > pxb-test/results/decision_lines.txt
136+
tar czf /tmp/pxb3862_results.tgz pxb-test/results/ pxb-test/logs/
137+
```
138+
139+
## 9. Predictions (split-disk, ~165us / ~1.5GB/s class storage)
140+
141+
| density | old build (÷2, budget ~125KB) | this build (÷1.5, ~160KB) |
142+
|---|---|---|
143+
| 5% | refused (correct, beats full scan) | identical |
144+
| **10%** | **refused -> 28.4s** | **bridged -> ~17s = pin-10** |
145+
| 15-100% | bridged (wins/ties) | identical |
146+
147+
On fast shared-disk NVMe the 10% verdict is expected to be the
148+
OPPOSITE (refusal, with pin-10 measurably slower) — each machine
149+
correctly for its own storage; the calibration and refusal log lines
150+
state each decision's numbers.
50.1 KB
Binary file not shown.

0 commit comments

Comments
 (0)