PXC-5251: SST: opt-in [sst] transfer-fifo-streams=N for parallel TCP … - #2309
Open
satya-bodapati wants to merge 1 commit into
Open
PXC-5251: SST: opt-in [sst] transfer-fifo-streams=N for parallel TCP …#2309satya-bodapati wants to merge 1 commit into
satya-bodapati wants to merge 1 commit into
Conversation
…transport xtrabackup ≥ 8.0.29 exposes --fifo-streams=N which fans out the backup stream to N parallel named pipes (one per data-copy thread). xbstream consumes the matching N pipes back into a single output. PXC SST today wires xtrabackup's stdout into a single TLS-wrapped TCP socket, so even on fast LAN/WAN networks SST throughput is bounded by what one TLS-encrypted TCP stream can carry (~1 Gbps; socat+OpenSSL is single-core-bound for the per-flow crypto), and by xtrabackup's single stdout mutex (all data-copy threads serialize there). Adds an opt-in [sst] transfer-fifo-streams=N option (default 1 = legacy single-stream behaviour) that: - Runs the donor's xtrabackup with --fifo-streams=N --fifo-dir=... so each data-copy thread feeds its own FIFO. - Runs the joiner's xbstream with the same flags, consuming N FIFOs. - Bridges with N independent socat sessions (encrypt=4 TLS or plain TCP) on TSST_PORT..TSST_PORT+N-1. Typical impact: - 1 GbE LAN: no benefit, leave default 1. - 10 GbE LAN with encrypt=4: N=8 → ~8 Gbps. - 25 GbE+ LAN, NVMe disks: N=16. - Cross-region WAN (high RTT): N=8 for TCP window aggregation. The legacy single-TCP code path is left completely untouched; takes effect only when N > 1. Includes a new mtr test variant galera_sst_xtrabackup-v2_fifo16 with N=16 covering both enc_off and enc_on combinations. Restrictions when N>1 (documented inline near the validation block in read_cnf): transferfmt must be socat, streamfmt must be xbstream, and sst.compressor/decompressor is not supported (users should use xtrabackup's own --compress=zstd via [sst] inno-backup-opts instead, which compresses inside xtrabackup before bytes hit the FIFOs). https://perconadev.atlassian.net/browse/PXC-5251
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…transport
xtrabackup ≥ 8.0.29 exposes --fifo-streams=N which fans out the backup stream to N parallel named pipes (one per data-copy thread). xbstream consumes the matching N pipes back into a single output. PXC SST today wires xtrabackup's stdout into a single TLS-wrapped TCP socket, so even on fast LAN/WAN networks SST throughput is bounded by what one TLS-encrypted TCP stream can carry (~1 Gbps; socat+OpenSSL is single-core-bound for the per-flow crypto), and by xtrabackup's single stdout mutex (all data-copy threads serialize there).
Adds an opt-in [sst] transfer-fifo-streams=N option (default 1 = legacy single-stream behaviour) that:
Typical impact:
The legacy single-TCP code path is left completely untouched; takes effect only when N > 1. Includes a new mtr test variant galera_sst_xtrabackup-v2_fifo16 with N=16 covering both enc_off and enc_on combinations. Restrictions when N>1 (documented inline near the validation block in read_cnf): transferfmt must be socat, streamfmt must be xbstream, and sst.compressor/decompressor is not supported (users should use xtrabackup's own --compress=zstd via [sst] inno-backup-opts instead, which compresses inside xtrabackup before bytes hit the FIFOs).
https://perconadev.atlassian.net/browse/PXC-5251