/dev/stdin will not always pipe / work in all environments

[Adel-Magebinary]

Incident Report: Galera SST Failures in Some Docker Environment

1. Issue Description

Galera State Snapshot Transfer (SST) scripts (specifically wsrep_sst_common and its derivatives) were hanging or failing during the joiner-side post-processing phase. The failures occurred primarily during temporary mysqld startup and internal credential validation via mysqladmin.

2. Root Cause Analysis

The issue was traced to a fundamental filesystem namespace limitation in the Triton (SmartOS/LX-brand) container environment regarding file descriptor mapping.

Technical Breakdown:

Broken Symlinks: The standard /dev/stdin path in Triton is often a relative symlink pointing to ../proc/self/fd/0. This resolution fails inside many containerized shells, returning No such file or directory.

Restricted Kernel Paths: Direct access to the kernel path /proc/self/fd/0 (standard Linux "magic" for stdin) is restricted or namespaced in the Triton LX-brand zones, preventing binaries from opening it as a regular file.

Blocking Named Pipes: Attempting to simulate stdin via mkfifo (Named Pipes) resulted in deadlocks, as FIFOs are blocking by nature and require perfectly synchronized reader/writer processes, which the asynchronous nature of SST scripts does not provide.

3. Resolution

The resolution required a "Buffer-to-Disk" architectural change to the shell scripts, moving away from streaming data via pipes to persistent temporary files.

Key Changes:

Elimination of /dev/stdin: All references to the magic path /dev/stdin were replaced with unique temporary files generated by mktemp.

Removal of Process Substitution: Logic using <(command) (which relies on /dev/fd/ mapping) was refactored to write command output to a temp file, read the file, and then delete it.

Two-Step Initialization: For the critical mysqld --init-file startup:

Original: cat <<EOF | mysqld --init-file=/dev/stdin (Failed due to broken path).

Fixed: The SQL commands are written to /tmp/sst_init.XXXXXX first, and mysqld is pointed to the absolute path of this regular file.

Credential Buffering: Authentication details are now dumped into a disk-backed .cnf file in /tmp for the duration of the SST process, ensuring mysqladmin and mysql clients have a reliable path to access credentials.

4. Verification Results

Test 1 (Standard Redirection): FAILED (Confirmed /dev/stdin unreachable).

Test 2 (Direct FD Path): FAILED (Confirmed /proc/self/fd/0 unreachable).

Test 3 (Regular File Buffer): SUCCESS (Confirmed reliable execution).

5. Implementation Notes

The updated wsrep_sst_common script now utilizes the mktemp utility to ensure unique file names and implements immediate cleanup (rm -f) to ensure no sensitive credential data persists in /tmp after the transfer is complete.

[it-percona-cla]

All committers have signed the CLA.