Skip to content

Commit eef5db6

Browse files
robertbpughclaude
andauthored
Docker: Skip busy ports when seeding a worktree's instance (#53935)
* Docker: skip busy host ports when seeding a worktree's instance seed-worktree-env.sh now checks 127.0.0.1 for a listener before it hands out a port, and moves to the next free one. Before, it could pick a port another app (an SSH tunnel on 8080, say) was already using. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFjM4HdEDHT7uiZDdjhUMQ * Docker: bound the port probe and check IPv6 localhost too A listener with a full accept queue can drop the connect, so give each probe about a second and treat a timeout as busy. Probe ::1 as well as 127.0.0.1. In the fixture test, compute expected ports before each run, fail clearly if the test listener never binds, and use a larger backlog. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFjM4HdEDHT7uiZDdjhUMQ * Docker: bound the fixture's port probe and test a full accept queue The test's listening helper now gives up like the script does, and a new case checks a listener with a full accept queue is skipped without stalling the seeder. The README says reused ports are kept as is. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFjM4HdEDHT7uiZDdjhUMQ * Docker: make the full-queue fixture case check its own setup Fill the queue with listen(1) and confirm a fresh connect times out before running the case, polling for the listener instead of a fixed sleep. Where the OS still accepts the connect, the case is skipped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFjM4HdEDHT7uiZDdjhUMQ --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent c15c6e7 commit eef5db6

4 files changed

Lines changed: 167 additions & 26 deletions

File tree

‎.agents/skills/work-on.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -385,7 +385,7 @@ Written by `checkpoint.sh` at every phase boundary; read by coordinators. Never
385385

386386
Main `jetpack_dev` uses: WP 80, phpMyAdmin 8181, Mailpit inbox 1080, Mailpit SMTP 25, SFTP 1022. The seeder allocates the first free value at or above a per-service base — WP `8080`, phpMyAdmin `8282`, inbox `1180`, SMTP `2525`, SFTP `2222` — so a first worktree usually lands exactly on those, and later ones step up from there. Values are persisted in the worktree's `tools/docker/.env` and stay stable across restarts.
387387

388-
The seeder reserves ports recorded in other worktrees' `.env` files, but does not probe for live-bound host ports and does not lock against two worktrees seeding at the same instant. A genuine clash surfaces as `address already in use` from `jp docker up` — edit the port in `tools/docker/.env` and retry, then update `.work-on/env.json` to match. See `tools/docker/README.md` § "Parallel development environments" for the authoritative rules.
388+
The seeder skips ports recorded in other worktrees' `.env` files and ports that already have a listener on localhost, but does not lock against two worktrees seeding at the same instant. A genuine clash surfaces as `address already in use` from `jp docker up` — edit the port in `tools/docker/.env` and retry, then update `.work-on/env.json` to match. See `tools/docker/README.md` § "Parallel development environments" for the authoritative rules.
389389

390390
## Docker instance tracking
391391

‎tools/docker/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ This writes a unique `COMPOSE_PROJECT_NAME` (derived from git's worktree id —
169169
- **idempotent** — it only fills in the keys that are missing (so it also backfills ports for a name-only `.env` left by `up --name`) and is a no-op once fully configured, so it's safe to run before every `up`;
170170
- **a no-op in the primary checkout** — that always stays `jetpack_dev` on the default ports.
171171

172-
It reserves ports already recorded in other worktrees' `.env` files and the primary defaults, but does not probe for live-bound host ports, and does not lock against another worktree seeding at the same instant — so two worktrees seeded in parallel can, in principle, pick the same port. Either way a genuine clash surfaces when `jetpack docker up` reports `address already in use`; edit the port in `.env` and retry.
172+
When it picks a new port, it skips ports already recorded in other worktrees' `.env` files, the primary defaults, and any port that already has a listener on localhost (`127.0.0.1` or `::1`). A port already in this worktree's `.env` is kept as is. It does not lock against another worktree seeding at the same instant, so two worktrees seeded in parallel can, in principle, pick the same port. A genuine clash surfaces when `jetpack docker up` reports `address already in use`; edit the port in `.env` and retry.
173173

174174
To undo it, delete the seeded lines from `tools/docker/.env` (the file is git-ignored and removed with the worktree). To set the name/ports yourself instead, edit `.env` by hand or use the manual flags below.
175175

‎tools/docker/bin/seed-worktree-env.sh‎

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,10 @@
1515
# - idempotent (it only fills the keys that are missing; a no-op once fully configured), and
1616
# - a no-op in the primary checkout, which keeps using `jetpack_dev`.
1717
#
18-
# Known limitations (acceptable for a manual dev helper): it reserves ports recorded in other
19-
# worktrees' .env files and the primary defaults, but does NOT probe for live-bound host ports,
20-
# and does not lock against another worktree seeding at the same instant. A genuine clash
21-
# surfaces as a `jp docker up` "address already in use" error; re-run after editing .env.
18+
# It skips ports recorded in other worktrees' .env files, the primary defaults, and any port that
19+
# already has a listener on localhost. Known limitation (acceptable for a manual dev helper): it
20+
# does not lock against another worktree seeding at the same instant. A genuine clash surfaces as
21+
# a `jp docker up` "address already in use" error; re-run after editing .env.
2222
#
2323
# Written for portability down to the bash 3.2 that ships with macOS — no associative arrays.
2424
#
@@ -57,13 +57,38 @@ is_claimed() {
5757
esac
5858
}
5959

60+
# Is something already listening on localhost port $1, such as another app or an SSH tunnel?
61+
# A closed port refuses at once. A probe still waiting after about a second (a listener whose
62+
# accept queue is full can drop the connect) counts as busy rather than stalling the script.
63+
port_in_use() {
64+
local host pid waited
65+
for host in 127.0.0.1 ::1; do
66+
( : < "/dev/tcp/$host/$1" ) 2>/dev/null &
67+
pid=$!
68+
waited=0
69+
while kill -0 "$pid" 2>/dev/null; do
70+
if [ "$waited" -ge 20 ]; then
71+
kill "$pid" 2>/dev/null || true
72+
wait "$pid" 2>/dev/null || true
73+
return 0
74+
fi
75+
sleep 0.05
76+
waited=$(( waited + 1 ))
77+
done
78+
if wait "$pid"; then
79+
return 0
80+
fi
81+
done
82+
return 1
83+
}
84+
6085
# Reserve the first free port at or above $1 into CLAIMED and return it via ALLOCATED. Writing
6186
# to a global (rather than echoing via command substitution, which runs in a subshell and would
6287
# discard the CLAIMED reservation) keeps the five services from ever landing on the same number.
6388
ALLOCATED=""
6489
alloc_port() {
6590
local port="$1"
66-
while is_claimed "$port"; do
91+
while is_claimed "$port" || port_in_use "$port"; do
6792
port=$(( port + 1 ))
6893
done
6994
CLAIMED="${CLAIMED}${port} "

‎tools/docker/bin/seed-worktree-env.test.sh‎

Lines changed: 135 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -50,16 +50,49 @@ run() {
5050
RUN_OUT="$( cd "$1" && ./tools/docker/bin/seed-worktree-env.sh 2>&1 )" && RUN_RC=0 || RUN_RC=$?
5151
}
5252

53-
# --- a primary repo with the script committed at tools/docker/bin ----------
53+
# listening PORT — is localhost:PORT taken? Same rules as the script's port_in_use (a connect
54+
# still pending after about a second counts as taken), defined here so expectations don't come from it.
55+
listening() {
56+
local host pid waited
57+
for host in 127.0.0.1 ::1; do
58+
( : < "/dev/tcp/$host/$1" ) 2>/dev/null &
59+
pid=$!
60+
waited=0
61+
while kill -0 "$pid" 2>/dev/null; do
62+
if [ "$waited" -ge 20 ]; then
63+
kill "$pid" 2>/dev/null || true
64+
wait "$pid" 2>/dev/null || true
65+
return 0
66+
fi
67+
sleep 0.05
68+
waited=$(( waited + 1 ))
69+
done
70+
if wait "$pid"; then return 0; fi
71+
done
72+
return 1
73+
}
74+
# first_free PORT — the first port at or above PORT with no listener. The script skips busy
75+
# ports, so on a dev machine with 8080 taken the expected values move up with it.
76+
first_free() {
77+
local p="$1"
78+
while listening "$p"; do p=$(( p + 1 )); done
79+
printf '%s' "$p"
80+
}
81+
82+
# new_primary DIR — a primary repo with the script committed at tools/docker/bin.
83+
new_primary() {
84+
mkdir -p "$1/tools/docker/bin"
85+
git -C "$1" init -q
86+
git -C "$1" config user.email test@example.test
87+
git -C "$1" config user.name seed-test
88+
cp "$SUT" "$1/tools/docker/bin/seed-worktree-env.sh"
89+
chmod +x "$1/tools/docker/bin/seed-worktree-env.sh"
90+
git -C "$1" add -A
91+
git -C "$1" commit -qm init
92+
}
93+
5494
PRIMARY="$TMP/primary"
55-
mkdir -p "$PRIMARY/tools/docker/bin"
56-
git -C "$PRIMARY" init -q
57-
git -C "$PRIMARY" config user.email test@example.test
58-
git -C "$PRIMARY" config user.name seed-test
59-
cp "$SUT" "$PRIMARY/tools/docker/bin/seed-worktree-env.sh"
60-
chmod +x "$PRIMARY/tools/docker/bin/seed-worktree-env.sh"
61-
git -C "$PRIMARY" add -A
62-
git -C "$PRIMARY" commit -qm init
95+
new_primary "$PRIMARY"
6396

6497
add_wt() { git -C "$PRIMARY" worktree add -q "$TMP/$1" -b "br-$1"; }
6598
env_of() { printf '%s/tools/docker/.env' "$TMP/$1"; }
@@ -71,10 +104,11 @@ assert_absent "$PRIMARY/tools/docker/.env" 'COMPOSE_PROJECT_NAME' "primary .env
71104

72105
# --- a linked worktree seeds a unique name + base ports --------------------
73106
add_wt wt-a
107+
WP_A="$( first_free 8080 )"
74108
run "$TMP/wt-a"
75109
assert_rc zero "linked worktree seeds (exit 0)"
76110
assert_grep "$( env_of wt-a )" '^COMPOSE_PROJECT_NAME=jetpack_wt-a$' "name derives from the git worktree id"
77-
assert_grep "$( env_of wt-a )" '^PORT_WORDPRESS=8080$' "base WordPress port lands at 8080"
111+
assert_grep "$( env_of wt-a )" "^PORT_WORDPRESS=${WP_A}\$" "WordPress port lands on the first free port from 8080"
78112

79113
# --- re-running is idempotent ----------------------------------------------
80114
run "$TMP/wt-a"
@@ -84,10 +118,11 @@ if [ "$( grep -c '^COMPOSE_PROJECT_NAME=' "$( env_of wt-a )" )" = 1 ]; then pass
84118
# --- name-only .env gets its ports backfilled around the sibling -----------
85119
add_wt wt-b
86120
printf 'COMPOSE_PROJECT_NAME=jetpack_beta\n' > "$( env_of wt-b )"
121+
WP_B="$( first_free $(( WP_A + 1 )) )"
87122
run "$TMP/wt-b"
88123
assert_rc zero "name-only .env backfills (exit 0)"
89124
assert_grep "$( env_of wt-b )" '^COMPOSE_PROJECT_NAME=jetpack_beta$' "hand-set name is kept"
90-
assert_grep "$( env_of wt-b )" '^PORT_WORDPRESS=8081$' "backfilled port avoids the sibling's 8080"
125+
assert_grep "$( env_of wt-b )" "^PORT_WORDPRESS=${WP_B}\$" "backfilled port avoids the sibling's port"
91126

92127
# --- a name a sibling already uses is refused (concern 1) ------------------
93128
add_wt wt-c
@@ -206,14 +241,7 @@ assert_rc nonzero "an unbalanced-quote name is rejected (Compose-invalid)"
206241
# sibling pins base ports via a colon-space and an export form; without the lenient sibling reader
207242
# the fresh worktree would reuse them, colliding at `up`.
208243
PRIMARY2="$TMP/primary2"
209-
mkdir -p "$PRIMARY2/tools/docker/bin"
210-
git -C "$PRIMARY2" init -q
211-
git -C "$PRIMARY2" config user.email test@example.test
212-
git -C "$PRIMARY2" config user.name seed-test
213-
cp "$SUT" "$PRIMARY2/tools/docker/bin/seed-worktree-env.sh"
214-
chmod +x "$PRIMARY2/tools/docker/bin/seed-worktree-env.sh"
215-
git -C "$PRIMARY2" add -A
216-
git -C "$PRIMARY2" commit -qm init
244+
new_primary "$PRIMARY2"
217245
git -C "$PRIMARY2" worktree add -q "$TMP/p2-sib" -b br-p2-sib
218246
git -C "$PRIMARY2" worktree add -q "$TMP/p2-fresh" -b br-p2-fresh
219247
printf 'COMPOSE_PROJECT_NAME: jetpack_p2sib\nPORT_WORDPRESS: 8080\nexport PORT_PHPMY=8282\n' > "$TMP/p2-sib/tools/docker/.env"
@@ -228,6 +256,94 @@ printf 'COMPOSE_PROJECT_NAME=jetpack_p2sib\n' > "$TMP/p2-dupe/tools/docker/.env"
228256
run "$TMP/p2-dupe"
229257
assert_rc nonzero "a name duplicating a colon-space sibling is rejected"
230258

259+
# --- a port something is already listening on is skipped -------------------
260+
# A fresh repo, so no sibling claims anything; hold the port the script would otherwise pick.
261+
PRIMARY3="$TMP/primary3"
262+
new_primary "$PRIMARY3"
263+
git -C "$PRIMARY3" worktree add -q "$TMP/p3-wt" -b br-p3-wt
264+
BUSY="$( first_free 8080 )"
265+
if command -v python3 >/dev/null 2>&1; then
266+
# Bounded sleep, so the listener exits on its own even if this script dies before the kill.
267+
python3 -c 'import socket, sys, time
268+
s = socket.socket()
269+
s.bind(("127.0.0.1", int(sys.argv[1])))
270+
s.listen(16)
271+
time.sleep(30)' "$BUSY" &
272+
LISTENER=$!
273+
for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do
274+
listening "$BUSY" && break
275+
sleep 0.25
276+
done
277+
if listening "$BUSY"; then
278+
EXPECTED="$( first_free $(( BUSY + 1 )) )"
279+
run "$TMP/p3-wt"
280+
assert_rc zero "seeds beside a busy port (exit 0)"
281+
assert_grep "$TMP/p3-wt/tools/docker/.env" "^PORT_WORDPRESS=${EXPECTED}\$" "busy port ${BUSY} is skipped for the next free one"
282+
else
283+
bad "test listener never came up on port ${BUSY}, so the busy-port case could not run"
284+
fi
285+
kill "$LISTENER" 2>/dev/null || true
286+
wait "$LISTENER" 2>/dev/null || true
287+
288+
# A listener whose accept queue is full drops new connects instead of refusing them. Fresh repo
289+
# again, so the port the previous case wrote to .env is not claimed.
290+
PRIMARY4="$TMP/primary4"
291+
new_primary "$PRIMARY4"
292+
git -C "$PRIMARY4" worktree add -q "$TMP/p4-wt" -b br-p4-wt
293+
FULL="$( first_free 8080 )"
294+
READY="$TMP/full-queue-ready"
295+
# Writes "full" to READY only once a fresh connect really times out, so the probe's bound is
296+
# what this case exercises (an OS that still accepts it writes "open" and the case is skipped).
297+
python3 -c 'import socket, sys, time
298+
port, ready = int(sys.argv[1]), sys.argv[2]
299+
s = socket.socket()
300+
s.bind(("127.0.0.1", port))
301+
s.listen(1)
302+
held = []
303+
for _ in range(16):
304+
c = socket.socket()
305+
c.setblocking(False)
306+
try:
307+
c.connect(("127.0.0.1", port))
308+
except BlockingIOError:
309+
pass
310+
held.append(c)
311+
probe = socket.socket()
312+
probe.settimeout(0.5)
313+
try:
314+
probe.connect(("127.0.0.1", port))
315+
state = "open"
316+
except OSError:
317+
state = "full"
318+
with open(ready, "w") as f:
319+
f.write(state)
320+
time.sleep(30)' "$FULL" "$READY" &
321+
LISTENER=$!
322+
for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do
323+
[ -s "$READY" ] && break
324+
sleep 0.25
325+
done
326+
case "$( cat "$READY" 2>/dev/null )" in
327+
full )
328+
EXPECTED="$( first_free $(( FULL + 1 )) )"
329+
START=$SECONDS
330+
run "$TMP/p4-wt"
331+
if [ $(( SECONDS - START )) -le 10 ]; then
332+
pass "a full accept queue does not stall the seeder"
333+
else
334+
bad "a full accept queue does not stall the seeder (took $(( SECONDS - START ))s)"
335+
fi
336+
assert_grep "$TMP/p4-wt/tools/docker/.env" "^PORT_WORDPRESS=${EXPECTED}\$" "port ${FULL} with a full accept queue is skipped"
337+
;;
338+
open ) echo " skip - this OS accepted a connect past the full queue, so the timeout case did not run" ;;
339+
* ) bad "full-queue test listener never came up on port ${FULL}" ;;
340+
esac
341+
kill "$LISTENER" 2>/dev/null || true
342+
wait "$LISTENER" 2>/dev/null || true
343+
else
344+
echo " skip - python3 not found, so the busy-port cases did not run"
345+
fi
346+
231347
# --- reaching the primary through a symlinked path is still a no-op ---------
232348
# (Guards the pwd -P handling that resolves --absolute-git-dir's symlink resolution.)
233349
ln -s "$PRIMARY" "$TMP/primary-link"

0 commit comments

Comments
 (0)