Skip to content

Commit ce7c6df

Browse files
cppudgeclaude
andcommitted
modules: align the module API surface for 0.2.0 (audit group A)
Uniform pass-through matrix across all seven modules: Redis gains env/label/network(x2)/alias/reuse/startup_timeout/startup_attempts, Kafka gains with_label, RabbitMQ and MongoDB gain with_env + with_label. Env ownership is guarded where append-order cannot protect it: Redis rejects with_env REDISCLI_AUTH alongside a password at render (the key is read by exec'd redis-cli, where the FIRST duplicate wins) and MongoDB rejects MONGO_INITDB_ROOT_* (keyfile requirement under --replSet). Command-arg spelling unified: with_command_arg (single) / with_command_args (batch) on Redis and the MySQL family. PostgreSQL's connection_string(scheme) split into connection_string() + connection_string_with_scheme() - a string argument to connection_string(...) now always means the database (MongoDB), never the scheme. with_startup_timeout is a per-phase budget everywhere: MongoDB's PRIMARY poll derives from it (chunked ~15s execs; the hook's steady_clock deadline is the one authority - the buffered exec runs with the transport deadline disabled) instead of a fixed ~20s cap. New umbrella header testcontainers/modules.hpp, compile-checked by the Redis unit TU. Pipeline: MSVC 559 units, live module suite 34/34, WSL gcc 553 units, clang-tidy and clang-format clean. Opus review applied: the mongo chunking rationale misattributed the safety to the transport idle deadline; reworded in code and feature-notes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141x8KrNdD9NfmCh1vSMbpg
1 parent afa5494 commit ce7c6df

19 files changed

Lines changed: 419 additions & 65 deletions

docs/feature-notes.md

Lines changed: 34 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -428,7 +428,16 @@ module: `with_customizer(fn)` queues a callback over the underlying `GenericImag
428428
render time AFTER the module's own rendering (what it sets wins — the same precedence idea
429429
as `with_create_body_patch` over typed fields), and `to_generic()` renders the full config
430430
into a plain `GenericImage` for when a raw `Container` is wanted. Each module documents the
431-
small surface it owns (cmd/env keys); everything else passes through untouched. Opt-in by
431+
small surface it owns (cmd/env keys); everything else passes through untouched. Aligned in
432+
the pre-0.2.0 audit pass (2026-07-12): every module surfaces the same curated pass-through
433+
set — env / label / network (name or `Network`) / network-alias / reuse / startup-timeout /
434+
startup-attempts; `with_startup_timeout` is a PER-PHASE budget (one phase for most modules;
435+
Kafka and MongoDB run a second, hook-driven phase with a fresh allowance of the same size);
436+
command-line arguments use one spelling pair wherever they apply (`with_command_arg` single
437+
/ `with_command_args` batch — Redis and the MySQL family); and a string argument to
438+
`connection_string(...)` always means the DATABASE (MongoDB) — scheme variants get their
439+
own name (PostgreSQL's `connection_string_with_scheme`). `testcontainers/modules.hpp` is
440+
the modules umbrella header (compile-checked by the Redis unit TU). Opt-in by
432441
include + link: headers under `include/testcontainers/modules/`, target
433442
`testcontainers::modules` — the one spelling that means the same thing on both consumption
434443
paths. Around it the paths differ: the plain-CMake install exports
@@ -450,10 +459,14 @@ wait (a log wait races the listener; a raw TCP probe false-positives through Doc
450459
Desktop's host proxy). `with_password` renders `{"redis-server","--requirepass",pw}` AND
451460
sets container-level `REDISCLI_AUTH`, so the unchanged probe — and any `redis-cli` the user
452461
execs — authenticates automatically (honored by redis-cli ≥ 4.0.10; the pin is far past
453-
it). `with_command_args` appends server arguments after `--requirepass`; argv[0] stays
462+
it). `with_command_arg[s]` append server arguments after `--requirepass`; argv[0] stays
454463
`redis-server` so the official entrypoint's protected-mode handling keeps applying. The
455464
module owns the container command (iff a password or args are set) and the `REDISCLI_AUTH`
456-
env key — nothing else. `connection_string(db = 0)` renders
465+
env key — nothing else. The env key is guarded, not just documented: `with_env` carrying
466+
REDISCLI_AUTH alongside a password throws at render — the key is read by EXEC'D redis-cli,
467+
where the FIRST duplicate of a key wins, so the append-last ordering that protects the DB
468+
modules' bash-read credential keys could not make the module's entry win here.
469+
`connection_string(db = 0)` renders
457470
`redis://[:pass@]host:port[/db]`; database selection is client-side in Redis, so there is
458471
no server-side database setter. Known limits: a config FILE must be the first server
459472
argument, so a config file combined with `with_password` is unsupported (drop to a
@@ -481,7 +494,8 @@ executable (a non-executable .sh is *sourced* into the entrypoint's shell, where
481494
editing a script changes the reuse hash). `with_config_option` renders `postgres -c k=v`;
482495
`with_wait` REPLACES the default probe (a customizer-added wait runs in addition).
483496
`StartedPostgreSQL` adds `conninfo()` (libpq keyword/value with its quoting rules; the
484-
password keyword omitted when empty) beside the URI `connection_string(scheme)`, and
497+
password keyword omitted when empty) beside the URI `connection_string()` /
498+
`connection_string_with_scheme()`, and
485499
`exec_sql()` through the in-container psql (`-X -tA`, local-socket trust — no password).
486500
Curated pass-throughs (env / label / network / alias / reuse / timeout / attempts) forward
487501
to the embedded builder; everything else rides `with_customizer`.
@@ -508,7 +522,7 @@ names). 500ms poll (each attempt is a fresh exec connection; 200ms is churn agai
508522
measured in tens of seconds). Init scripts and `.cnf` drop-ins reuse the PostgreSQL
509523
staging rules (NNNN- registration-order prefix, extension whitelist, .sh 0755;
510524
`/etc/mysql/conf.d` names must end in .cnf — the include glob skips others silently).
511-
`with_command_arg` values become the container cmd verbatim (the entrypoints forward
525+
`with_command_arg[s]` values become the container cmd verbatim (the entrypoints forward
512526
'-'-prefixed args to the server binary). Both `connection_string()`s emit the **mysql://**
513527
scheme — MariaDB speaks the MySQL wire protocol and URL-parsing clients widely reject
514528
"mariadb://"; `root_password()` documents the root≡user invariant at call sites. Known
@@ -593,16 +607,21 @@ set (`--replSet rs0 --bind_ip_all`): transactions and change streams are why the
593607
exists, a standalone rejects both, and the whole cost is a ~1–2s election on top of the
594608
boot. There is deliberately no standalone mode and no auth surface — MongoDB requires a
595609
cluster keyfile the moment auth meets a replica set, so MONGO_INITDB_ROOT_* is a
596-
boot-breaker under --replSet (the header warns customizer users off it; initdb.d scripts
597-
are equally unsupported — they trigger a temporary double-start whose log line would
598-
release the wait early). Boot choreography: log wait "Waiting for connections" (exact 4.4+
599-
casing; appears exactly once since nothing triggers the initdb phase) → listening_port
600-
(proves the HOST side of the mapping) → a started hook that execs `rs.initiate({_id,
601-
members: [{_id: 0, host: '127.0.0.1:27017'}]})` (deterministic self-check member address;
602-
AlreadyInitialized tolerated for pre-initiated volumes and reuse) and then polls
603-
`db.hello().isWritablePrimary` in-shell (200×100ms ≈ 20s cap) — the PRIMARY wait cannot be
604-
a wait strategy because waits run before hooks, i.e. before rs.initiate exists. THE DSN
605-
decision: `connection_string()` emits
610+
boot-breaker under --replSet (`with_env` rejects both root keys at render; the header
611+
warns customizer users off them; initdb.d scripts are equally unsupported — they trigger
612+
a temporary double-start whose log line would release the wait early). Boot choreography:
613+
log wait "Waiting for connections" (exact 4.4+ casing; appears exactly once since nothing
614+
triggers the initdb phase) → listening_port (proves the HOST side of the mapping) → a
615+
started hook that execs `rs.initiate({_id, members: [{_id: 0, host:
616+
'127.0.0.1:27017'}]})` (deterministic self-check member address; AlreadyInitialized
617+
tolerated for pre-initiated volumes and reuse) and then polls
618+
`db.hello().isWritablePrimary` in-shell on 100ms ticks under a fresh phase budget (the
619+
configured startup timeout — the same per-phase contract as Kafka's two-phase boot),
620+
chunked into execs of ~15s so the wall-clock phase deadline is re-evaluated between
621+
chunks and a transiently failing mongosh retries (the buffered exec itself runs with the
622+
transport deadline disabled — the hook's steady_clock deadline is the one authority) —
623+
the PRIMARY wait cannot be a wait strategy because waits run before hooks, i.e. before
624+
rs.initiate exists. THE DSN decision: `connection_string()` emits
606625
`mongodb://host:port/<db>?directConnection=true` and NEVER `replicaSet=`
607626
direct mode pins single-server behavior in every spec-compliant driver instead of relying
608627
on per-driver legacy defaults (the "works in tc-java, ServerSelectionTimeoutError in

docs/public-api-test-coverage.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -332,15 +332,18 @@ three has a Windows-mode test (file modes and privileged exec are Unix concepts
332332
Module images are Linux-only, so the Windows columns are n/a by construction —
333333
the `tc_module_tests` exe runs on the Windows CI job solely to prove the engine
334334
guards self-skip. Rendering rules (cmd/env ownership, customizer precedence,
335-
idempotence) are unit-tested per module via `to_generic()`.
335+
idempotence) are unit-tested per module via `to_generic()`. The modules umbrella
336+
header (`testcontainers/modules.hpp`) is compile-checked by the Redis unit TU,
337+
which includes it in place of the individual header.
336338

337339
| Function | Works on Linux | Works on Windows | Integration-tested (Linux) | Integration-tested (Windows) |
338340
|---|---|---|---|---|
339341
| `GenericImage::with_image` (core setter) ||| ❌ (unit-tested: GenericImage.WithImageReplacesReferenceKeepingOptions; every module `with_image` uses it) ||
340342
| `RedisContainer()` defaults + `start()` || n/a (Linux image) | ✅ RedisModule.StartsServesAndBuildsDsn | n/a |
341343
| `RedisContainer::with_image` || n/a | ❌ (unit-tested: RedisModuleConfig.WithImageRewritesReference) | n/a |
342344
| `RedisContainer::with_password` || n/a | ✅ RedisModule.PasswordIsEnforcedAndWired | n/a |
343-
| `RedisContainer::with_command_args` || n/a | ✅ RedisModule.CommandArgsReachTheServer | n/a |
345+
| `RedisContainer::with_command_arg[s]` || n/a | ✅ RedisModule.CommandArgsReachTheServer (batch form; single twin unit-tested: RedisModuleConfig.CommandArgSingleTwinAccumulates) | n/a |
346+
| `RedisContainer` pass-throughs (env/label/network/alias/reuse/timeout/attempts) || n/a | ❌ (unit-tested: RedisModuleConfig.PassThroughsLandOnTheImage, RedisModuleConfig.ManagedAuthEnvConflictThrowsAtRender; thin forwards to core setters, each integration-tested under GenericImage above) | n/a |
344347
| `RedisContainer::with_customizer` || n/a | ❌ (unit-tested: RedisModuleConfig.CustomizerRunsLastAndWins — pure rendering, no daemon interaction of its own) | n/a |
345348
| `RedisContainer::to_generic` || n/a | ❌ (unit-tested: RedisModuleConfig.*`start()` goes through it) | n/a |
346349
| `StartedRedis::host` / `port` / `connection_string` / `password` || n/a | ✅ RedisModule.StartsServesAndBuildsDsn, RedisModule.PasswordIsEnforcedAndWired | n/a |
@@ -353,12 +356,12 @@ idempotence) are unit-tested per module via `to_generic()`.
353356
| `PostgreSQLContainer::with_wait` / `with_env` || n/a | ❌ (unit-tested: PostgreSQLModuleConfig.CustomWaitReplacesDefaultProbe, PostgreSQLModuleConfig.CredentialTrioAppendedLastWinsOverRawEnv) | n/a |
354357
| `PostgreSQLContainer` other pass-throughs (label/network/alias/timeout/attempts) || n/a | ❌ (thin forwards to core setters, each integration-tested under GenericImage above) | n/a |
355358
| `PostgreSQLContainer::with_customizer` / `to_generic` || n/a | ✅ PostgreSQLModule.CustomizerReachesCreateBody | n/a |
356-
| `StartedPostgreSQL::host` / `port` / `connection_string` / `conninfo` || n/a | ✅ PostgreSQLModule.DefaultsStartAndConnect, PostgreSQLModule.HostSidePgHandshake | n/a |
359+
| `StartedPostgreSQL::host` / `port` / `connection_string[_with_scheme]` / `conninfo` || n/a | ✅ PostgreSQLModule.DefaultsStartAndConnect, PostgreSQLModule.HostSidePgHandshake | n/a |
357360
| `StartedPostgreSQL::exec_sql` || n/a | ✅ every PostgreSQLModule test | n/a |
358361
| `MySQLContainer()` defaults + `start()` || n/a | ✅ MySQLModule.DefaultsBootAndConnect | n/a |
359362
| `MySQLContainer::with_username/password/database` (incl. root modes) || n/a | ✅ MySQLModule.CustomCredsAndOrderedInitScripts, MySQLModule.RootOnlyModes | n/a |
360363
| `MySQLContainer::with_init_script` (host file / in-memory) || n/a | ✅ MySQLModule.CustomCredsAndOrderedInitScripts | n/a |
361-
| `MySQLContainer::with_command_arg` || n/a | ✅ MySQLModule.CharsetCommandArg | n/a |
364+
| `MySQLContainer::with_command_arg[s]` || n/a | ✅ MySQLModule.CharsetCommandArg (batch twin unit-tested: MySqlFamilyConfig.CommandArgsBecomeCmd) | n/a |
362365
| `MySQLContainer::with_config_file` || n/a | ❌ (unit-tested staging: MySqlFamilyConfig.InitScriptsAndConfigFilesStageOrderedAndValidated; the copy mechanics are MariaDB-integration-tested — shared core) | n/a |
363366
| `MariaDBContainer()` defaults + `start()` (healthcheck.sh wait) || n/a | ✅ MariaDBModule.DefaultsBootAndConnect | n/a |
364367
| `MariaDBContainer::with_init_script` / `with_config_file` || n/a | ✅ MariaDBModule.InitScriptAndConfigFile | n/a |
@@ -367,7 +370,7 @@ idempotence) are unit-tested per module via `to_generic()`.
367370
| `KafkaContainer()` defaults + `start()` (two-phase boot) || n/a | ✅ KafkaModule.StartsAndExposesBootstrap, KafkaModule.AdvertisedListenersCarryMappedPort (the mapped-port money test), KafkaModule.ProduceConsumeRoundTrip | n/a |
368371
| `KafkaContainer::with_topic` || n/a | ✅ KafkaModule.WithTopicPreCreatesPartitions | n/a |
369372
| `KafkaContainer::with_network` / `with_network_alias` (advertised internal listener) || n/a | ✅ KafkaModule.TwoContainersOverNetwork | n/a |
370-
| `KafkaContainer::with_env` / `with_cluster_id` / `with_customizer` / `to_generic` + detail helpers || n/a | ❌ (unit-tested: KafkaModuleConfig.*, KafkaDetail.* — env order, cluster-id validation, starter script, placeholder command, topics label) | n/a |
373+
| `KafkaContainer::with_env` / `with_label` / `with_cluster_id` / `with_customizer` / `to_generic` + detail helpers || n/a | ❌ (unit-tested: KafkaModuleConfig.*, KafkaDetail.* — env order, label order vs the reserved topics label, cluster-id validation, starter script, placeholder command) | n/a |
371374
| `StartedKafka` getters (`bootstrap_servers` bare host:port, `internal_bootstrap_servers`, `cluster_id`) || n/a | ✅ KafkaModule.StartsAndExposesBootstrap, KafkaModule.TwoContainersOverNetwork | n/a |
372375
| `RabbitMQContainer()` defaults + `start()` (ordered log→exec readiness) || n/a | ✅ RabbitMQModule.DefaultsStartAndUrls | n/a |
373376
| `RabbitMQContainer::with_username/password/vhost` || n/a | ✅ RabbitMQModule.CustomCredentialsAndVhost | n/a |

include/testcontainers/modules.hpp

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#pragma once
2+
3+
/// Umbrella header for the ecosystem modules (namespace
4+
/// testcontainers::modules; link testcontainers::modules). The core API has
5+
/// its own umbrella, testcontainers/testcontainers.hpp — the module headers
6+
/// pull in only the core pieces they build on.
7+
8+
#include "testcontainers/modules/KafkaContainer.hpp"
9+
#include "testcontainers/modules/MariaDBContainer.hpp"
10+
#include "testcontainers/modules/MongoDBContainer.hpp"
11+
#include "testcontainers/modules/MySQLContainer.hpp"
12+
#include "testcontainers/modules/PostgreSQLContainer.hpp"
13+
#include "testcontainers/modules/RabbitMQContainer.hpp"
14+
#include "testcontainers/modules/RedisContainer.hpp"

include/testcontainers/modules/KafkaContainer.hpp

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,14 @@ class KafkaContainer {
7979
return *this;
8080
}
8181

82+
/// Attach a metadata label. The module's reuse-visibility label
83+
/// (`org.testcontainers.kafka.topics`) is applied after these, so it
84+
/// wins on a duplicate key.
85+
KafkaContainer& with_label(std::string key, std::string value) {
86+
image_.with_label(std::move(key), std::move(value));
87+
return *this;
88+
}
89+
8290
/// Set the KRaft cluster id: 22 characters of URL-safe base64 (a 16-byte
8391
/// UUID, e.g. from `kafka-storage.sh random-uuid`). Defaults to a fixed
8492
/// module-wide id, which keeps restarts and container reuse
@@ -145,8 +153,8 @@ class KafkaContainer {
145153
}
146154

147155
/// Register a callback that customizes the underlying `GenericImage` —
148-
/// the channel for options this module does not surface (labels, pull
149-
/// policy, mounts, ...). Customizers run when the config is rendered
156+
/// the channel for options this module does not surface (pull policy,
157+
/// mounts, ...). Customizers run when the config is rendered
150158
/// (`start()` / `to_generic()`), in registration order, AFTER the
151159
/// module's own rendering. Caveats: the boot choreography lives in the
152160
/// rendered command, wait, and started hook — replacing any of them

include/testcontainers/modules/MariaDBContainer.hpp

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
#include <string>
88
#include <string_view>
99
#include <utility>
10+
#include <vector>
1011

1112
#include "testcontainers/Container.hpp"
1213
#include "testcontainers/GenericImage.hpp"
@@ -102,6 +103,15 @@ class MariaDBContainer {
102103
return *this;
103104
}
104105

106+
/// Append several server options at once — the batch twin of
107+
/// `with_command_arg`. Repeatable; calls accumulate in order.
108+
MariaDBContainer& with_command_args(std::vector<std::string> args) {
109+
for (std::string& arg : args) {
110+
opts_.command_args.push_back(std::move(arg));
111+
}
112+
return *this;
113+
}
114+
105115
/// Copy a .cnf file into /etc/mysql/conf.d (included by the image's
106116
/// configuration; the name must end in ".cnf" or this throws). Copied
107117
/// with mode 0644 — the server ignores world-writable config files.

include/testcontainers/modules/MongoDBContainer.hpp

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,22 @@ class MongoDBContainer {
7777

7878
// --- Pass-throughs to the underlying builder ---
7979

80+
/// Set an extra environment variable (TZ and friends). Do not set
81+
/// MONGO_INITDB_ROOT_USERNAME / MONGO_INITDB_ROOT_PASSWORD here: an
82+
/// auth-enabled replica-set member refuses to start without a cluster
83+
/// keyfile (see the class note), so `start()` throws up front on either
84+
/// key. The other MONGO_INITDB_* keys only affect initdb.d scripts,
85+
/// which this module deliberately does not stage.
86+
MongoDBContainer& with_env(std::string key, std::string value) {
87+
image_.with_env(std::move(key), std::move(value));
88+
return *this;
89+
}
90+
91+
MongoDBContainer& with_label(std::string key, std::string value) {
92+
image_.with_label(std::move(key), std::move(value));
93+
return *this;
94+
}
95+
8096
/// Join a user-defined network; peers resolve this container by
8197
/// name/alias at `<alias>:27017` (kPort, not the mapped host port).
8298
MongoDBContainer& with_network(std::string network) {
@@ -98,9 +114,11 @@ class MongoDBContainer {
98114
return *this;
99115
}
100116

101-
/// Budget for the container-level readiness phase (default: 60s; the
102-
/// replica-set initiation that follows carries its own ~20s cap). Image
103-
/// pull time does not count against it.
117+
/// Budget for EACH of the two startup phases — the container-level
118+
/// readiness wait, and the replica-set initiation (rs.initiate + the
119+
/// PRIMARY election poll) that follows it — so the worst-case total is
120+
/// about twice this. Default 60s per phase. Image pull time does not
121+
/// count against it.
104122
MongoDBContainer& with_startup_timeout(std::chrono::milliseconds timeout) {
105123
image_.with_startup_timeout(timeout);
106124
return *this;
@@ -137,8 +155,9 @@ class MongoDBContainer {
137155
/// hatch when you need a raw `Container` instead of a StartedMongoDB.
138156
/// Extend the result, don't rebuild it: replacing the command or the
139157
/// hooks disables the replica-set choreography. Throws Error on an
140-
/// invalid config (a replica-set name outside [A-Za-z0-9_-]) before any
141-
/// daemon contact.
158+
/// invalid config (a replica-set name outside [A-Za-z0-9_-];
159+
/// MONGO_INITDB_ROOT_* env set through with_env) before any daemon
160+
/// contact.
142161
GenericImage to_generic() const;
143162

144163
/// Create and start the container, initiate the replica set, and wait

include/testcontainers/modules/MySQLContainer.hpp

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
#include <string>
88
#include <string_view>
99
#include <utility>
10+
#include <vector>
1011

1112
#include "testcontainers/Container.hpp"
1213
#include "testcontainers/GenericImage.hpp"
@@ -113,6 +114,15 @@ class MySQLContainer {
113114
return *this;
114115
}
115116

117+
/// Append several server options at once — the batch twin of
118+
/// `with_command_arg`. Repeatable; calls accumulate in order.
119+
MySQLContainer& with_command_args(std::vector<std::string> args) {
120+
for (std::string& arg : args) {
121+
opts_.command_args.push_back(std::move(arg));
122+
}
123+
return *this;
124+
}
125+
116126
/// Copy a configuration file into /etc/mysql/conf.d, which the image's
117127
/// my.cnf includes. The file name must end in ".cnf" (the include glob;
118128
/// anything else throws here). Copied with mode 0644 — the server

0 commit comments

Comments
 (0)