fix(dream-cli): schema-driven secret masking + macOS Bash 4 validation#994
fix(dream-cli): schema-driven secret masking + macOS Bash 4 validation#994yasinBursali wants to merge 5 commits intoLight-Heart-Labs:mainfrom
Conversation
|
Audit follow-up: needs revision for no- Schema-driven secret masking is useful, but the CLI only learns the schema secret flags through |
The previous implementation used a narrow regex anchored to `=`:
grep -qiE "(secret|pass|token|key)="
which required the keyword to appear immediately before `=`. It caught
`_SECRET=`, `_KEY=`, `_TOKEN=`, but missed `_PASSWORD=` (because `pass`
is followed by `word`, not `=`), `_SALT=` (not in the alternation), and
failed-open for any future secret name not ending in a matching suffix.
Real env keys that leaked in plaintext from `dream config show`:
- LANGFUSE_DB_PASSWORD
- LANGFUSE_SALT
- LANGFUSE_CLICKHOUSE_PASSWORD
- LANGFUSE_REDIS_PASSWORD
- LANGFUSE_INIT_USER_PASSWORD
- OPENCODE_SERVER_PASSWORD
Fix: drive masking from `.env.schema.json` — the canonical source of
truth that already marks the above keys `"secret": true`. Keys listed
there are masked; everything else is shown raw. When the schema or jq
is unavailable (older installs, minimal environments), fall back to a
case-insensitive substring match on secret/password/pass/token/key/salt
so the default behaviour errs on the side of masking rather than
leaking.
The schema already marks every known-leaking key above as
`"secret": true`; no schema change is required. A small number of
operational keys (TOKEN_SPY_PORT, TOKEN_SPY_URL, TARGET_API_KEY,
ANTHROPIC_API_KEY, OPENAI_API_KEY, TOGETHER_API_KEY, LIVEKIT_API_KEY,
LANGFUSE_PROJECT_PUBLIC_KEY) are not marked secret in the schema —
most are legitimately public (ports, URLs, publishable keys) or are
user-supplied. Left for a follow-up review; the regex fallback covers
them defensively if the schema ever goes missing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`dream config validate` crashed on every fresh macOS install. The cause was a shebang mismatch: `scripts/validate-env.sh` is `#!/bin/bash`, and macOS ships /bin/bash 3.2 (licensing-frozen) which has no `declare -A`. The script uses associative arrays at lines 75, 76, 77, and 176, so it aborted immediately with `declare: -A: invalid option` before doing any work. Two-part fix: - dream-cli invokes `validate-env.sh` through "$BASH" (the currently- running shell, guaranteed Bash 4+ by the version check at the top of dream-cli itself). This works regardless of the target script's shebang. Linux and WSL2 are unaffected (/bin/bash is already 4+ there, so the invocation is a no-op change). - `validate-env.sh` now starts with a Bash 4+ guard that prints a helpful "install a modern Bash" message and exits 1 before any `declare -A` is reached. This protects users running the script standalone (e.g. from the installer or CI), not just the dream-cli path. Wording matches the existing guard at the top of dream-cli. `scripts/validate-manifests.sh` does not use associative arrays, so it is invoked unchanged. Other `declare -A` users audited but not touched in this PR (fix is the same pattern — either call through "$BASH" or add a top-of-script Bash 4+ guard, depending on how each is invoked): scripts/pre-download.sh scripts/dream-test-functional.sh lib/service-registry.sh lib/progress.sh installers/phases/03-features.sh Follow-up audit can cover each of these in scope. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…asking Two small follow-up fixes flagged by the tester for the config subcommand hardening: 1. tests/test-validate-env.sh invoked scripts/validate-env.sh by raw path, so shebang resolution picked /bin/bash = Bash 3.2 on macOS. The new Bash-4+ guard in validate-env.sh correctly blocks this invocation, which regressed the previously-passing macOS test cases (missing-.env and missing-schema, both expected exit 3 via early file-check) along with the Bash-4-dependent ones. Invoke the script through "$BASH" in the test harness, mirroring the dream-cli invocation path. One helper variable (VALIDATE_ENV_BASH), five invocation sites updated. Linux CI was unaffected; this fixes the macOS-only harness break. 2. Added "*bearer*" to the fallback substring match in cmd_config show's _cmd_config_is_secret, so FOO_BEARER=... does not leak when the schema is unavailable. Schema-driven path already covers the real DreamServer secrets; this closes a theoretical gap in the defensive fallback. Did not add *auth* (would false-positive on AUTHOR_NAME- style keys) or other substrings — bearer is the only real gap the tester surfaced. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The schema-authoritative check in the previous commit leaks keys that are present in .env.schema.json with secret: false, which today miscovers five real upstream-provider credentials (TARGET_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, TOGETHER_API_KEY, LIVEKIT_API_KEY). A malformed schema (valid JSON but empty jq output) has the same silent-leak failure mode. After a schema-miss under _schema_loaded=1, fall through to the keyword substring match instead of returning "not secret" immediately. Schema still defines what IS definitely a secret; the keyword pass adds defense in depth for schema gaps. Over-masking of LANGFUSE_PROJECT_PUBLIC_KEY, TOKEN_SPY_PORT, and TOKEN_SPY_URL is acceptable — 'show' should default to over-masking, and raw values remain available via cat .env. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…NIT_USER_EMAIL Maintainer audit on PR Light-Heart-Labs#994 (Lightheartdevs, 2026-04-28) flagged that schema-driven secret detection in `_cmd_config_load_secret_schema` ONLY fires when `jq` is on PATH. In Git Bash on Windows, `jq` is not installed by default, so the helper silently leaves `_cmd_config_schema_loaded=0` and `dream config show` falls through to the keyword regex `*secret*|*password*|*pass*|*token*|*key*|*salt*|*bearer*` — which doesn't cover the user/email-suffixed names recently flagged as secret in `.env.schema.json`. `N8N_USER` and `LANGFUSE_INIT_USER_EMAIL` print in clear. Two-layer fix: 1. Add a Python3 fallback to `_cmd_config_load_secret_schema`. If `jq` is absent, parse `.env.schema.json` via a heredoc'd `python3` and extract `secret == True` keys. The installer guarantees Python 3, and Git Bash on Windows usually has it via `winget install Python`. Schema-driven masking now works without jq. 2. Extend the keyword fallback regex to include `*user*|*email*` so the truly-no-jq-no-python case still masks the new schema secrets. Light over-mask risk on operational keys like USER_HOME but `cat .env` remains the unmasked escape hatch — `dream config show` defaults to over-mask on purpose. New regression `tests/test-dream-config-secret-mask.sh` (7 cases, wired into `make test`) exercises all three PATH conditions: - jq+python3 both present → schema-driven jq path - jq absent, python3 present → python fallback - neither present → keyword regex fallback Each case asserts `N8N_USER=***` and `LANGFUSE_INIT_USER_EMAIL=***` appear in stdout AND that the actual sensitive values do NOT leak. A sanity case verifies non-secret `DREAM_VERSION` is shown in clear. The PATH conditions are simulated by symlinking selected binaries into a tempdir, then running dream-cli with `PATH="$tempdir"`. shellcheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
9236994 to
2b49c8f
Compare
|
Pushed audit follow-up ( Closes the jq-less Git Bash leak with two independent layers:
New
Local CG APPROVED (no warnings). Sabotage-checked: with both fallbacks reverted, 4 cases fail; with only the keyword extension reverted, 2 cases fail. Each layer is independently load-bearing. |
The `_docker_cmd_arr: returns sudo docker when DOCKER_CMD is sudo docker` test in `tests/bats-tests/docker-phase.bats` was failing in every CI run on `upstream/main`, blocking the integration-smoke job across all 12 currently-open PRs. Root cause: the test stub at line 94 (mirroring the real function at `installers/phases/05-docker.sh:29`) does `echo "sudo" "docker"`, which emits a single space-joined line `"sudo docker\n"`. The assertion at line 100 expected `$'sudo\ndocker'` — two newline-separated lines. That output never matched; the test had been silently red on main. The real function's consumer at `installers/phases/05-docker.sh:36` correctly word-splits the single-line output via `local -a cmd=($(_docker_cmd_arr))`, so the function works in production. The bug was assertion-only. Fix: change `assert_output $'sudo\ndocker'` to `assert_output 'sudo docker'`. Inline comment captures the single-line + word-splitting contract so a future contributor doesn't "fix" it back to a two-line assertion. Verified: `tests/bats/bats-core/bin/bats tests/bats-tests/docker-phase.bats` — case 2 now passes (was the only one failing in CI; local case 1 fail is environmental — no Docker daemon on dev machine — and unrelated). Unblocks `integration-smoke` for the 12 open PRs (#974, #994, #998, #1000, #1015-1057 etc.) that all base on upstream/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lightheartdevs
left a comment
There was a problem hiding this comment.
This fixes the original no-jq secret leak finding: the branch now has a Python fallback for schema secret keys, and if neither jq nor python3 is available it over-masks user/email keys so N8N_USER and LANGFUSE_INIT_USER_EMAIL do not print in the clear.
Local proof:
tests/test-dream-config-secret-mask.shpassed 7/7 locally.bash -npassed for the touched shell scripts.
Still not merge-ready because the branch is stale/dirty against current main:
- A local merge simulation conflicts in
dream-server/Makefileanddream-server/dream-cli. - The test emits some Git Bash symlink cleanup noise locally after reporting success; worth cleaning if it also shows up in CI logs, but the blocking item is the rebase/conflict resolution.
Please rebase onto current main. After that, this looks like a useful security hardening PR.
What
Five related hardenings to
dream configand its helpers:dream config show— replace the narrow keyword regex (which missed_PASSWORD,_SALT,_PASSsuffixed fields) with.env.schema.json-driven detection.dream config validatenow invokesvalidate-env.shthrough"$BASH", and the target script adds its own Bash-4 guard. Fixes a macOS-only crash under/bin/bash3.2._schema_loaded=1, fall through to the keyword match instead of returning "not secret." Covers schema gaps and malformed schemas.dream preset diff—_cmd_config_is_secretwas nested insidecmd_config show, sodream preset diffstill used the original narrow regex and leakedN8N_PASS,LANGFUSE_SALT, and admin email fields in plaintext. Helper is now file-scope; both commands use it.N8N_USERandLANGFUSE_INIT_USER_EMAILare also markedsecret: truein.env.schema.json.python3fallback in_cmd_config_load_secret_schemawhenjqis absent (Git Bash on Windows is the canonical case). Plus*user*|*email*keyword extension as belt-and-suspenders for the no-jq-no-python3 case.Why
The narrow regex
(SECRET|PASS|KEY|TOKEN)=missed suffix-form names likeLANGFUSE_DB_PASSWORD,LANGFUSE_SALT,OPENCODE_SERVER_PASSWORD, etc.dream config validatecrashed on every fresh macOS install becausevalidate-env.shusesdeclare -A(Bash 4+ only) but was invoked through/bin/bash3.2. Anddream preset diffwas a forgotten sibling code path.Maintainer audit (2026-04-28) flagged that the schema-driven path only fires when
jqis on PATH. In Git Bash on Windows (no jq by default), the helper silently set_cmd_config_schema_loaded=0anddream config showfell through to the keyword regex, which didn't cover*user*|*email*-suffixed names — soN8N_USERandLANGFUSE_INIT_USER_EMAILprinted in clear.How
Five commits, each self-contained:
d4d754ae— schema-driven masking incmd_config show.cc6c40ac— Bash 4+ gate +"$BASH"invocation path.ce37c119— fallback-masking hardening (*bearer*added to keyword fallback) + test-harness fix for macOS.9cee345d— schema-miss falls through to keywords + helper promoted to file-scope +dream preset diffnow uses the helper +N8N_USER/LANGFUSE_INIT_USER_EMAILmarkedsecret: true.2b49c8ff— audit follow-up: python3 fallback in_cmd_config_load_secret_schema,*user*|*email*keyword extension, regression test exercising all three PATH conditions.Audit follow-up details (commit
2b49c8ff)_cmd_config_load_secret_schemanow follows this order:jqavailable → use jq (existing fast path).python3available → parse.env.schema.jsonvia heredoc'dpython3and extract keys wherespec.get("secret") is True. Sets_cmd_config_schema_loaded=1._cmd_config_schema_loaded=0._cmd_config_is_secretfalls through to the keyword regex which now also includes*user*|*email*to cover the new schema secrets.Verified: jq and python3 produce identical 36-key outputs on the live
.env.schema.json(CG diff'd both, no delta).Testing
make lint,make test,shellcheckall green.tests/test-dream-config-secret-mask.sh(7 cases, wired intomake test):DREAM_VERSION=2.0.0-testshown in clear (no over-mask)KEY=***appears in stdout) AND negative (literal sensitive value does NOT leak). Sentinel values:actual-admin-username(N8N_USER),admin@example.test(LANGFUSE_INIT_USER_EMAIL).dream config show/dream config validate/dream preset difftesting still passes.Review
Local CG APPROVED (no warnings). Notes from review (informational only):
null.env.schema.jsonwould triggerAttributeErrorin the python heredoc. Suppressed by2>/dev/nulland degrades to keyword regex fallback (functionally safe, just an ungraceful traceback). Could widen theexceptclause but project style discourages broad lists.PATH=...debugging; standardmake testis unaffected).*user*|*email*extension is zero in current.env.schema.json— all 4 user/email-bearing keys are alreadysecret: true. Hypothetical future operational keys (e.g.,USER_HOME) would be over-masked butcat .envremains the unmasked escape hatch.Known Considerations
A small number of operational keys (
TOKEN_SPY_PORT,TOKEN_SPY_URL,LANGFUSE_PROJECT_PUBLIC_KEY) remain over-masked by the keyword fallback. This is intentional:dream config showshould default to over-masking; raw values remain accessible viacat .env.Platform Impact