Skip to content

docs(web/guides): correct 20 audited testing-guide claims (batch 2) - #3099

Merged
bpamiri merged 1 commit into
developfrom
peter/docs-audit2-testing-guides
Jun 12, 2026
Merged

docs(web/guides): correct 20 audited testing-guide claims (batch 2)#3099
bpamiri merged 1 commit into
developfrom
peter/docs-audit2-testing-guides

Conversation

@bpamiri

@bpamiri bpamiri commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Batch 2 of the guide behavioral-audit docs wave — all 20 docs-affected claims for work item p1-15-testing (testing/*.mdx). Every correction below was live-verified by the audit (raw evidence in the p1 batch-2 verifier output); broken-but-unfixed behavior is documented as-is with the tracking issue cited.

index.mdx

  • IDX-10 — full engine x DB matrix runs on a weekly cron + manual dispatch, not "on every PR"; engine list corrected to Lucee 6/7, Adobe 2023/2025, BoxLang (compat-matrix.yml:1-23). Same fix applied to the CI-integration paragraph lower on the page for consistency with ci-integration.mdx.
  • IDX-11wheels test --ci is accepted for forward-compat but currently changes nothing (ciMode parsed, never used; verified byte-identical output and exit codes). Browser gating is the server env (WHEELS_CI/WHEELS_BROWSER_CI_ENABLE), not this flag.
  • IDX-07 — runner-URL example used the wrong directory shape for the core endpoint; rewritten around /wheels/app/tests?directory=tests.specs.models with the core form (directory=wheels.tests.specs.model) noted. Unrecognized directory values silently run the FULL suite — cited #3083.
  • IDX-08 — populate is opt-out (?populate=false), defaulting true when absent (runner.cfm:113); page previously said opt-in via ?populate=true, contradicting its sibling fixtures-and-test-data.mdx.
  • Setup-files table: tableName()table() (the setter; tableName() is a getter no-op, docs+model: guides and CLAUDE.md use non-existent tableName("x") setter — silent no-op, models fall back to the convention table (real setter is table()) #3079 finding).

model-tests.mdx

  • MOD-03.persisted() does not exist (throws); the API is isPersisted() / isNew() (miscellaneous.cfc:244/228).

view-and-form-tests.mdx

  • VFT-01assertSee/assertDontSee/assertSeeInOrder are case-insensitive (FindNoCase, verified live); page said case-sensitive.
  • VFT-06 — the encoded apostrophe is ' (hex) on both Lucee 7 and Adobe 2023, so the assertSee("can't be empty") example failed everywhere probed. Example rewritten to the split-assertion pattern (primary recommendation) with the corrected ' literal documented; trailing bullet fixed too.

functional-tests.mdx

  • FUN-03 — both engines emit text/html;charset=UTF-8 (no space) and assertHeader is exact-match; the "text/html; charset=UTF-8" example failed as written. Fixed the literal + added an exact-match note.
  • Same '' correction in the split-assertion explanation.

fixtures-and-test-data.mdx

  • FIX-03application.wo.execute(...) does not exist ("No matching function [EXECUTE] found", verified live); execute() lives only on wheels.migrator.Migration. The minimal populate.cfm example would crash at boot. Rewritten to native queryExecute(..., {datasource: application.wheels.dataSourceName}) via a small runSql() helper; scope-gotcha section corrected (application.wo.model() still works and is kept).
  • tableName("test_posts")table("test_posts") in the TestPost example (getter no-op; all framework test models use table()). This also covers the p1-13 cross-file sketch entry for this file.
  • Added an isolation caution: the runner mutates the live application.wheels (settings/path swaps leak into normal requests) — cites #3025.

browser-tests.mdx

  • BRW-04browserDescribe()'s aroundEach auto-skips when browserTestSkipped is set (BrowserTest.cfc:140-152); the hand-written if (this.browserTestSkipped) return; guard is only needed inside plain describe(). Prose updated and the guard line stripped from all example specs.
  • BRW-08 — dialog error type is Wheels.BrowserDialogNotSupported, not Wheels.DialogSupportMissing.
  • BRW-15 — form helpers emit a dashed id="post-title", not camelCase id="postTitle" (matches sibling page).
  • BRW-16 — no beforeAll fixture-route re-include exists; replaced with a warning that route-clearing specs must restore the route table.
  • BRW-14 — stale cfwheels/cfwheels CLAUDE.md link → wheels-dev/wheels (branding rule).
  • loginAs: noted app-side mounting requires set(loadBrowserTestFixtures=true) (default false) in testing/development.

running-tests-locally.mdx

  • RUN-03--core auto-detect is removed (Module.cfc:738 comment; live wheels test at the framework repo root runs the app suite). Aside rewritten: pass --core explicitly; --core flag description aligned.
  • RUN-06 — single-spec-file filtering via directory= matches 0 bundles; the working form is testBundles=wheels.tests.specs.model.callbacksSpec (verified live; also replaced the nonexistent postSpec). Filter table updated; added the directory allowlist silent-fallback note citing #3083.
  • RUN-10compose.yml ships mysql, postgres, sqlserver, cockroachdb, oracle services; there is no mssql or h2 service (H2/SQLite are embedded). The docker-compose.db-*.yml files are demo-app dev overlays, now noted as such. "You'll learn" DB list aligned.
  • RUN-11 — the soft-fail CI database is Oracle (SOFT_FAIL_DBS="oracle"), not CockroachDB.
  • RUN-12JAVA_HOME preflight qualified: the brew wrapper exports JAVA_HOME itself, so the refusal can't trigger on brew installs (prereqs + failure-modes sections).
  • --ci example comment aligned with IDX-11 (non-zero exit applies to every run).

ci-integration.mdx

  • CI-01 — reporter table was stale: on released 4.0.3, --reporter=tap emits TAP v13 and --reporter=json emits the raw JSON document (verified live; Module.cfc reporter dispatch). Table rewritten; --ci row kept with the universal non-zero-exit note.
  • Pinned LuCLI bumped v0.3.3 → v0.3.7 to match pr.yml's LUCLI_VERSION.
  • Browser-gating paragraph aligned with BRW-04 (auto-skip via browserDescribe, no per-it guard).

Verification

pnpm verify:docs over all 8 changed files: 38 tagged blocks, 38 passed, 0 failed (exit 0).

🤖 Generated with Claude Code

Fixes the docs-wrong findings from the p1-15-testing behavioral audit:
runner directory/testBundles filtering, populate opt-out default,
weekly compat matrix, --ci/--reporter behavior, isPersisted(),
assertSee case-insensitivity, &##x27; entity, exact Content-Type match,
populate.cfm queryExecute rewrite, table() setter, browserDescribe
auto-skip, dialog error type, dashed form ids, fixture-route warning,
core-mode default, compose DB services, Oracle soft-fail, JAVA_HOME
qualifier. Cites ##3025 and ##3083 for open runner issues.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>

@wheels-bot wheels-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wheels Bot — Reviewer

TL;DR: This PR corrects 20 audited factual claims across the eight v4-0-0 testing-guide pages (batch 2 of the behavioral audit). I verified every load-bearing claim in the diff against the framework source and CI config, and they all check out — several at the exact line numbers cited in the PR body. No correctness, cross-engine, or security findings; the commit message conforms to commitlint. One minor docs nit below (an H2 omission in the index page's matrix database list). Verdict: comment — nothing blocking.

Verification audit (claims checked against source)

Claim Evidence
Full matrix is weekly cron + manual dispatch; engines Lucee 6/7, Adobe 2023/2025, BoxLang .github/workflows/compat-matrix.yml:5-23 (cron: '0 2 * * 0', workflow_dispatch, matrix list)
Oracle (not CockroachDB) is the soft-fail DB compat-matrix.yml:293SOFT_FAIL_DBS="oracle"
--ci is parsed but currently changes nothing cli/lucli/Module.cfc:731,755,4735ciMode is captured and passed into runTests() but never read in its body
--reporter=tap emits TAP v13, --reporter=json emits the raw document Module.cfc:4816-4824 reporter dispatch; TAP v13 emitters at Module.cfc:4869+
--core auto-detect removed; app suite is the default Module.cfc:738-745 — comment documents exactly the removal and the silent-wrong-suite rationale the doc now states
Populate is opt-out, default true vendor/wheels/tests/runner.cfm:112StructKeyExists(url, "populate") ? url.populate : true
Unrecognized directory silently runs the full suite (#3083) runner.cfm:126-135 — allowlist regex falls through to the default wheels.tests.specs scope
testBundles= filters to a single spec file vendored TestBox reads url.testBundles (vendor/wheels/wheelstest/system/TestBox.cfc:411-412); the runner itself only passes directory=
.persisted() doesn't exist; isPersisted()/isNew() do vendor/wheels/model/miscellaneous.cfc:244 / :228 — exact lines cited in the PR body
table() is the setter, tableName() the getter miscellaneous.cfc:35 (table) vs :177 (tableName)
assertSee/assertDontSee/assertSeeInOrder are case-insensitive vendor/wheels/wheelstest/TestClient.cfc:251,264,280 — all FindNoCase
assertHeader is exact-match TestClient.cfc:374hdrs[arguments.name] != arguments.value
browserDescribe() auto-skips via aroundEach when browserTestSkipped is set vendor/wheels/wheelstest/BrowserTest.cfc:138-152 — the skip path even has a comment saying hand-written guards are unnecessary
Dialog error type is Wheels.BrowserDialogNotSupported vendor/wheels/wheelstest/BrowserClient.cfc:950,970; no DialogSupportMissing anywhere in the tree
No automatic fixture-route re-include in beforeAll (BRW-16) BrowserTest.cfc:61+ — no route include exists; the old claim was indeed wrong
Form helpers emit dashed ids (post-title) vendor/wheels/view/miscellaneous.cfc:596-617 ($tagId joins with -); the doc comment at :626 even calls ##post-title the historical selector
loadBrowserTestFixtures defaults false, gated to testing/development vendor/wheels/events/init/views.cfm:30; vendor/wheels/Global.cfc:1526-1531
execute() exists only on migration CFCs only definition under vendor/wheels/ is vendor/wheels/migrator/Migration.cfc; both real populate files use native <cfquery datasource="#application.wheels.dataSourceName#"> (tests/populate.cfm:63-95), matching the rewritten example
compose ships mysql, postgres, sqlserver, cockroachdb, oracle (no mssql/h2 service); the docker-compose.db-*.yml files are dev overlays compose.yml service list; docker-compose.db-h2.yml / db-mysql.yml / db-postgres.yml exist at the repo root
Pinned LuCLI 0.3.7 matches CI .github/workflows/pr.yml:33LUCLI_VERSION: "0.3.7"

The commit (05f57a869) is a valid conventional commit (docs(web/guides): …, 66-char header), DCO-signed matching the author identity. A changelog fragment is not required for a docs-type change. The PR body reports pnpm verify:docs green over all 8 files (38/38 tagged blocks); the script exists at web/sites/guides/package.json:13.

Docs

  • web/sites/guides/src/content/docs/v4-0-0/testing/index.mdx:70 — the corrected matrix sentence now reads "…and every supported database (MySQL, PostgreSQL, SQL Server, SQLite, CockroachDB, Oracle)", dropping H2 from the list (the pre-PR text included it). But the weekly matrix does run H2 on the Lucee legs — compat-matrix.yml:41-44 appends h2 to DATABASES for Lucee engines — and this same PR's running-tests-locally.mdx "You'll learn" list correctly names H2 as a supported target. Suggest keeping the enumeration complete with its caveat, e.g. "(MySQL, PostgreSQL, SQL Server, SQLite, CockroachDB, Oracle, plus H2 on Lucee)". Minor — does not block.

Everything else in the diff is accurate, internally consistent across the sibling pages, and properly cites the open tracking issues (#3025, #3083) for broken-but-unfixed behavior instead of papering over it.

@bpamiri
bpamiri merged commit 5e6adca into develop Jun 12, 2026
14 checks passed
@bpamiri
bpamiri deleted the peter/docs-audit2-testing-guides branch June 12, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant