Core test suite has high failure/error rate on BoxLang (35 failures, 125 errors, 3526 passing)

[zainforbjs]

Describe the bug
When running the Wheels core test suite on BoxLang, a significant number of tests are not passing. The current results are:

3526 passing
35 failing
125 errors

This indicates partial compatibility, but also a substantial number of issues affecting core functionality under BoxLang.

To Reproduce
Steps to reproduce the behavior:

1. Run the Wheels core test suite using BoxLang.
2. Execute full test run.
3. Observe test summary output.

Expected behavior
All core tests should pass successfully on BoxLang, or failures/errors should be significantly reduced with clear compatibility alignment.

Screenshots

Desktop:

• OS: Any
• CFML Engine: Boxlang

[wheels-bot]

Wheels Bot — Triage

Classified as other (meta-tracking issue / aggregate compatibility report).

This issue summarizes the current BoxLang test result counts (35 failing, 125 errors, 3526 passing) but does not isolate a specific failing test, error message, stack trace, or reproducible code path. It describes a state of affairs across ~160 distinct failures rather than a single defect that can be fixed in one PR.

To enable downstream bot triage and fix workflows, this should be broken down into individual issues — one per failing test or per distinct error signature — each with the test name, the assertion that failed, and (where available) the error/stack trace. Aggregate reports like this need a human maintainer to either decompose into discrete issues or set scope/priority for a broader BoxLang compatibility initiative.

No further bot action will be taken on this issue.

<!-- wheels-bot:triage:2649 -->
<!-- wheels-bot:triage-class:other -->

[bpamiri]

Wheels Bot — Fix on hold for human review

The proposed fix touches a sensitive area (cross-engine BoxLang compatibility, aggregate scope) and the bot's safety net (/propose-fix Step 4) requires a human in the loop before any code is written. Three independent reasons converge on this outcome:

1. Triage class is other, not bug or framework-design. The bot's earlier triage correctly identified this as a meta/aggregate tracking report rather than a single defect. The propose-fix workflow only auto-fires on bug / framework-design classes — not aggregate compatibility reports.
2. Cross-engine concern. Even an individual BoxLang fix must clear regression checks on Lucee 6, Lucee 7, Adobe 2023, and Adobe 2025. The propose-fix safety net is explicitly tripped by anything that "needs different code paths for Lucee/Adobe/BoxLang." That is the entire nature of this issue.
3. The cited counts cannot currently be verified from CI. The compat-matrix BoxLang job has been failing at container boot — CF engine not ready after 60 attempts — on the last three consecutive scheduled runs (2026-05-03, 2026-05-10, 2026-05-12). That is a separate CI-infrastructure problem (BoxLang docker image / startup), and it means the screenshot's 3526 / 35 / 125 numbers reflect the reporter's local run, not a current CI baseline.

Recommended path forward (human-driven)

• Restore the BoxLang job in compat-matrix first. Until BoxLang boots in CI, every "fix one failure" PR is flying blind. Investigate the CF engine not ready symptom — likely a BoxLang container readiness probe or image-version mismatch in compose.yml / compat-matrix.yml.
• Re-baseline the failure counts after merging BoxLang: Internal Wheels routes fail with Function [$blockInProduction] not found #2646 and fix(dispatch): preserve receiver context in BoxLang invokeMethod (#2646) #2648. Two BoxLang-specific fixes landed since the screenshot was captured ($blockInProduction not found, and invokeMethod receiver context). The current 35 / 125 numbers are probably already lower.
• Decompose remaining failures into individual issues. Each issue should carry: the spec file path, the failing assertion, the BoxLang error / stack trace, and (where possible) a minimal reproduction. Those discrete issues are what the bot's triage → propose-fix pipeline is shaped for.
• Consider whether BoxLang stays in the hard-gated set or moves to SOFT_FAIL_DBS-equivalent for engines. That's a release-gating policy call, not a code change.

No bot fix PR will be opened from this issue in its current form.

[bpamiri]

Decomposition complete — 7 child issues filed

Following the fix-held comment above and after #2657 improved compat-matrix diagnostics + #2648 fixed the BoxLang boot regression that was blocking the matrix from running BoxLang tests at all, I pulled the test-result artifacts from matrix run #25837380625 (2026-05-13) and clustered the ~700 BoxLang errors by normalized exception class. The result: 7 distinct root causes, not ~160.

Methodology

For each engine × DB, the matrix uploads a JSON artifact (test-results-<engine>) with full bundleStats → suiteStats → specStats hierarchy. Per failing/erroring spec I captured the bundle, the spec path, the status, and a normalized failMessage (paths/line-numbers/hashes stripped). Grouping by normalized message reveals that one signature alone — Error getting method [getStatus] for class [ortus.boxlang.servlet.BoxPageContext] — accounts for 600 of ~700 BoxLang errors (86%).

Tier 1 — BoxLang framework bug, headline impact

• #2659 — getStatusCode() throws on BoxLang because BoxLangAdapter.getResponse() returns GetPageContext() (which lacks getStatus()). 600 errors. 5-line fix.

Tier 2 — cross-engine bugs (affect BoxLang + Lucee)

• #2660 — Photo model rejects file uploads as array on non-SQLite DBs. 36 errors total (16 BoxLang + 20 Lucee 6). Needs investigation.
• #2661 — migrator.addColumnOptionsSpec DEFAULT-clause substring assertions fail on CockroachDB + Postgres on both engines. 17 errors. Pure test fix.
• #2662 — bulkOperationsSpec uses toBeTypeOf("component") which doesn't accept BoxLang-classed instances. 12 errors. Pure test fix.

Tier 3 — DB-environment and test-cleanup (not framework)

• #2663 — Oracle datasource broken: invalid URL on Lucee 6, missing DBMS_LOCK package, constraint cleanup. 130+ errors on BoxLang, 0-tests-run on Lucee 6. CI-infra work.
• #2664 — Migrator FK constraint cleanup is non-idempotent on MySQL/H2/SQL Server. Test cleanup fix.
• #2665 — lockingSpec doesn't guard against engines/DBs that lack advisory locks (H2) or require explicit transactions (SQL Server). Test fix.

Suggested landing order

1. #2659 first — lowest effort, highest impact, drops BoxLang from ~125 errors/DB to ~25 errors/DB. Makes everything downstream actually readable.
2. #2661, #2662 in parallel — both pure test fixes, no framework changes, safe to ship without cross-engine analysis.
3. #2660 when someone has a half-day for investigation.
4. #2663, #2664, #2665 as a separate workstream — these aren't BoxLang-specific and have been failing on Lucee for at least a month, just masked by continue-on-error: true at the workflow level and the empty SOFT_FAIL_DBS.

Numbers projection

Engine	Today	After #2659 alone	After #2659–#2662
BoxLang	~30 fail + ~700 err	~30 fail + ~100 err	~5 fail + ~50 err
Lucee 6	9 fail + 28 err	unchanged	reduced (#2660, #2661 also help here)

After all seven land, BoxLang should be roughly at parity with Lucee 6 on pass rate. That's the milestone this issue should close on.

Recommend keeping this issue open

…as the rollup tracker for the seven child issues. Close once all are resolved and a fresh matrix run confirms BoxLang test counts are at Lucee-parity.