ModelingToolkitBase initialization test errors on Julia 1.11 when @inferred now passes

[ChrisRackauckas-Claude]

BoundaryValueDiffEq master downstream CI is failing in the ModelingToolkitBase initialization tests on Julia 1.11 because the Julia < 1.12 branch uses @test_broken @inferred .... When inference succeeds, @inferred returns the expression value, so @test_broken receives a non-Boolean and reports an error instead of a broken test or a clear unexpected pass.

Observed in BoundaryValueDiffEq downstream CI:

• Repo/run: SciML/BoundaryValueDiffEq.jl, IntegrationTest run https://github.com/SciML/BoundaryValueDiffEq.jl/actions/runs/29209708744
• Job: ModelingToolkit.jl/All / Downstream Tests - All, job id 86695245419
• BoundaryValueDiffEq SHA: 4dfc3ef2872deace50764f53e8727cfbe7623d99
• ModelingToolkit checkout in that job: 7a1f575eace164fce6a9bd943289b54606e8a512
• Julia: 1.11.9

Relevant CI error:

Type-stability of `remake`: Error During Test at downstream/lib/ModelingToolkitBase/test/initializationsystem.jl:1750
  Expression evaluated to non-Boolean
  Expression: @inferred remake(prob; u0 = 2 .* prob.u0, p = prob.p)
  Value: SciMLBase.ODEProblem{...}

Type-stability of `remake`: Error During Test at downstream/lib/ModelingToolkitBase/test/initializationsystem.jl:1751
  Expression evaluated to non-Boolean
  Expression: @inferred solve(prob)
  Value: [1.0 1.0 1.0; 1.0 1.0 1.0]

ERROR: Some tests did not pass: 793 passed, 0 failed, 2 errored, 12 broken.

This same failure was already present in an older BoundaryValueDiffEq master IntegrationTest run on 2026-06-17:

• Run: 27681092307
• Job: 81868459288
• Summary: 779 passed, 0 failed, 2 errored, 12 broken
• Same Type-stability of remake errors at initializationsystem.jl:1745 and :1746

The relevant test lines were introduced by:

• d893f4205a56dc1bdfdb729661cdbd03c4ae6da7 (test: disable inference test on broken versions)
• File: lib/ModelingToolkitBase/test/initializationsystem.jl

That commit changed the non-1.12 path to:

@test_broken @inferred remake(prob; u0 = 2 .* prob.u0, p = prob.p)
@test_broken @inferred solve(prob)

Local macro reproducer of the test-structure problem on Julia 1.11:

using Test
@testset "macrocheck" begin
    @test_broken @inferred 1 + 1
end

Output:

ERROR: Some tests did not pass: 0 passed, 0 failed, 1 errored, 0 broken.
macrocheck: Error During Test at none:1
  Expression evaluated to non-Boolean
  Expression: @inferred 1 + 1
       Value: 2

And when inference is genuinely unstable, the same structure does report a broken test:

using Test
f(x) = x == 1 ? 1 : 1.0
@testset "macrocheck_unstable" begin
    @test_broken @inferred f(Ref(1)[])
end
# => 0 exit, 1 broken

So the downstream CI failure appears to mean the two inference checks now pass on Julia 1.11 in this environment, and the stale @test_broken @inferred wrapper converts that into an error.

Suggested fixes:

• If these inference checks are now expected to pass on Julia 1.11, enable the plain @inferred assertions for 1.11 as well.
• If this should remain an xfail-style test, make the @test_broken expression Boolean, e.g. a block that runs @inferred ... and returns true, so success is reported as a clear Unexpected Pass rather than Expression evaluated to non-Boolean.

Local notes from the BoundaryValueDiffEq investigation:

• Clean checkout: /home/crackauc/sandbox/tmp_20260708_121627_10236/bvd_mtk_master_ci_investigation
• BVD checkout SHA: 4dfc3ef2872deace50764f53e8727cfbe7623d99
• Current local MTK checkout SHA used for additional inspection: ea1c2d00727ec7af3bd4521ea7981eb6bc2782d9
• Downloaded CI log path: master_mtk_downstream_job_86695245419.log
• Older matching CI log path: logs/run_27681092307_job_81868459288.log
• Local GROUP=All downstream reproduction command with timeout 3600 timed out before completion while running downstream tests; it did not reach a clean final summary locally.

[ChrisRackauckas-Claude]

Additional local reproduction completed after filing:

Command run from /home/crackauc/sandbox/tmp_20260708_121627_10236/bvd_mtk_master_ci_investigation:

JULIA_DEPOT_PATH="$PWD/.julia_depot" GROUP=Initialization timeout 3600 \
  /home/crackauc/.juliaup/bin/julia +1.11 --startup-file=no --project=downstream --color=yes \
  -e 'using Pkg; Pkg.test(; coverage=false)' \
  > logs/reproduce_master_downstream_initialization.log 2>&1

Result: exit code 1, reproduces the same two errors locally.

Key lines from logs/reproduce_master_downstream_initialization.log:

Type-stability of `remake`: Error During Test at .../downstream/lib/ModelingToolkitBase/test/initializationsystem.jl:1750
  Expression: @inferred remake(prob; u0 = 2 .* prob.u0, p = prob.p)

Type-stability of `remake`: Error During Test at .../downstream/lib/ModelingToolkitBase/test/initializationsystem.jl:1751
  Expression: @inferred solve(prob)
       Value: [1.0 1.0 1.0; 1.0 1.0 1.0]

ERROR: Some tests did not pass: 793 passed, 0 failed, 2 errored, 12 broken.
Initialization | 793 2 12 807 26m21.3s
Type-stability of `remake` | 2 errors
ERROR: Package ModelingToolkit errored during testing

This was against BVD 4dfc3ef2872deace50764f53e8727cfbe7623d99 and local MTK ea1c2d00727ec7af3bd4521ea7981eb6bc2782d9 after Pkg.develop(PackageSpec(path=".")); Pkg.update() had developed the local BVD checkout into the downstream environment.