Skip to content

Windows toolchain test suite got noticeably slower on main vs release/6.4.x — mostly in lldb-api Swift tests #13736

Description

@shahmishal

Comparing two Windows toolchain CI runs — release/6.4.x vs main — the lit/lldb-api test suites on main are noticeably slower, with the lldb retry pass roughly doubling in wall time. A cluster of resilience and Swift/C++ backward-interop-stepping tests that weren't previously among the slowest 20 tests now dominate that list, which points at a shared root cause rather than N unrelated slow tests.

Two Windows toolchain CI builds from the same day:

Suite-level timing

Suite release/6.4.x main Change
Swift lit (~13k tests, compiler/stdlib) 596.36s 624.36s +4.7%
lldb-api Swift (initial pass) 334.13s 430.16s +28.7%
lldb-api Swift (retry pass) 196.56s 391.17s +99.1%
Small suite (~32 tests) 36.96s 47.82s +29.4%

The lldb-api Swift suite is where this is really showing up — the retry pass nearly doubled.

Individual tests, lldb-api suite — largest regressions

Straight from each build's Slowest Tests: block:

release/6.4.x:  53.18s: lldb-api :: lang/swift/expression/import_search_paths/TestSwiftImportSearchPaths.py
main:           105.38s: lldb-api :: lang/swift/expression/import_search_paths/TestSwiftImportSearchPaths.py
release/6.4.x:  58.89s: lldb-api :: lang/swift/parseable_interfaces/shared/TestSwiftInterfaceNoDebugInfo_part01.py
main:           109.26s: lldb-api :: lang/swift/parseable_interfaces/shared/TestSwiftInterfaceNoDebugInfo_part01.py
release/6.4.x:  65.98s: lldb-api :: lang/swift/implementation_only_imports/library_indirect/TestLibraryIndirect.py
main:           100.47s: lldb-api :: lang/swift/implementation_only_imports/library_indirect/TestLibraryIndirect.py

Full set of matched regressions:

Test release/6.4.x main Change
TestSwiftImportSearchPaths.py 53.18s 105.38s +98%
TestSwiftInterfaceNoDebugInfo_part01.py 58.89s 109.26s +86%
TestSwiftInterfaceNoDebugInfo_part02.py 59.19s 108.22s +83%
TestLibraryIndirect.py 65.98s 100.47s +52%
TestArchetypeInConditionalBreakpoint.py 63.78s 94.93s +49%
TestSwiftBackwardInteropSteppingClass_part02.py 65.72s 96.41s +47%
TestSwiftPrintObjectPointerAndTypeName_part01.py 68.90s 101.59s +47%
TestMarkerProtocolExistential.py 76.77s 111.58s +45%
TestSwiftPrintObjectPointerAndTypeName_part03.py 72.58s 102.37s +41%
TestSwiftTaskGroupSynthetic.py 69.88s 97.77s +40%
TestSwiftPersistentResult.py 70.20s 94.52s +35%
TestSwiftClosureVarNotCaptured.py 90.37s 121.13s +34%

The real signal: two whole test clusters jumped into the slow list together

These didn't even make the release/6.4.x top-20 slowest, but now sit at 90–110s each on main:

main: 107.17s: lldb-api :: lang/swift/resilience_superclass_other_mod/TestSwiftResilienceSuperclassOtherMod.py
main: 103.77s: lldb-api :: lang/swift/resilience_other_module/TestSwiftResilienceOtherModule.py
main:  95.22s: lldb-api :: lang/swift/resilience_superclass/TestSwiftResilienceSuperclass.py
main:  92.09s: lldb-api :: lang/swift/resilience_superclass_mod/TestSwiftResilienceSuperclassMod.py
main:  97.35s: lldb-api :: .../TestSwiftBackwardInteropSteppingClass_part03.py
main:  97.29s: lldb-api :: .../TestSwiftBackwardInteropSteppingStruct_part01.py
main:  96.92s: lldb-api :: .../TestSwiftBackwardInteropSteppingStruct_part02.py
main:  96.41s: lldb-api :: .../TestSwiftBackwardInteropSteppingClass_part02.py

Every cross-module resilience test and every Swift/C++ backward-interop stepping test moved together, from "not slow" to "dominates the slow list." That's a stronger lead than a pile of unrelated individual regressions — it suggests something shared across debugger attach/step/symbolication paths for resilient or interop-stepping scenarios, rather than N independent test-level issues.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions