ci: restructure CI workflow to optimize SauceLabs usage and build times #2072
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Unit Test Results 137 files 137 suites 1m 1s ⏱️ Results for commit 3cbd0d9. ♻️ This comment has been updated with latest results. |
Artur-
changed the title
Artur-
force-pushed
the
one-workflow-concurrently
branch
from
3cbd0d9 to
890f70d
Compare
…ld times Completely restructure the validation workflow to separate concerns and respect SauceLabs concurrent session limits while maximizing parallelism. ## New Job Structure 1. **format-check** - Code style validation (unchanged) 2. **build** - Build entire project once, generate javadoc, cache artifacts 3. **unit-tests** - Run all unit tests using cached build (no SauceLabs) 4. **integration-tests** - Run only SauceLabs browser tests using cached build - Matrix: JUnit 4, 5, 6 variants - Sequential execution within workflow (max-parallel: 1) - Global concurrency control across all PRs ## Key Improvements **Build once, use everywhere:** - Build job runs `mvn clean install -DskipTests` once - Javadoc generated once in build job (not 3x in integration tests) - Artifacts cached with run-specific key: `github.run_id` - Both unit-tests and integration-tests restore from cache - Eliminates redundant builds (was building 3x, once per JUnit version) **Independent unit test execution:** - Runs `mvn test` separately from integration tests - No SauceLabs dependency (fast, cheap, can parallelize) - Separate test reports: surefire-reports (unit tests) - Added TestBench license setup for Pro feature validation **Optimized integration tests:** - Runs `mvn verify -Dsurefire.skip=true -pl <module> -am -P validation` - `-Dsurefire.skip=true` skips only unit tests (already ran in separate job) - Runs only failsafe integration tests (*IT.java files) - Uses full SauceLabs parallelism: `testsInParallel=5`, `forkCount=5` - Separate test reports: failsafe-reports (integration tests) - Removed javadoc generation (moved to build job) **SauceLabs concurrency control:** - Job-level concurrency group: `saucelabs-testbench` - Ensures only one set of integration tests runs globally - Matrix strategy runs JUnit 4/5/6 sequentially (max-parallel: 1) - Total concurrent sessions: exactly 5 (within SauceLabs limit) ## Resource Utilization **Multiple PRs can run in parallel:** - format-check: ✅ parallel across all PRs - build: ✅ parallel across all PRs (includes javadoc) - unit-tests: ✅ parallel across all PRs - integration-tests: ❌ queued globally (SauceLabs constraint only) **Performance benefits:** - Faster feedback: format/build/unit-tests run immediately in parallel - Faster builds: build once vs 3x (33% of previous time) - Javadoc generated once vs 3x (67% reduction) - Better SauceLabs utilization: full 5-session parallelism per test run - No wasted concurrent sessions from multiple workflows - Integration tests focus purely on browser testing (no javadoc overhead) ## Maven Cache Strategy Cache key includes `github.run_id` for per-workflow isolation: - build job: creates cache with `actions/cache@v4` - unit-tests job: restores cache with `actions/cache/restore@v4` - integration-tests job: restores cache with `actions/cache/restore@v4`
Artur-
force-pushed
the
one-workflow-concurrently
branch
from
890f70d to
98143a2
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Completely restructure the validation workflow to separate concerns and
respect SauceLabs concurrent session limits while maximizing parallelism.
New Job Structure
Key Improvements
Build once, use everywhere:
mvn clean install -DskipTestsoncegithub.run_idIndependent unit test execution:
mvn testseparately from integration testsOptimized integration tests:
mvn verify -DskipTests -pl <module> -am -P validation-DskipTestsskips unit tests (already ran in separate job)testsInParallel=5,forkCount=5SauceLabs concurrency control:
saucelabs-testbenchResource Utilization
Multiple PRs can run in parallel:
Performance benefits:
Maven Cache Strategy
Cache key includes
github.run_idfor per-workflow isolation:actions/cache@v4actions/cache/restore@v4actions/cache/restore@v4