[RFC] Move to a more modular structure for tests in code and CI

[JCQuintas]

What's the problem?

As teams, features and tests grow, the tests and CI workflows will keep getting slower.
Our current unit tests take 18 minutes to complete.

But our teams are mostly independent from each other, changes in Pickers don't affect Charts.
Which means we can divide possibly long running tasks into smaller ones, more focused on the components that did change in the PRs.

This would also allow us to divide the tests and configurations in our current test folder into more specialised ones. Only catering for the needs of a single library or goal. Eg: Charts tests don't need Date manipulation, but might require SVG-DOM support. Or a playwright test doesn't need any dependency from mocha, since it can run all tests by itself.

What are the requirements?

• Faster start-to-finish pipelines
  • Pipelines should only runs on PRs that change their related component or global libs.
    • Changes to packages/x-grid* run pipelines related to x-grid
    • Changes to docs/package.json run pipelines related to docs
    • Changes to package.json run all pipelines
• Standalone test packages/projects/folders for each team/library or goal.
  • Eg: team/lib: grid/treeview/etc, goal: performance/unit/etc
  • We can have folders named e2e-grid, unit-pickers, performance-charts, etc.

What are our options?

An option would be to try to divide and parallelise the current unit tests by filtering the folders. But it wouldn't solve folder structure and organisation.

Proposed solution

• Divide each test into their purpose-team packages
  • We can keep a single CI while migrating to different packages
• Create folder-aware workflows for the new structure.
  • Each workflow should be able to run independently from each other
  • Each workflow should only run when related files are changed
  • CircleCI: might be a bit more complex, as they don't seem to support filtering by file changed out of the box, but is possible
    • https://discuss.circleci.com/t/how-to-avoid-ci-if-only-certain-files-change/44360
    • https://circleci.com/docs/dynamic-config/#use-path-filtering-for-monorepos
  • Github Actions: Possible and easy to setup
    • https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushpull_requestpull_request_targetpathspaths-ignore

Resources and benchmarks

No response

Search keywords:

[JCQuintas]

related #13718

[Janpot]

Create folder-aware workflows for the new structure.

IMO the CI is not the right place to solve this problem. It will require us to manually micromanage a parallel dependency structure that mimics our already existing packages dependency graph. We recently started more and more relying on nx as a task runner. Have you already looked into their affected code feature?

[michelengelen]

Shouldn't this be a shaping page instead?

[oliviertassinari]

Related to:

• https://groups.google.com/a/mui.com/g/code-infra/c/eEVbYmMVOF4 (CI speed prioritzation)
• [core] Migrate test runner to vitest pigment-css#71 (Vitest could allow us to run tests in parallel)