Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom action and unified Linux build #207

Merged
merged 1 commit into from
Jan 31, 2025
Merged

Conversation

mshabunin
Copy link
Contributor

@mshabunin mshabunin commented Dec 10, 2024

Changes:

  • create single workflow for multiple containerized Linux builds using matrix build (branch, ubuntu version)
    • enable AVX2 for 24.04, enable plugins for 22.04, enable python limited API for 20.04
  • simplify dnn models download (just run the script)
  • extract separate actions for clone+merge, configure+build and test
    • use grouping to make collapsible blocks and error/notice formatting to show failures in subtests
    • combine build and build+contrib in single job
      • check for warnings using new modified script
    • add wrapper script for gtest runs (runner.py)
      • use json test plan file for list of modules, filters and other options
      • show only test failures and summary
      • save test output as artifacts
      • collect test status to markdown summary
    • do not save java test results as artifact
  • simplify environment variables, move some variables to their specific actions, add some new variables controlling build/test behavior (e.g. CMAKE_BUILD_PARALLEL_LEVEL)
  • The same workflow is also applicable to opencv_contrib repository

Things to be discussed:

  • add CUDA and OpenVINO builds to the set - we can add them as a separate job, or as new configurations in the Ubuntu job
  • test set for opencv_contrib repository - currently it runs all tests, we can limit it to the opencv_contrib only
  • timeouts - currently we have 360 min for whole workflow (default), 60 min for dnn models download (explicit) and 10 min for each module test (runner script)
  • thread count - currently it is set to 8 for build and test

Things which can be improved:

  • check cmake warnings
  • add more information to the summary (e.g. warnings, test statistics, timings, system information, build configuration, etc.)
  • move dnn models update to separate action or to the checkout-and-merge action
  • show beginning of test output in the log
  • done implement better timeout handling in the runner script (e.g. using asyncio)
  • sync home directory, user name and other common things in Ubuntu docker images
  • warn or exit if default branch name has been used as a source in the PR (4.x, 5.x) to avoid merging unexpected changes from other repositories
  • re-enable ASAN builds

Known issues:

  • Sporadic connection problems when uploading artifacts: Failed to CreateArtifact: Unable to make request: ECONNRESET If you are using self-hosted runners, please make sure your runner has access to all GitHub endpoints: https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#communication-between-self-hosted-runners-and-github

@mshabunin mshabunin force-pushed the add-custom-action branch 2 times, most recently from 88e181d to 6a3c62f Compare December 12, 2024 07:47
@asmorkalov
Copy link
Contributor

Let's handle it after 4.11 release. I do not think it urgent.

@mshabunin mshabunin force-pushed the add-custom-action branch 3 times, most recently from 71de8af to 12d5ab4 Compare January 23, 2025 12:12
@mshabunin
Copy link
Contributor Author

mshabunin commented Jan 28, 2025

@asmorkalov , I've updated the PR: disabled ASAN build temporarily, split test plan file, squashed commits.

@mshabunin mshabunin merged commit fb0caf0 into main Jan 31, 2025
7 checks passed
@mshabunin mshabunin deleted the add-custom-action branch January 31, 2025 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants