ENH: Physics-informed cardiac motion with a neo-Hookean loss, tutorials 16-18 #457

Workflow file for this run

	name: CI

	# Comprehensive CI workflow combining unit tests, integration tests, GPU tests, and code quality checks
	#
	# Test organization:
	# - Unit tests: Run on all PRs and pushes, cross-platform (Ubuntu + Windows), multiple Python versions
	# - Integration tests: Run with external data, Ubuntu only
	# - GPU tests: Self-hosted runners with CUDA support
	# - Code quality: Linting and formatting checks
	#
	# Test markers (gated by opt-in pytest flags; default = skip):
	# - slow -> --run-slow
	# - requires_gpu -> --run-gpu
	# - requires_simpleware -> --run-simpleware (also implies GPU)
	# - requires_physicsnemo -> --run-physicsnemo (needs the [physicsnemo] extra)
	# - tutorial -> --run-tutorials
	# --run-all enables every bucket above at once.
	# Tests that need external data download it automatically via fixtures.

	on:
	push:
	branches: [ main, master, develop ]
	pull_request:
	branches: [ main, master, develop ]
	workflow_dispatch:

	jobs:
	# ============================================================================
	# Cross-Platform Unit Tests (Ubuntu + Windows)
	# ============================================================================
	unit-tests:
	name: Unit Tests (${{ matrix.os }}, Python ${{ matrix.python-version }})
	runs-on: ${{ matrix.os }}
	strategy:
	fail-fast: false
	matrix:
	os: [ubuntu-latest, windows-latest]
	python-version: ['3.11', '3.12', '3.13']

	steps:
	- name: Checkout code
	uses: actions/checkout@v6
	with:
	lfs: true

	- name: Free up disk space on Ubuntu
	if: matrix.os == 'ubuntu-latest'
	run: \|
	echo "Disk space before cleanup:"
	df -h
	sudo rm -rf /usr/share/dotnet
	sudo rm -rf /usr/local/lib/android
	sudo rm -rf /opt/ghc
	sudo rm -rf /opt/hostedtoolcache/CodeQL
	sudo docker image prune --all --force
	echo "Disk space after cleanup:"
	df -h

	- name: Set up Python ${{ matrix.python-version }}
	uses: actions/setup-python@v6
	with:
	python-version: ${{ matrix.python-version }}
	cache: 'pip'
	cache-dependency-path: pyproject.toml

	- name: Cache pip packages
	uses: actions/cache@v4
	with:
	path: ~/.cache/pip
	key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('pyproject.toml') }}
	restore-keys: \|
	${{ runner.os }}-pip-${{ matrix.python-version }}-
	${{ runner.os }}-pip-

	- name: Install system dependencies (Ubuntu)
	if: matrix.os == 'ubuntu-latest'
	run: \|
	sudo apt-get update
	sudo apt-get install -y \
	libgl1 \
	libglib2.0-0 \
	libgomp1 \
	libsm6 \
	libxrender1 \
	libxext6 \
	libxrandr2 \
	libxi6 \
	xvfb \
	libosmesa6
	sudo apt-get clean
	sudo rm -rf /var/lib/apt/lists/*

	- name: Upgrade pip and build tools
	run: \|
	python -m pip install --upgrade pip setuptools wheel

	- name: Install package with test dependencies
	run: \|
	pip install -e ".[test]"

	- name: Clear pip cache
	run: \|
	pip cache purge \|\| true

	- name: List installed packages
	run: \|
	pip list

	- name: Run unit tests (fast, no GPU/slow/tutorial) - Ubuntu
	if: matrix.os == 'ubuntu-latest'
	run: \|
	xvfb-run -a --server-args="-screen 0 1024x768x24" \
	pytest tests/ -v --cov=physiotwin4d --cov-report=xml --cov-report=term --cov-report=html

	- name: Run unit tests (fast, no GPU/slow/tutorial) - Windows
	if: matrix.os == 'windows-latest'
	run: \|
	pytest tests/ -v --cov=physiotwin4d --cov-report=xml --cov-report=term --cov-report=html

	- name: Upload coverage to Codecov
	uses: codecov/codecov-action@v4
	if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
	with:
	token: ${{ secrets.CODECOV_TOKEN }}
	file: ./coverage.xml
	flags: unittests
	name: codecov-unit-${{ matrix.os }}-py${{ matrix.python-version }}
	fail_ci_if_error: false

	- name: Upload coverage artifacts
	uses: actions/upload-artifact@v6
	if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
	with:
	name: coverage-report-unit-tests
	path: htmlcov/
	retention-days: 7

	# ============================================================================
	# Integration Tests with External Data (Ubuntu only, on PRs)
	# ============================================================================
	integration-tests:
	name: Integration Tests (with data)
	runs-on: ubuntu-latest
	needs: unit-tests
	if: github.event_name == 'pull_request'

	steps:
	- name: Checkout code
	uses: actions/checkout@v6
	with:
	lfs: true

	- name: Free up disk space
	run: \|
	echo "Disk space before cleanup:"
	df -h
	sudo rm -rf /usr/share/dotnet
	sudo rm -rf /usr/local/lib/android
	sudo rm -rf /opt/ghc
	sudo rm -rf /opt/hostedtoolcache/CodeQL
	sudo docker image prune --all --force
	echo "Disk space after cleanup:"
	df -h

	- name: Set up Python 3.11
	uses: actions/setup-python@v6
	with:
	python-version: '3.11'
	cache: 'pip'
	cache-dependency-path: pyproject.toml

	- name: Cache pip packages
	uses: actions/cache@v4
	with:
	path: ~/.cache/pip
	key: ${{ runner.os }}-pip-integration-${{ hashFiles('pyproject.toml') }}
	restore-keys: \|
	${{ runner.os }}-pip-integration-
	${{ runner.os }}-pip-

	- name: Cache test data
	uses: actions/cache@v4
	with:
	path: \|
	tests/data/
	tests/results/
	key: test-data-${{ hashFiles('tests/test_*.py') }}-v2
	restore-keys: \|
	test-data-

	- name: Install system dependencies
	run: \|
	sudo apt-get update
	sudo apt-get install -y \
	libgl1 \
	libglib2.0-0 \
	libgomp1 \
	libsm6 \
	libxrender1 \
	libxext6 \
	libxrandr2 \
	libxi6 \
	xvfb \
	libosmesa6
	sudo apt-get clean
	sudo rm -rf /var/lib/apt/lists/*

	- name: Upgrade pip and build tools
	run: \|
	python -m pip install --upgrade pip setuptools wheel

	- name: Install package with test dependencies
	run: \|
	pip install -e ".[test]"

	- name: Clear pip cache
	run: \|
	pip cache purge \|\| true

	- name: Run data download tests
	run: \|
	pytest tests/test_download_heart_data.py -v --cov=physiotwin4d --cov-report=xml
	continue-on-error: true

	- name: Run data conversion tests
	run: \|
	pytest tests/test_convert_image_4d_to_3d.py -v --cov=physiotwin4d --cov-append --cov-report=xml
	continue-on-error: true

	- name: Run contour tools tests
	run: \|
	pytest tests/test_contour_tools.py -v --cov=physiotwin4d --cov-append --cov-report=xml
	continue-on-error: true

	- name: Run USD conversion tests
	run: \|
	xvfb-run -a --server-args="-screen 0 1024x768x24" \
	pytest tests/test_convert_vtk_to_usd.py -v --cov=physiotwin4d --cov-append --cov-report=xml
	continue-on-error: true

	- name: Run USD utility tests
	run: \|
	xvfb-run -a --server-args="-screen 0 1024x768x24" \
	pytest tests/test_usd_merge.py tests/test_usd_time_preservation.py -v --cov=physiotwin4d --cov-append --cov-report=xml
	continue-on-error: true

	- name: Run all integration tests
	run: \|
	xvfb-run -a --server-args="-screen 0 1024x768x24" \
	pytest tests/ -v
	continue-on-error: true

	- name: Upload coverage to Codecov
	uses: codecov/codecov-action@v4
	with:
	token: ${{ secrets.CODECOV_TOKEN }}
	file: ./coverage.xml
	flags: integration-tests
	name: codecov-integration
	fail_ci_if_error: false

	# ============================================================================
	# GPU Tests (Self-hosted runners with CUDA)
	# ============================================================================
	# NOTE: GPU tests are DISABLED for automatic runs because they wait indefinitely
	# if no self-hosted runner is available. Enable manually via workflow_dispatch
	# or by setting the 'run-gpu-tests' label on the PR.
	gpu-tests:
	name: GPU Tests
	runs-on: [self-hosted, Windows, X64, gpu]
	needs: unit-tests
	timeout-minutes: 30
	# Only run GPU tests on manual trigger or if 'run-gpu-tests' label is present
	if: github.event_name == 'workflow_dispatch' \|\| contains(github.event.pull_request.labels.*.name, 'run-gpu-tests')

	steps:
	- name: Checkout code
	uses: actions/checkout@v6
	with:
	lfs: true

	- name: Create venv in RUNNER_TEMP
	# The oldest interpreter the project supports, so this job is the one
	# that catches a regression at the floor. nightly-health.yml runs the
	# newest (3.13) for the opposite reason. Both are on the runner.
	run: \|
	& "C:\Program Files\Python311\python.exe" -m venv "$env:RUNNER_TEMP\physiotwin4d-venv"
	echo "$env:RUNNER_TEMP\physiotwin4d-venv\Scripts" >> $env:GITHUB_PATH

	- name: Check GPU availability
	run: nvidia-smi

	- name: Check nvcc availability
	# torch-scatter compiles CUDA kernels from source when no matching
	# pre-built wheel exists. Without nvcc on PATH the build silently falls
	# back to a CPU-only extension that only fails later, at test time, with
	# an opaque scatter error. Fail here instead.
	run: \|
	$nvcc = Get-Command nvcc -ErrorAction SilentlyContinue
	if (-not $nvcc) {
	Write-Error "nvcc not found on PATH; torch-scatter would build CPU-only"
	exit 1
	}
	Write-Output "nvcc found at $($nvcc.Source)"
	nvcc --version
	if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

	- name: Cache uv packages
	uses: actions/cache@v4
	with:
	path: ~\AppData\Local\uv\cache
	key: ${{ runner.os }}-uv-${{ hashFiles('pyproject.toml') }}
	restore-keys: \|
	${{ runner.os }}-uv-

	- name: Upgrade pip and build tools
	run: \|
	python -m pip install --upgrade pip setuptools wheel uv

	- name: Install package with test dependencies
	# uv respects [tool.uv.sources], routing torch to pytorch-cu130 for cuda13.
	# The [physicsnemo] extra brings in nvidia-physicsnemo so Tutorial 9 and
	# requires_physicsnemo-marked tests can run on this GPU runner.
	# Invoke via python -m uv so uv targets the active venv interpreter.
	run: \|
	python -m uv pip install -e ".[cuda13]"
	if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
	python -m uv pip install -e ".[test,cuda13,physicsnemo]" --no-build-isolation-package torch-scatter

	- name: Assert CUDA is accessible
	run: \|
	python -c "
	import sys, torch, cupy
	print(f'PyTorch {torch.__version__} \| CUDA toolkit {torch.version.cuda} \| CuPy {cupy.__version__}')
	if not torch.cuda.is_available():
	print('ERROR: torch.cuda.is_available() returned False', file=sys.stderr)
	sys.exit(1)
	n = torch.cuda.device_count()
	if n == 0:
	print('ERROR: torch.cuda.device_count() == 0', file=sys.stderr)
	sys.exit(1)
	cn = cupy.cuda.runtime.getDeviceCount()
	if cn == 0:
	print('ERROR: cupy.cuda.runtime.getDeviceCount() == 0', file=sys.stderr)
	sys.exit(1)
	print(f'OK: {n} GPU(s) visible to PyTorch and CuPy')
	"

	- name: List installed packages
	run: \|
	pip list

	- name: Run GPU tests
	# A pre-merge gate, so it runs the GPU buckets only and finishes inside
	# timeout-minutes. The exhaustive --run-all sweep, tutorials included,
	# belongs to nightly-health.yml: running it here too would duplicate six
	# hours of work on this same runner and could never fit the timeout.
	#
	# --require-tutorial-data turns missing data into a failure rather than a
	# skip, so this job cannot report green having run nothing.
	run: \|
	pytest tests/ -v --run-gpu --run-physicsnemo --require-tutorial-data --max-test-seconds=600 --timeout=1800 --cov=physiotwin4d --cov-report=xml --cov-report=term --cov-report=html
	env:
	CUDA_VISIBLE_DEVICES: 0
	# The datasets, the results and the trained networks live outside the
	# checkout, which actions/checkout wipes on every run. Each root has a
	# "test" subtree that this suite reads and writes, so a CI run never
	# touches a full run's files. Unset, each falls back to its in-repo
	# default; see data/README.md. These are the same paths
	# nightly-health.yml uses, because both jobs run on this same runner.
	PHYSIOTWIN_INPUT_DATA_DIR: D:\PhysioTwin4D\nightly-runner\data
	PHYSIOTWIN_OUTPUT_DATA_DIR: D:\PhysioTwin4D\nightly-runner\output
	PHYSIOTWIN_WEIGHTS_DIR: D:\PhysioTwin4D\nightly-runner\network_weights

	- name: Upload coverage to Codecov
	uses: codecov/codecov-action@v4
	with:
	token: ${{ secrets.CODECOV_TOKEN }}
	file: ./coverage.xml
	flags: gpu-tests
	name: codecov-gpu
	fail_ci_if_error: false

	- name: Upload coverage artifacts
	uses: actions/upload-artifact@v6
	with:
	name: coverage-report-gpu
	path: htmlcov/
	retention-days: 7

	# ============================================================================
	# Code Quality Checks
	# ============================================================================
	code-quality:
	name: Code Quality Checks
	runs-on: ubuntu-latest

	steps:
	- name: Checkout code
	uses: actions/checkout@v6
	with:
	lfs: true

	- name: Set up Python 3.11
	uses: actions/setup-python@v6
	with:
	python-version: "3.11"
	cache: 'pip'

	- name: Install dev dependencies
	run: \|
	python -m pip install --upgrade pip
	# Ruff runs via pre-commit so its version is pinned in one place
	# (.pre-commit-config.yaml); mypy is still invoked directly below.
	pip install pre-commit mypy
	pip install -e ".[dev]"

	- name: Check formatting with Ruff (via pre-commit)
	run: \|
	pre-commit run ruff-format --all-files
	continue-on-error: false

	- name: Lint with Ruff (via pre-commit)
	run: \|
	pre-commit run ruff-check --all-files
	continue-on-error: false

	- name: Type check with mypy
	run: \|
	mypy src/
	continue-on-error: false

	# ==============================================================================
	# Notes on Excluded Tests
	# ==============================================================================
	#
	# The following tests are excluded from CI and should be run locally:
	#
	# Slow/GPU-intensive tests:
	# - tests/test_register_images_ants.py (slow, computationally intensive)
	# - tests/test_register_images_icon.py (requires CUDA for ICON)
	# - tests/test_transform_tools.py (depends on slow registration tests)
	# - tests/test_segment_chest_total_segmentator.py (requires CUDA for TotalSegmentator)
	#
	# Tutorial tests (SLOW - hours to complete):
	# - tests/test_tutorials.py (runs all 29 tutorial scripts end-to-end)
	# These tests are NEVER run in the PR CI and must be opted into
	# They execute end-to-end workflows that may take multiple hours
	#
	# To run locally:
	# pytest tests/ -v --run-slow # Run all slow tests
	# pytest tests/ -v --run-gpu --run-slow # GPU + slow (typical local dev profile)
	# pytest tests/ -v --run-simpleware --run-gpu --run-slow # Full Simpleware coverage
	# pytest tests/test_register_images_ants.py -v --run-slow
	#
	# Self-hosted GPU runner enables ALL buckets via --run-all
	# (--run-gpu --run-slow --run-simpleware --run-physicsnemo --run-tutorials).
	# That runner installs the [physicsnemo] extra in addition to [test,cuda13].
	#
	# To run tutorial tests (manual only, slow):
	# pytest tests/test_tutorials.py -v --run-tutorials
	# pytest tests/test_tutorials.py::TestTutorial01HeartGatedCTToUSD -v --run-tutorials

Navigation Menu

Uh oh!

ENH: Physics-informed cardiac motion with a neo-Hookean loss, tutorials 16-18 #457

Workflow file

ENH: Physics-informed cardiac motion with a neo-Hookean loss, tutorials 16-18 #457

Uh oh!

Workflow file for this run