Tests

Fix three fabricated-result bugs, close the RCE, and overhaul the documentation #244

Workflow file for this run

	name: Tests

	on:
	push:
	branches: [ master, main, develop ]
	pull_request:
	branches: [ master, main, develop ]

	permissions:
	contents: write
	pull-requests: read

	jobs:
	test:
	runs-on: ${{ matrix.os }}
	timeout-minutes: 15
	strategy:
	matrix:
	os: [ubuntu-latest, windows-latest, macos-latest]
	# 3.8 and 3.9 were never actually supported: click, requests,
	# huggingface_hub and python-dotenv all require >=3.10 at their current
	# versions, so those jobs installed years-old dependencies or failed
	# outright. 3.8 has been end-of-life since October 2024. (See #110.)
	python-version: ['3.10', '3.11', '3.12']
	exclude:
	# Reduce matrix size by excluding some combinations
	- os: windows-latest
	python-version: '3.10'
	- os: macos-latest
	python-version: '3.10'

	steps:
	- uses: actions/checkout@v4

	- name: Set up Python ${{ matrix.python-version }}
	uses: actions/setup-python@v5
	with:
	python-version: ${{ matrix.python-version }}

	- name: Install dependencies
	run: \|
	python -m pip install --upgrade pip
	pip install -e ".[dev,test,kubernetes,widget]"

	# Linting is version-independent, so run it once on the primary target
	# rather than on every matrix combination.
	- name: Lint with black
	if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
	run: \|
	black --check clustrix/ tests/ scripts/

	# No --exit-zero and no inline --extend-ignore: the flags made this step
	# incapable of failing, and the inline list was a second, divergent copy
	# of the project's lint policy. .flake8 is the single source of truth.
	- name: Lint with flake8
	if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
	run: \|
	flake8 clustrix/ tests/ scripts/

	- name: Test with pytest
	run: \|
	# Runs the whole non-billable suite, not just tests/unit/ (issue
	# #113). tests/real_world/conftest.py auto-applies the real_world
	# marker to everything under that directory, so -m "not real_world"
	# alone would already exclude it -- the --ignore is kept anyway as a
	# second, independent guard. tests/integration/ provisions real
	# billable AWS resources (#109) and must never run in ordinary CI.
	pytest tests/ -v --cov=clustrix --cov-report=xml --cov-report=html --cov-report=term-missing --cov-report=json -m "not real_world" --ignore=tests/real_world --ignore=tests/integration

	- name: Update coverage badge
	if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11' && github.ref == 'refs/heads/master'
	run: \|
	python .github/scripts/update_coverage_badge.py

	- name: Commit coverage badge update
	if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11' && github.ref == 'refs/heads/master'
	run: \|
	git config --local user.email "action@github.com"
	git config --local user.name "GitHub Action"
	git add README.md
	# Only push when there is something to push. The README carries no
	# coverage badge, so there never is; pushing anyway asked the runner to
	# resolve a branch it may not be on, for no change.
	if git diff --staged --quiet; then
	echo "No coverage badge change to commit."
	else
	git commit -m "Update coverage badge [skip ci]"
	git push
	fi
	env:
	GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

	- name: Test installation
	run: \|
	pip install .
	python -c "import clustrix; print('Clustrix installed successfully')"

	- name: Test CLI
	run: \|
	clustrix --help
	clustrix config --help

	integration-test:
	runs-on: ubuntu-latest
	timeout-minutes: 10
	needs: test

	steps:
	- uses: actions/checkout@v4

	- name: Set up Python 3.11
	uses: actions/setup-python@v5
	with:
	python-version: '3.11'

	- name: Install dependencies
	run: \|
	python -m pip install --upgrade pip
	pip install -e ".[dev,test,kubernetes,widget]"

	- name: Run integration tests
	run: \|
	# Run only local integration tests that don't require external resources
	pytest tests/ -k "integration and local and not (gpu or aws or azure or gcp or ssh or slurm or pbs or sge)" -v -x -m "not real_world"

	- name: Test example scripts
	run: \|
	python -c "
	from clustrix import cluster, configure
	configure(cluster_host=None) # Local execution

	@cluster(cores=2)
	def test_example():
	return sum(range(100))

	result = test_example()
	assert result == 4950
	print('Example test passed!')
	"

	docs-test:
	runs-on: ubuntu-latest
	timeout-minutes: 10

	steps:
	- uses: actions/checkout@v4

	- name: Set up Python 3.11
	uses: actions/setup-python@v5
	with:
	python-version: '3.11'

	- name: Install system dependencies
	run: \|
	# Bounded: an unresponsive Ubuntu mirror makes a bare `apt-get update`
	# hang until the job's 10-minute cap, which it did on two consecutive
	# runs while fetching noble-security InRelease. Each fetch now gives up
	# after 15s and retries three times, so a dead mirror costs seconds.
	sudo apt-get update \
	-o Acquire::Retries=3 \
	-o Acquire::http::Timeout=15 \
	-o Acquire::https::Timeout=15
	sudo apt-get install -y --no-install-recommends pandoc

	- name: Install dependencies
	run: \|
	python -m pip install --upgrade pip
	pip install -e ".[docs]"

	- name: Test documentation build
	run: \|
	cd docs
	make html

	- name: Test notebook execution
	run: \|
	pip install jupyter nbconvert
	jupyter nbconvert --to script docs/source/notebooks/basic_usage.ipynb
	# Note: Full notebook execution would require cluster setup

Navigation Menu

Uh oh!

Fix three fabricated-result bugs, close the RCE, and overhaul the documentation #244

Workflow file

Fix three fabricated-result bugs, close the RCE, and overhaul the documentation #244

Uh oh!

Workflow file for this run