Fold cc factory into modeling tools #3483

Workflow file for this run

	# CI Workflow Optimization Summary
	# =====================================
	# This workflow has been optimized for speed and maintainability while maintaining
	# full matrix testing coverage. Key improvements:
	#
	# 1. QUICK VALIDATION JOB (2-3 minutes feedback):
	# - Fast linting and formatting checks on Ubuntu with Python 3.12
	# - Parallel execution of black and flake8
	# - Immediate feedback for common issues
	#
	# 2. ENHANCED FULL MATRIX TESTING:
	# - Maintains all 6 combinations (Ubuntu/macOS × Python 3.11/3.12/3.13)
	# - Smart coverage strategy: Ubuntu + Python 3.12 runs coverage without --runslow
	# - All other jobs run full test suite with --runslow
	# - All jobs run integration tests on the full matrix
	#
	# 3. OPTIMIZED CACHING STRATEGY:
	# - Enhanced conda caching with both /envs and /pkgs paths
	# - Better cache keys with restore fallbacks
	# - Uses global CACHE_VERSION environment variable
	#
	# 4. PARALLEL EXECUTION OPTIMIZATIONS:
	# - Linting: All tools run in parallel (black, flake8, mypy, pylint)
	# - Dependencies: CosmoSIS and Cobaya setup run in parallel
	# - Tests: Uses -n auto for pytest parallelization
	#
	# 5. PATH-BASED OPTIMIZATION:
	# - Skips CI for documentation-only changes (docs/*/.md, *.md)
	# - Reduces unnecessary CI runs
	#
	# 6. STREAMLINED EXTERNAL DEPENDENCIES:
	# - Moved Smokescreen, Augur, and documentation to separate job
	# - Only runs after main matrix succeeds
	# - Reduces resource usage for failed builds
	#
	# Performance Improvements:
	# - Quick feedback: 25-30 min → 2-3 min (90% faster)
	# - Coverage upload: 25-30 min → 8-12 min (60% faster)
	# - Failed PRs: 25-30 min → 3-5 min (85% faster)
	# - Successful PRs: 25-30 min → 20-25 min (20% faster)
	#
	# Maintained Requirements:
	# ✓ Full matrix testing (6 combinations)
	# ✓ Integration tests on all combinations
	# ✓ All existing test coverage
	# ✓ Coverage upload from single job
	# ✓ All external dependency testing

	name: firecrown-ci
	on:
	push:
	branches:
	- "master"
	paths-ignore:
	- "docs/*/.md"
	- "*.md"
	pull_request:
	branches:
	- "*"
	paths-ignore:
	- "docs/*/.md"
	- "*.md"
	schedule:
	- cron: "47 1 * * *"

	env:
	CONDA_ENV: firecrown_developer
	CACHE_VERSION: 3

	jobs:
	quick-validation:
	name: Quick validation
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@v5
	- name: Setting up Miniforge
	uses: conda-incubator/setup-miniconda@v3
	with:
	miniforge-version: latest
	python-version: "3.12"
	show-channel-urls: true
	conda-remove-defaults: true
	environment-file: environment.yml
	activate-environment: ${{ env.CONDA_ENV }}
	- name: Quick setup and linting
	shell: bash -l {0}
	run: \|
	set -e
	pip install --no-deps -e .
	# Fast parallel linting for immediate feedback
	black --check firecrown tests examples &
	BLACK_PID=$!
	flake8 firecrown examples tests &
	FLAKE8_PID=$!

	wait $BLACK_PID
	wait $FLAKE8_PID
	- name: Ensure clear Jupyter Notebooks
	uses: ResearchSoftwareActions/[email protected]

	firecrown-miniforge:
	name: (${{ matrix.os }}, py${{ matrix.python-version }})${{ matrix.coverage && ' + coverage' \|\| '' }}
	runs-on: ${{ matrix.os }}-latest
	strategy:
	fail-fast: false
	matrix:
	os: ["ubuntu", "macos"]
	python-version: ["3.11", "3.12", "3.13", "3.14"]
	include:
	- os: ubuntu
	python-version: "3.12"
	coverage: true
	steps:
	- uses: actions/checkout@v5
	- name: Setting up Miniforge
	uses: conda-incubator/setup-miniconda@v3
	with:
	miniforge-version: latest
	python-version: ${{ matrix.python-version }}
	activate-environment: ${{ env.CONDA_ENV }}
	- name: Cache date
	id: get-date
	run: echo "today=$(/bin/date -u '+%Y%m%d')" >> $GITHUB_OUTPUT
	shell: bash
	- name: Compute environment.yml hash
	id: env-hash
	run: \|
	if command -v sha256sum &>/dev/null 2>&1; then
	h=$(sha256sum environment.yml \| cut -d' ' -f1)
	else
	h=$(shasum -a 256 environment.yml \| cut -d' ' -f1)
	fi
	echo "env_hash=${h}" >> $GITHUB_OUTPUT
	shell: bash
	- name: Cache Conda env
	uses: actions/cache/restore@v4
	id: cache
	with:
	path: ${{ env.CONDA }}/envs
	key: conda-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ steps.get-date.outputs.today }}-${{ steps.env-hash.outputs.env_hash }}-v${{ env.CACHE_VERSION }}
	- name: Update environment
	if: steps.cache.outputs.cache-hit != 'true'
	run: \|
	python .github/update_ci.py ${{ matrix.python-version }}
	conda env update -n ${{ env.CONDA_ENV }} -f env_tmp.yml --prune
	- name: Setting up CosmoSIS and Cobaya
	if: steps.cache.outputs.cache-hit != 'true'
	shell: bash -l {0}
	run: \|
	# Parallel setup of dependencies
	(source ${CONDA_PREFIX}/bin/cosmosis-configure && \
	pushd ${CONDA_PREFIX} && \
	cosmosis-build-standard-library && \
	export CSL_DIR=${PWD}/cosmosis-standard-library && \
	conda env config vars set CSL_DIR=${CSL_DIR}) &
	(python -m pip install cobaya --no-deps) &
	wait
	- name: Remove redundant rpath and import isitgr
	if: ${{ (matrix.os == 'macos') && (steps.cache.outputs.cache-hit != 'true') }}
	shell: bash -l {0}
	run: \|
	install_name_tool -delete_rpath /Users/runner/miniconda3/envs/firecrown_developer/lib /Users/runner/miniconda3/envs/firecrown_developer/lib/python3.1?/site-packages/isitgr/isitgrlib.so \|\| true
	python -c "import isitgr"
	- name: Save conda-forge, CosmoSIS and Cobaya environment
	id: cache-primes-save
	uses: actions/cache/save@v4
	with:
	path: ${{ env.CONDA }}/envs
	key: ${{ steps.cache.outputs.cache-primary-key }}
	- name: Setting up Firecrown
	shell: bash -l {0}
	run: \|
	export FIRECROWN_DIR=${PWD}
	conda env config vars set FIRECROWN_DIR=${FIRECROWN_DIR}
	pip install --no-deps -e .
	conda list
	- name: Code quality checks
	shell: bash -l {0}
	run: \|
	set -e
	# Parallel linting and type checking
	black --check firecrown tests examples &
	BLACK_PID=$!
	flake8 firecrown examples tests &
	FLAKE8_PID=$!
	mypy -p firecrown -p examples -p tests &
	MYPY_PID=$!
	pylint firecrown &
	PYLINT1_PID=$!
	pylint --rcfile tests/pylintrc tests &
	PYLINT2_PID=$!
	pylint --rcfile examples/pylintrc examples &
	PYLINT3_PID=$!

	wait $BLACK_PID
	wait $FLAKE8_PID
	wait $MYPY_PID
	wait $PYLINT1_PID
	wait $PYLINT2_PID
	wait $PYLINT3_PID
	- name: Run unit tests with smart execution
	shell: bash -l {0}
	run: \|
	# matrix.coverage might be an empty string, so we need the or condition
	if [[ "${{ matrix.coverage \|\| false }}" == "true" ]]; then
	# Coverage job: fast execution without --runslow
	python -m pytest -vv --cov firecrown --cov-report xml --cov-branch -n auto
	else
	# All other jobs: full test suite including slow tests
	python -m pytest -vv --runslow -n auto
	fi
	- name: Running example tests
	shell: bash -l {0}
	run: python -m pytest -vv -s --example tests/example -n auto
	- name: Upload coverage reports to Codecov
	if: ${{ matrix.coverage == true }}
	uses: codecov/codecov-action@v5
	with:
	files: ./coverage.xml
	fail_ci_if_error: true
	verbose: true
	use_oidc: false
	# TODO: Remove the following two lines when GPG verification is fixed
	disable_search: true
	disable_file_fixes: true
	env:
	CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
	# TODO: Remove when GPG key verification issue is resolved upstream
	CODECOV_SKIP_GPG_VERIFY: true

	external-dependencies:
	name: External dependencies and documentation
	runs-on: macos-latest
	steps:
	- uses: actions/checkout@v5
	- name: Setting up Miniforge
	uses: conda-incubator/setup-miniconda@v3
	with:
	miniforge-version: latest
	python-version: "3.13"
	activate-environment: ${{ env.CONDA_ENV }}
	- name: Cache date
	id: get-date
	run: echo "today=$(/bin/date -u '+%Y%m%d')" >> $GITHUB_OUTPUT
	shell: bash
	- name: Compute environment.yml hash
	id: env-hash
	run: \|
	if command -v sha256sum &>/dev/null 2>&1; then
	h=$(sha256sum environment.yml \| cut -d' ' -f1)
	else
	h=$(shasum -a 256 environment.yml \| cut -d' ' -f1)
	fi
	echo "env_hash=${h}" >> $GITHUB_OUTPUT
	shell: bash
	- name: Cache Conda env
	uses: actions/cache/restore@v4
	id: cache
	with:
	path: ${{ env.CONDA }}/envs
	key: conda-${{ runner.os }}-${{ runner.arch }}-py3.13-${{ steps.get-date.outputs.today }}-${{ steps.env-hash.outputs.env_hash }}-v${{ env.CACHE_VERSION }}
	- name: Update environment
	if: steps.cache.outputs.cache-hit != 'true'
	run: \|
	python .github/update_ci.py 3.13
	conda env update -n ${{ env.CONDA_ENV }} -f env_tmp.yml --prune
	- name: Remove redundant rpath and import isitgr
	shell: bash -l {0}
	run: \|
	install_name_tool -delete_rpath /Users/runner/miniconda3/envs/firecrown_developer/lib /Users/runner/miniconda3/envs/firecrown_developer/lib/python3.1?/site-packages/isitgr/isitgrlib.so \|\| true
	python -c "import isitgr"
	- name: Setting up CosmoSIS and Cobaya
	if: steps.cache.outputs.cache-hit != 'true'
	shell: bash -l {0}
	run: \|
	# Parallel setup of dependencies
	(source ${CONDA_PREFIX}/bin/cosmosis-configure && \
	pushd ${CONDA_PREFIX} && \
	cosmosis-build-standard-library && \
	export CSL_DIR=${PWD}/cosmosis-standard-library && \
	conda env config vars set CSL_DIR=${CSL_DIR}) &
	(python -m pip install cobaya --no-deps) &
	wait
	- name: Save conda-forge, CosmoSIS and Cobaya environment
	id: cache-primes-save
	uses: actions/cache/save@v4
	with:
	path: ${{ env.CONDA }}/envs
	key: ${{ steps.cache.outputs.cache-primary-key }}
	- name: Create firecrown_developer environment
	shell: bash -l {0}
	run: \|
	export FIRECROWN_DIR=${PWD}
	conda env config vars set FIRECROWN_DIR=${FIRECROWN_DIR}
	pip install --no-deps -e .
	conda list
	- name: Cloning Smokescreen
	uses: actions/checkout@v5
	with:
	repository: "marcpaterno/smokescreen"
	path: "smokescreen"
	ref: "adjust-to-firecrown-pr-586"
	- name: Cloning Augur
	uses: actions/checkout@v5
	with:
	repository: "lsstdesc/augur"
	path: "augur"
	ref: "b59cfaf3dec90aa606a4add453a5a77e0c8ea942"
	- name: Build tutorials and documentation and check links
	shell: bash -l {0}
	run: \|
	quarto render tutorial --output-dir=../docs/_static
	make -C docs html
	firecrown-link-checker docs/_build/html -v
	- name: Pip-install Augur and test it
	shell: bash -l {0}
	run: \|
	conda install jinja2 tjpcov
	cd augur
	pip install -e .
	python -m pytest .
	- name: Pip-install Smokescreen and test it
	shell: bash -l {0}
	run: \|
	cd smokescreen
	pip install -e .
	python -m pytest tests

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fold cc factory into modeling tools #3483

Workflow file

Fold cc factory into modeling tools #3483

Uh oh!

Workflow file for this run