This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
- Run all tests:
pytest - Run a single test:
pytest tests/path/to/test_file.py::test_function_name -v - Format code:
ruff format - Lint code:
ruff check --fix - Type check:
mypy --exclude tests/test_package src tests
-
Formatting: Follow Google style convention. Use ruff for formatting
-
Imports: Use isort order (enforced by ruff)
-
Types: Strict typing is required. All functions must have type annotations
-
Naming: Use snake_case for variables, functions, methods; PascalCase for classes
-
Docstrings: Google-style docstrings required for public APIs
-
Error Handling: Use appropriate exception types; include context in error messages
-
Testing: Write tests with pytest; maintain high coverage. See "Testing Async Code" below for async test conventions.
-
Async Concurrency: Use
inspect_ai._util._async.tg_collect()instead ofasyncio.gather()for running concurrent async tasks. Useinspect_ai.util.collect()only inside sample subtasks (it adds transcript span grouping). -
File Paths: All code that handles file paths must support
s3://URLs,file://URIs, and plain local paths. Usefilesystem()frominspect_ai._util.filefor filesystem operations andlocal_path()to resolvefile://URIs to local paths before passing to APIs that only accept local paths (e.g.ZipFile). -
**Respect existing code patterns when modifying files. Run linting before committing changes.
All async test functions automatically run under both asyncio and trio backends via anyio (applied by the pytest_pycollect_makeitem hook in tests/conftest.py). Trio variants are skipped by default; use --runtrio to enable them.
- Do NOT use
@pytest.mark.asyncio— it conflicts with anyio and is blocked by conftest. Just writeasync def test_...and the hook handles the rest. - Use
anyio.sleep()notasyncio.sleep()in tests;anyio.Event()notasyncio.Event();tg_collect()notasyncio.gather(). - Use
@skip_if_trio(fromtest_helpers.utils) for tests that cannot run under trio (e.g. they test asyncio-specific fallback paths). @pytest.mark.anyiois not required but harmless — use it to signal intentional dual-backend coverage.
Additional files provide context when working in specific areas:
design/ contains architecture notes, subsystem internals, and documentation of repo/CI/development processes and workflows. Browse it before diving into an unfamiliar area.