Skip to content

Commit 6ecde70

Browse files
committed
Update AGENTS.md to document native ARM runner CI architecture
- Document 3-job pattern for each flavor (amd64, arm64, manifest) - Add build job flow diagram showing parallel architecture chains - Document ARM64 test jobs that run on PRs with docker changes - Add ARM Test Skipping section with code examples - Document registry strategy (GHCR for CI, Quay for production) - Remove outdated references to QEMU emulation
1 parent dbb268f commit 6ecde70

File tree

1 file changed

+62
-17
lines changed

1 file changed

+62
-17
lines changed

AGENTS.md

Lines changed: 62 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -328,37 +328,80 @@ docker build --build-arg BASEIMAGE=viral-ngs:core \
328328

329329
The `.github/workflows/docker.yml` workflow handles building and testing:
330330

331-
**Build Jobs:**
332-
1. **paths-filter**: Detect which code paths changed (using `dorny/paths-filter`)
333-
2. **get-version**: Extract version from git describe
334-
3. **build-baseimage**: Build base image with conda/python
335-
4. **build-core**: Build core image (depends on baseimage)
336-
5. **build-derivatives**: Build assemble, classify, phylo in parallel (depend on core)
337-
6. **build-mega**: Build all-in-one image (depends on core)
331+
**Build Architecture:**
332+
Each image flavor is built using 3 parallel jobs for native multi-arch support:
333+
1. `build-{flavor}-amd64` - runs on `ubuntu-latest`
334+
2. `build-{flavor}-arm64` - runs on `ubuntu-24.04-arm` (native ARM runner)
335+
3. `create-manifest-{flavor}` - combines arch-specific images into multi-arch manifest
336+
337+
This approach is 3-5x faster than QEMU emulation for ARM builds.
338+
339+
**Build Job Flow:**
340+
```
341+
paths-filter + get-version (parallel)
342+
343+
build-baseimage-amd64 ←→ build-baseimage-arm64 (parallel)
344+
↓ ↓
345+
create-manifest-baseimage
346+
347+
build-core-amd64 ←→ build-core-arm64 (parallel)
348+
↓ ↓
349+
create-manifest-core
350+
351+
build-{assemble,classify,phylo,mega}-amd64 ←→ build-{...}-arm64 (parallel)
352+
353+
create-manifest-{flavor}
354+
355+
test-{flavor} + test-{flavor}-arm64 (ARM64 tests only on PRs with docker changes)
356+
357+
deploy-to-quay (push/tag events only)
358+
```
338359

339360
**Test Jobs:**
340-
- **test-core**: Runs after build-core, tests `tests/unit/core/`
341-
- **test-assemble**: Runs after build-derivatives, tests `tests/unit/assemble/`
342-
- **test-classify**: Runs after build-derivatives, tests `tests/unit/classify/`
343-
- **test-phylo**: Runs after build-derivatives, tests `tests/unit/phylo/`
361+
- **test-core**: Runs on x86, tests `tests/unit/core/`
362+
- **test-assemble**: Runs on x86, tests `tests/unit/assemble/`
363+
- **test-classify**: Runs on x86, tests `tests/unit/classify/`
364+
- **test-phylo**: Runs on x86, tests `tests/unit/phylo/`
365+
- **test-{flavor}-arm64**: Runs on native ARM, only on PRs when docker files change
344366

345367
**Smart Test Scoping:**
346368
Tests only run when relevant code changes:
347369
- Core tests: `src/viral_ngs/*.py`, `core/**`, `util/**`, `tests/unit/core/**`
348370
- Assemble tests: `assemble/**`, `assembly.py`, or core changes
349371
- Classify tests: `classify/**`, `metagenomics.py`, `taxon_filter.py`, or core changes
350372
- Phylo tests: `phylo/**`, `interhost.py`, `intrahost.py`, `ncbi.py`, or core changes
351-
- Docker changes trigger all tests
373+
- Docker changes trigger all tests (including ARM64 tests on PRs)
352374

353375
**Coverage:**
354-
Each test job uploads coverage to Codecov with flavor-specific flags.
376+
Each x86 test job uploads coverage to Codecov with flavor-specific flags.
355377

356378
### Multi-Architecture Support
357379

358-
- Images built for `linux/amd64` and `linux/arm64`
380+
- Images built natively for `linux/amd64` and `linux/arm64` using parallel runners
381+
- Multi-arch manifests created with OCI annotations using `docker buildx imagetools create`
359382
- x86-only packages (novoalign, mvicuna, bmtagger, kallisto, kb-python, table2asn) handled via `--x86-only:` prefix in `install-conda-deps.sh`
360383
- Python tool wrappers still importable on ARM64; only runtime execution fails for missing binaries
361-
- Cache stored on Quay.io registry (GHA 10GB limit too small)
384+
- Tests using x86-only tools have `@unittest.skipIf(IS_ARM, ...)` decorators
385+
- Architecture-specific caches prevent cross-arch cache pollution
386+
387+
### ARM Test Skipping
388+
389+
Tests that use x86-only bioconda packages must be decorated to skip on ARM:
390+
391+
```python
392+
from tests import IS_ARM
393+
394+
SKIP_X86_ONLY_REASON = "tool requires x86-only bioconda package (not available on ARM)"
395+
396+
@unittest.skipIf(IS_ARM, SKIP_X86_ONLY_REASON)
397+
class TestSomeTool(TestCaseWithTmp):
398+
...
399+
400+
# Or at method level:
401+
@unittest.skipIf(IS_ARM, SKIP_X86_ONLY_REASON)
402+
def test_specific_tool(self):
403+
...
404+
```
362405

363406
### Documentation Build
364407

@@ -367,9 +410,11 @@ The `docs.yml` workflow builds Sphinx documentation. Key points:
367410
- When adding new imports to source code, add corresponding mocks to `MOCK_MODULES` in `docs/conf.py`
368411
- Runs `sphinx-build -W` (warnings as errors)
369412

370-
### Feature Branch Images
413+
### Registry Strategy
371414

372-
Feature branch images get `quay.expires-after=10w` label for automatic cleanup.
415+
- **GHCR (ghcr.io)**: Primary build registry, images pushed during CI for all events including PRs
416+
- **Quay.io**: Production registry, images copied from GHCR after tests pass (push/tag events only)
417+
- Feature branch images should be cleaned up periodically from Quay.io
373418

374419
---
375420

0 commit comments

Comments
 (0)