Skip to content

Commit f34b9e2

Browse files
tapiostapios
authored
Add dev guides (#2515)
* Squashed 'docs/dev-guides/' content from commit b8c205b7e git-subtree-dir: docs/dev-guides git-subtree-split: b8c205b7edba94997617711487ff2f999373c9b3 * Update local dev guides * Update of test comments
1 parent f691a73 commit f34b9e2

38 files changed

Lines changed: 3646 additions & 0 deletions

‎.github/workflows/update_dev_guides.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Drop this file into a consumer repo at `.github/workflows/update_dev_guides.yml`.
2+
# It pulls the latest DeveloperGuides into `docs/dev-guides/` via `git subtree`
3+
# on the 1st of every month and opens a PR when there are changes.
4+
#
5+
# Prerequisites:
6+
# - DeveloperGuides has already been added as a subtree at `docs/dev-guides/`:
7+
# git subtree add --prefix docs/dev-guides \
8+
# https://github.com/CliMA/DeveloperGuides.git main --squash
9+
# - The repo's GitHub Actions are allowed to open pull requests
10+
# (Settings → Actions → General → "Allow GitHub Actions to create and
11+
# approve pull requests").
12+
13+
name: Update Dev Guides
14+
on:
15+
schedule:
16+
- cron: '0 0 1 * *' # 1st of every month at 00:00 UTC
17+
workflow_dispatch:
18+
19+
jobs:
20+
update-subtree:
21+
runs-on: ubuntu-latest
22+
steps:
23+
- uses: actions/checkout@v4
24+
with:
25+
fetch-depth: 0 # Full history is required for subtree pull
26+
27+
- name: Configure Git
28+
run: |
29+
git config user.name "github-actions[bot]"
30+
git config user.email "github-actions[bot]@users.noreply.github.com"
31+
32+
- name: Pull latest dev guides
33+
run: |
34+
git checkout -b update-dev-guides
35+
# Exit 0 when there is nothing new to merge (subtree pull errors in that case).
36+
git subtree pull --prefix docs/dev-guides \
37+
https://github.com/CliMA/DeveloperGuides.git main --squash \
38+
-m "chore: sync dev guides from central repo" || true
39+
40+
- name: Check for updates
41+
id: check
42+
run: |
43+
if [ "$(git rev-parse HEAD)" != "$(git rev-parse origin/main)" ]; then
44+
echo "changes=true" >> $GITHUB_OUTPUT
45+
else
46+
echo "changes=false" >> $GITHUB_OUTPUT
47+
fi
48+
49+
- name: Create Pull Request
50+
if: steps.check.outputs.changes == 'true'
51+
env:
52+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
53+
run: |
54+
git push -f origin update-dev-guides
55+
gh pr create --title "chore: sync dev guides with central repo" \
56+
--body "Automated PR to sync \`docs/dev-guides\` with the latest changes from \`CliMA/DeveloperGuides\`." \
57+
--base main --head update-dev-guides || true

‎AGENTS.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# AGENTS.md
2+
3+
Read these documents in the order listed when starting work on this repository.
4+
5+
## Before you act: agent autonomy
6+
7+
Before making changes that are externally visible or scientifically consequential (`git push`, version bumps, reproducibility-test edits, CI config changes, public API renames), check [docs/dev-guides/workflow/agent_autonomy.md](docs/dev-guides/workflow/agent_autonomy.md). The boundaries listed there require explicit user approval.
8+
9+
## Shared guides (via DeveloperGuides subtree)
10+
11+
The shared engineering guidelines are vendored as a Git subtree at `docs/dev-guides/`. Julia's `Pkg` does not resolve git submodules, so subtree is the standard mechanism across the CliMA ecosystem. The full index lives at [docs/dev-guides/AGENTS.md](docs/dev-guides/AGENTS.md). Start there for:
12+
- Architecture, design patterns, and cross-repo contracts
13+
- GPU performance, type stability, and AD compatibility
14+
- Code style, testing, and PR review
15+
16+
Edits to shared guidelines belong in [CliMA/DeveloperGuides](https://github.com/CliMA/DeveloperGuides), not in the vendored copy here. A scheduled workflow (`.github/workflows/update_dev_guides.yml`) syncs the subtree monthly.
17+
18+
## Repo-specific guide
19+
20+
- [docs/clima_core_specific.md](docs/clima_core_specific.md) — directory layout, key abstractions, test groups, and conventions specific to this package.
21+
22+
## Self-correction
23+
24+
If this file is discovered to be stale or missing a section, update it.

‎docs/clima_core_specific.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# ClimaCore.jl — Repo-Specific Guide
2+
3+
## Package overview
4+
5+
ClimaCore.jl provides the dynamical core infrastructure for [CliMA](https://clima.caltech.edu/)'s Earth System Model. It supplies flexible and composable discretization tools — spectral element and finite difference operators, field abstractions, data layouts, and GPU-portable kernels — that downstream packages (ClimaAtmos.jl, ClimaLand.jl, ClimaCoupler.jl) build on to solve the governing equations of their component models.
6+
7+
## Directory map
8+
9+
| Layer | Directory | Description |
10+
|:---|:---|:---|
11+
| Core module | `src/ClimaCore.jl` | Top-level module, re-exports all sub-modules |
12+
| DataLayouts | `src/DataLayouts/` | Low-level array-of-structs / struct-of-arrays data storage backends |
13+
| Geometry | `src/Geometry/` | Coordinate types, axis tensors, covariant/contravariant transforms |
14+
| Domains | `src/Domains/` | Abstract domain definitions (intervals, rectangles, spheres) |
15+
| Meshes | `src/Meshes/` | Mesh generation: interval, rectangle, cubed-sphere |
16+
| Topologies | `src/Topologies/` | Distributed topologies, DSS (direct stiffness summation) connectivity |
17+
| Quadratures | `src/Quadratures/` | Gauss–Legendre and Gauss–Lobatto quadrature rules |
18+
| Grids | `src/Grids/` | Spectral element and finite-difference grid types |
19+
| Spaces | `src/Spaces/` | Function spaces built on grids (spectral element, finite-difference, extruded) |
20+
| Fields | `src/Fields/` | `Field` type — the primary user-facing data container on a space |
21+
| Operators | `src/Operators/` | Spectral element and finite-difference differential operators |
22+
| MatrixFields | `src/MatrixFields/` | Banded matrix fields for implicit vertical solvers |
23+
| Hypsography | `src/Hypsography/` | Terrain-following coordinate transforms |
24+
| Limiters | `src/Limiters/` | Flux limiters for transport |
25+
| Remapping | `src/Remapping/` | Interpolation and remapping between spaces |
26+
| InputOutput | `src/InputOutput/` | HDF5-based checkpointing and restart I/O |
27+
| CommonGrids | `src/CommonGrids/` | Pre-built convenience grid constructors |
28+
| CommonSpaces | `src/CommonSpaces/` | Pre-built convenience space constructors |
29+
| Utilities | `src/Utilities/` | Internal utilities (PlusHalf indexing, AutoBroadcaster, etc.) |
30+
| DebugOnly | `src/DebugOnly/` | Debug-mode-only utilities |
31+
| CUDA ext | `ext/ClimaCoreCUDAExt.jl`, `ext/cuda/` | CUDA GPU extension (loaded via Pkg extensions) |
32+
| Krylov ext | `ext/KrylovExt.jl` | Krylov.jl integration for iterative solvers |
33+
| Lib: Plots | `lib/ClimaCorePlots/` | Plots.jl recipes for ClimaCore fields |
34+
| Lib: Makie | `lib/ClimaCoreMakie/` | Makie.jl recipes for ClimaCore fields |
35+
| Lib: VTK | `lib/ClimaCoreVTK/` | VTK output for visualization |
36+
| Lib: TempestRemap | `lib/ClimaCoreTempestRemap/` | TempestRemap bindings for conservative remapping |
37+
| Lib: Spectra | `lib/ClimaCoreSpectra/` | Spectral analysis of fields on the sphere |
38+
39+
## Key abstractions
40+
41+
1. **`Field`** (`src/Fields/`) — the primary data type. A field wraps data on a space and supports broadcast, reductions, and operator application.
42+
2. **`Space`** (`src/Spaces/`) — represents a discretized function space (spectral element, finite-difference, or extruded hybrid). Constructed from a grid and a quadrature rule.
43+
3. **Operators** (`src/Operators/`) — lazy differential operators (gradient, divergence, curl, interpolation, restriction) that compose via Julia's broadcast system.
44+
4. **`DataLayout`** (`src/DataLayouts/`) — the storage backends (IJFH, VIJFH, VF, etc.) that determine memory layout for CPU vs GPU performance.
45+
5. **`MatrixFields`** (`src/MatrixFields/`) — banded-matrix field algebra used for implicit vertical solvers and Jacobian construction.
46+
47+
## Test groups
48+
49+
Tests are defined in `test/runtests.jl` using the `UnitTest` / `tabulated_tests` framework:
50+
51+
| Group | What it covers |
52+
|:---|:---|
53+
| CPU unit tests | 104 tests covering DataLayouts, Geometry, Meshes, Topologies, Quadratures, Spaces, Fields, Operators (spectral element + finite-difference), MatrixFields, Hypsography, Limiters, Remapping, InputOutput, Aqua, deprecations |
54+
| GPU tests (`:gpu_only`) | 9 tests: CUDA kernels, compiler stress regression, DataLayout GPU ops, spectral element CUDA, finite-difference CUDA, extruded sphere/3dbox CUDA, field map-reduce CUDA |
55+
| Buildkite CI | Runs the unit tests on an HPC cluster with CUDA, defined in `.buildkite/pipeline.yml` |
56+
| Lib CI workflows | Separate GitHub Actions per companion package: ClimaCoreMakie, ClimaCorePlots, ClimaCoreSpectra, ClimaCoreTempestRemap, ClimaCoreVTK |
57+
58+
## Repo-specific conventions
59+
60+
- **Module-per-directory**: each `src/` subdirectory is its own Julia sub-module, re-exported from `ClimaCore.jl`.
61+
- **`lib/` companion packages**: visualization and remapping packages live as independent Julia packages under `lib/`, each with its own `Project.toml`. They have separate CI workflows.
62+
- **`ext/` CUDA pattern**: GPU support uses Julia's package extension mechanism (`ext/ClimaCoreCUDAExt.jl`). CPU fallbacks are always provided.
63+
- **Coding style**: `TitleCase` for types, `snake_case` for objects/functions, spaces after commas. Formatting follows [YASGuide](https://github.com/jrevels/YASGuide) loosely, enforced via `JuliaFormatter` (v1.0.62) in CI.
64+
- **ColPrac**: the project follows the [ColPrac guide](https://github.com/SciML/ColPrac) for collaborative practices.
65+
- **Tabulated test runner**: tests use the custom `UnitTest` struct and `run_unit_tests!` / `tabulate_tests` helpers from `test/tabulated_tests.jl`, with leak detection enabled.
66+
67+
## Self-correction
68+
69+
If this guide is discovered to be stale or missing a pattern, update it.

‎docs/dev-guides/AGENTS.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# CliMA Developer Guides — Agent Index
2+
3+
Read this file first. It is the main index for all shared engineering guidelines. Each guide applies across the CliMA ecosystem unless stated otherwise.
4+
5+
In consumer repos, these guides live at `docs/dev-guides/` and are supplied by a git subtree from the canonical source <https://github.com/CliMA/DeveloperGuides>. The consumer's root `AGENTS.md` references this file and the repo-specific guide. Edit shared guides in the canonical repo, not in the subtree copy.
6+
7+
## Before you act: agent autonomy
8+
9+
Before making changes that are externally visible or scientifically consequential (`git push`, version bumps, reproducibility-test edits, CI config changes, public API renames), check [workflow/agent_autonomy.md](workflow/agent_autonomy.md). The boundaries listed there require explicit user approval.
10+
11+
## Architecture
12+
13+
1. [repo_structure.md](architecture/repo_structure.md) — how to navigate any CliMA Julia package.
14+
2. [ecosystem_conventions.md](architecture/ecosystem_conventions.md) — module aliases, `Y`/`Yₜ`/`p` state layout, `ᶜ`/`ᶠ` notation, CI structure, reproducibility, diagnostics.
15+
3. [architectural_boundaries.md](architecture/architectural_boundaries.md) — layered architecture and boundary rules.
16+
4. [cross_repo_contracts.md](architecture/cross_repo_contracts.md) — call-site conventions for ecosystem packages.
17+
5. [dependency_management.md](architecture/dependency_management.md) — runtime vs dev deps, compat bounds.
18+
19+
## Performance
20+
21+
1. [gpu_performance.md](performance/gpu_performance.md) — GPU kernel rules, broadcast patterns, allocation avoidance.
22+
2. [type_stability.md](performance/type_stability.md) — Float32 compatibility, inference checks, struct field rules.
23+
3. [numerical_robustness.md](performance/numerical_robustness.md) — denominator regularization, clamping, NaN/Inf avoidance.
24+
4. [ad_compatibility.md](performance/ad_compatibility.md) — AD-safe patterns for ForwardDiff and Enzyme.
25+
5. [allocation_debugging.md](performance/allocation_debugging.md) — locating heap allocations with `Profile.Allocs`, JET, `@code_warntype`, flame graphs.
26+
27+
## Code Quality
28+
29+
1. [code_style.md](code-quality/code_style.md) — formatting, variable locality, Git workflow, feature removal, naming conventions.
30+
2. [documentation_policy.md](code-quality/documentation_policy.md) — docstrings, repository-level docs, minimally viable documentation.
31+
3. [changelogs_and_versions.md](code-quality/changelogs_and_versions.md) — `NEWS.md` format, SemVer rules, and the release/tagging flow.
32+
4. [variable_list.md](code-quality/variable_list.md) — standardized CliMA variable naming conventions.
33+
5. [software_design_patterns.md](code-quality/software_design_patterns.md) — numbered SDPs: branchless logic, functors, parameter extraction, etc.
34+
35+
## Infrastructure
36+
37+
1. [testing_and_validation.md](infrastructure/testing_and_validation.md) — type-stability checks, Aqua.jl, allocation regression, AD tests.
38+
2. [clima_comms.md](infrastructure/clima_comms.md) — device-agnostic and MPI-distributed code patterns.
39+
40+
## Workflow
41+
42+
1. [onboarding.md](workflow/onboarding.md) — install Julia, clone a CliMA repo, set up Revise/Infiltrator/JuliaFormatter, first PR loop.
43+
2. [agent_autonomy.md](workflow/agent_autonomy.md) — actions that require explicit user approval.
44+
3. [debugging.md](workflow/debugging.md) — interactive debugging recipes: numerical instabilities, dispatch, `Field` plotting.
45+
4. [review.md](workflow/review.md) — PR review instructions and checklist.
46+
5. [ci_triage.md](workflow/ci_triage.md) — checklist for "passes locally, fails on CI" failure modes.
47+
6. [cross_repo_issue_pr_search.md](workflow/cross_repo_issue_pr_search.md) — org-scoped GitHub search to find and filter issues/PRs across CliMA.
48+
49+
## Self-correction
50+
51+
If this index is discovered to be stale or missing a guide, update it.

0 commit comments

Comments
 (0)