Make the unified DataLayouts API fast and correct on CPUs and GPUs [perf]

Verifies and refines the DataLayouts rewrite across CPUs, GPUs, docs, tests,
and downstream packages.

- Point indexing: getindex/setindex!/view constant-fold the Cartesian-to-linear
  conversion and use a constant-stride linear index into the parent of a
  property-view SubArray, bypassing Base's per-access column-major arithmetic
  and its linear-to-Cartesian reindex (div/rem, SignedMultiplicativeInverse).
  This bounds the deeply-inlined finite-difference stencil body that
  compile-killed the ClimaAtmos EDMF Larcform1 GPU job (OOM): implicit_tendency!
  compile ~110s -> ~32s, with column FD-op runtime gains and bit-identical
  values. The offset folds on device via unrolled_reduce (no InvalidIRError).

- Property views / IndexStyle: build slice and property SubArrays from Colons
  (Base.Slice) so VIJHF views are fast-linear (IndexLinear) while VIJFH views
  stay IndexCartesian; IndexStyle defers to the parent, so pointwise broadcasts
  never linear-index an IndexCartesian SubArray.

- GPU support: DataScopes map onto the CUDA execution hierarchy, loops and
  reductions launch through auto_launch! with occupancy-based configurations,
  and Utilities.stable_view keeps slice/property views inference-stable (kernel
  arguments and closures follow isbits rules). Each view is adapted to a compact
  device view (Int32 offsets instead of a SubArray) so large EDMF broadcasts fit
  the sm_60 4 KiB kernel-parameter limit, and DataScopes combine by pairwise
  recursion so inference does not widen scopes to Any.

- CPU runtime matches main: unmasked point loops vectorize under @simd with an
  inlined point function, nested loops avoid closure allocations, and GPU point
  loops iterate each thread's strided CartesianIndices subset through an
  indexable isbits wrapper, keeping kernel launch latency at main's level.

- Compile time: @maybe_propagate_inbounds requests inlining of stencil
  expression nodes only when check-bounds is off, taking the FCT advection
  examples from 40-58 minutes to about a minute in CI.

- Reductions and masks: order-insensitive pairwise safe_mapreduce (no
  linear-indexing assumption), masks are keyword arguments, equality ignores
  padding, and field2array / the distributed HDF5 writer handle all layout
  shapes. Fix the GPU MPI DSS exchange buffer index that overlapped items when
  Nv > 1 and Nf > 1, and apply the QuasiMonotoneLimiter through one scalar view
  per component (avoiding an uncompilable reshape of a device SubArray).

- Compatibility, docs, tests: deprecation aliases (DataLayouts/deprecated.jl,
  Fields.ColumnField) and the universal CartesianIndex{5} convention keep
  downstream packages working; zero-size fields stay hidden from propertynames;
  docs and NEWS are expanded and Aqua ambiguities resolved; tests restore
  check_basetype, benchmark_fill, and the VIJFH-F64 stencil set, and add
  DataLayouts mask/reduction jobs, device-aware dss tests, and latency baselines
  at their original tolerance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: dennisYatunin

.buildkite/Manifest.toml

@@ -12,6 +12,7 @@ ClimaCore = "d414da3d-4745-48bb-8d80-42e94e092884"
 ClimaCorePlots = "cf7c7e5a-b407-4c48-9047-11a94a308626"
 ClimaCoreTempestRemap = "d934ef94-cdd4-4710-83d6-720549644b70"
 ClimaCoreVTK = "c8b6d40d-e815-466f-95ae-c48aefa668fa"
+ClimaInterpolations = "dd0f122e-fa3b-47f3-bcf0-93bbc60d885e"
 ClimaParams = "5c42b081-d73a-476f-9059-fd94b934656c"
 ClimaTimeSteppers = "595c0a79-7f3d-439a-bc5a-b232dc3bde79"
 Colors = "5ae59095-9a9b-59fe-a467-6f913c188581"

.buildkite/Project.toml

@@ -26,6 +26,7 @@ steps:
       # Instantiate and dev install ClimaCore
       - julia --project=$COUPLER_PATH/experiments/AMIP/ -e 'using Pkg; Pkg.instantiate(;verbose=true)'
       - julia --project=$COUPLER_PATH/experiments/AMIP/ -e 'using Pkg; Pkg.develop(path=".")'
+      - julia --project=$COUPLER_PATH/experiments/AMIP/ -e 'using Pkg; Pkg.develop(path="lib/ClimaCoreMakie")'
       - julia --project=$COUPLER_PATH/experiments/AMIP/ -e 'using Pkg; Pkg.add("MPI")'
       - julia --project=$COUPLER_PATH/experiments/AMIP/ -e 'using Pkg; Pkg.precompile()'
       - julia --project=$COUPLER_PATH/experiments/AMIP/ -e 'using Pkg; Pkg.status()'

.buildkite/perf/pipeline.yml

@@ -114,6 +114,16 @@ steps:
         retry: *retry_policy
         command: "julia --threads=4 --color=yes --check-bounds=yes --project=.buildkite test/DataLayouts/unit_fill_and_copyto.jl"
 
+      - label: "Unit: data loops (1 thread)"
+        key: unit_data_loops
+        retry: *retry_policy
+        command: "julia --color=yes --check-bounds=yes --project=.buildkite test/DataLayouts/unit_loops.jl"
+
+      - label: "Unit: data loops (4 threads)"
+        key: threaded_unit_data_loops
+        retry: *retry_policy
+        command: "julia --threads=4 --color=yes --check-bounds=yes --project=.buildkite test/DataLayouts/unit_loops.jl"
+
       - label: "Unit: mapreduce (1 thread)"
         key: unit_data_mapreduce
         retry: *retry_policy
@@ -1568,6 +1578,22 @@ steps:
   - group: "Perf: DataLayouts"
     steps:
 
+      - label: "Perf: DataLayouts fill!"
+        key: "cpu_datalayouts_fill"
+        retry: *retry_policy
+        command: "julia --color=yes --project=.buildkite test/DataLayouts/benchmark_fill.jl"
+
+      - label: "Perf: DataLayouts fill"
+        key: "gpu_datalayouts_fill"
+        retry: *retry_policy
+        command:
+          - "julia --project=.buildkite -e 'using CUDA; CUDA.versioninfo()'"
+          - "julia --color=yes --project=.buildkite test/DataLayouts/benchmark_fill.jl"
+        env:
+          CLIMACOMMS_DEVICE: "CUDA"
+        agents:
+          slurm_gpus: 1
+
       - label: "Perf: DataLayouts copyto!"
         key: "cpu_datalayouts_copyto"
         retry: *retry_policy

.buildkite/pipeline.yml

@@ -18,7 +18,11 @@ jobs:
       - name: Install dependencies
         run: >
           julia --project=docs -e 'using Pkg; Pkg.develop(path=".");
-          Pkg.develop(path="lib/ClimaCoreVTK"); 
+          Pkg.develop(path="lib/ClimaCoreVTK");
+          Pkg.develop(path="lib/ClimaCoreTempestRemap");
+          Pkg.develop(path="lib/ClimaCoreMakie");
+          Pkg.develop(path="lib/ClimaCorePlots");
+          Pkg.develop(path="lib/ClimaCoreSpectra");
           Pkg.instantiate(;verbose=true)'
       - name: Build and deploy
         env:

.github/workflows/docs.yml

@@ -4,6 +4,17 @@ ClimaCore.jl Release Notes
 main
 -------
 
+- Refactor DataLayouts module [2522](https://github.com/CliMA/ClimaCore.jl/pull/2522)
+  - All data layout types are unified into a single `DataLayout` type, and all
+    loops over data go through two communication primitives (`foreach_slice` and
+    `reduce_points`) that work the same way on CPUs, multi-threaded CPUs, and
+    GPUs. Runtime performance on CPUs and GPUs matches the previous release.
+  - Data in a `Field` is now indexed in the order `[v, i, j, h]`, and every
+    parent array has a 5-dimensional shape, with one axis used to store struct
+    fields. Code that indexes into `parent(field)` arrays must be updated.
+  - Several old names (`AbstractData`, `IJFH`, `IJHF`) are available as aliases,
+    and `[i, j, f, v, h]` indexing is still allowed for backward compatibility.
+
 v0.14.55
 -------
 

NEWS.md

@@ -4,13 +4,84 @@
 CurrentModule = ClimaCore
 ```
 
+## Data layouts
+
 ```@docs
-DataLayouts
+DataLayouts.DataLayout
 DataLayouts.DataF
+DataLayouts.VIJHWithF
 DataLayouts.VIJFH
 DataLayouts.VIJHF
 DataLayouts.VIH1
 DataLayouts.IH1JH2
+```
+
+## Layout properties
+
+```@docs
+DataLayouts.layout_type
+DataLayouts.parent_type
+DataLayouts.f_dim
+DataLayouts.shape_params
+DataLayouts.inferred_size
+DataLayouts.has_inferred_size
+DataLayouts.vijh_params
+DataLayouts.nlevels
+DataLayouts.nquadpoints
+DataLayouts.nelems
+DataLayouts.ncomponents
+DataLayouts.layout_constructor
+DataLayouts.rebuild
+DataLayouts.reassign
+```
+
+## Data scopes
+
+```@docs
+DataLayouts.DataScope
+DataLayouts.ThisThread
+DataLayouts.ThisThreadPool
+DataLayouts.partition
+DataLayouts.is_subscope
+DataLayouts.num_threads
+DataLayouts.num_partitions
+DataLayouts.thread_rank
+DataLayouts.partition_rank
+DataLayouts.parallelize_over
+DataLayouts.synchronize
+DataLayouts.scoped_array
+DataLayouts.scoped_static_array
+DataLayouts.strided_access
+DataLayouts.subscope_indices
+```
+
+## Loops and reductions
+
+```@docs
+DataLayouts.each_slice_index
+DataLayouts.slice_subscope
+DataLayouts.foreach_slice
+DataLayouts.foreach_point
+DataLayouts.foreach_level
+DataLayouts.foreach_slab
+DataLayouts.foreach_column
+DataLayouts.reduce_points
+DataLayouts.column_reduce!
+```
+
+## Masks
+
+```@docs
+DataLayouts.DataMask
+DataLayouts.NoMask
+DataLayouts.IJHMask
+DataLayouts.set_mask_maps!
+DataLayouts.should_compute
+```
+
+## Struct storage
+
+```@docs
 DataLayouts.bitcast_struct
 DataLayouts.default_basetype
 DataLayouts.check_basetype
@@ -19,4 +90,14 @@ DataLayouts.num_basetypes
 DataLayouts.struct_field_view
 DataLayouts.set_struct!
 DataLayouts.get_struct
+DataLayouts.view_struct
+```
+
+## Broadcasting
+
+```@docs
+DataLayouts.DataStyle
+DataLayouts.LazyDataLayout
+DataLayouts.layout_args
+DataLayouts.modify_args
 ```

docs/src/APIs/datalayouts_api.md

@@ -9,7 +9,6 @@ Topologies.dss_transform
 Topologies.dss_transform!
 Topologies.dss_untransform!
 Topologies.dss_untransform
-Topologies.dss_local_vertices!
 Topologies.dss_local!
 Topologies.dss_local_ghost!
 Topologies.dss_ghost!

docs/src/APIs/dss_api.md

@@ -5,6 +5,7 @@ CurrentModule = ClimaCore
 ```
 
 ```@docs
+Utilities.stable_view
 Utilities.unionall_type
 Utilities.replace_type_parameter
 Utilities.fieldtype_vals
@@ -13,6 +14,7 @@ Utilities.is_inferred_type
 Utilities.return_type
 Utilities.unsafe_eltype
 Utilities.safe_eltype
+Utilities.safe_mapreduce
 ```
 
 ## Utilities.PlusHalf

docs/src/APIs/utilities_api.md

@@ -69,7 +69,7 @@ function ClimaCore.DebugOnly.post_op_callback(result, args...; kwargs...)
 end
 
 FT = Float64
-data = ClimaCore.DataLayouts.VIJFH{FT}(Array{FT}, zeros; Nv=5, Nij=2, Nh=2)
+data = ClimaCore.DataLayouts.VIJFH{FT, 5, 2, 2, 2}(Array{FT})
 @. data = NaN
 ClimaCore.DebugOnly.call_post_op_callback() = false # hide
 ```
@@ -221,8 +221,10 @@ function ClimaCore.DebugOnly.post_op_callback(result, args...; kwargs...)
 end
 
 FT = Float64
-data = ClimaCore.DataLayouts.VIJFH{FT}(Array{FT}, zeros; Nv=5, Nij=2, Nh=2)
-x = ClimaCore.DataLayouts.VIJFH{FT}(Array{FT}, zeros; Nv=5, Nij=2, Nh=2)
+data = ClimaCore.DataLayouts.VIJFH{FT, 5, 2, 2, 2}(Array{FT})
+x = ClimaCore.DataLayouts.VIJFH{FT, 5, 2, 2, 2}(Array{FT})
+fill!(parent(data), 0)
+fill!(parent(x), 0)
 parent(x)[1] = NaN # emulate incorrect initialization
 @. data = x + 1
 # Let's see what happened