From 8971469516fc39e32e19e5c04866fe6eee69ee83 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Wed, 8 Jul 2026 18:00:47 -0400 Subject: [PATCH 01/33] Add network SoA worktree handoff and Phase 0 spec scaffold Documents the decision to pause GPU network buffers until CPU network SoA adoption. Provides agent rules, session starting prompt, and Phase 0 design checklist for local/cpu-network-soa branched from master. --- AGENTS.md | 39 +++++++ GROK-NETWORK-SOA.md | 138 ++++++++++++++++++++++ doc/network-soa-phase0.md | 235 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 412 insertions(+) create mode 100644 AGENTS.md create mode 100644 GROK-NETWORK-SOA.md create mode 100644 doc/network-soa-phase0.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..b542a101d7 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,39 @@ +# Agent rules — NEURON CPU network SoA adoption + +Full handoff and starting prompt: **`GROK-NETWORK-SOA.md`** (read on new sessions). + +## Workspace + +- Repo worktree: `~/neuron/cpu_net_soa` (branch `local/cpu-network-soa`, tracks `origin/master`). +- Primary git object store: `~/neuron/nrngpu` — commit from **this** worktree cwd. +- Sibling GPU track (paused network buffers): `~/neuron/nrngpu` @ `local/gpu-native-qualification`. + +## Build + +```bash +source ~/neuron/bin/nrnenv nrngpu build-cpu-net-soa # create on first session if missing +# or: mkdir -p ~/neuron/cpu_net_soa/build && cd build && cmake .. -DNRN_ENABLE_GPU=OFF ... +``` + +Prefer CPU-only or default GPU-off builds until integration explicitly needs GPU mirrors. + +## Scope (this branch) + +- Network SoA: `Point_process`, `NetCon`, `PreSyn`, `weights`, `SelfEvent` in `neuron::container` style. +- HOC wrappers as permutation-stable handles over backing store — **not** a second pointer graph. +- **Out of scope:** Stage 2/3 GPU `net_buf_receive`, ringtest GPU network buffers (resume after SoA merges to master). + +## Execute, don’t delegate + +Run builds and tests yourself (`ctest`, ringtest CPU spike parity). Do not tell the user what to run unless blocked. + +## Key references + +| Topic | Path | +|-------|------| +| Handoff | `GROK-NETWORK-SOA.md` | +| Phase 0 scaffold | `doc/network-soa-phase0.md` | +| Node/mechanism SoA pattern | `src/neuron/container/soa_container.hpp`, `data_handle.hpp` | +| PreSyn `thvar_` handle (prototype) | `src/nrncvode/netcon.h` | +| CoreNEURON layout reference | `src/coreneuron/sim/multicore.hpp`, `network/netcon.hpp` | +| nrncore export (prior art) | `src/nrniv/nrncore_write/` | \ No newline at end of file diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md new file mode 100644 index 0000000000..eb4789d6b5 --- /dev/null +++ b/GROK-NETWORK-SOA.md @@ -0,0 +1,138 @@ +# Grok handoff: NEURON CPU network SoA (`cpu_net_soa`) + +Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_soa`. + +--- + +## Why a separate worktree + +Network SoA is a focused CPU/infrastructure PR. It should **not** carry the GPU-native qualification commit stack (`local/gpu-native-qualification`). + +| Worktree | Branch | Purpose | +|----------|--------|---------| +| `~/neuron/cpu_net_soa` | `local/cpu-network-soa` | Network SoA → PR to **master** | +| `~/neuron/nrngpu` | `local/gpu-native-qualification` | Mechanism GPU, Stage 1 buffer plumbing; Stages 2–3 **paused** until SoA lands | + +Created with: + +```bash +cd ~/neuron/nrngpu +git worktree add ~/neuron/cpu_net_soa -b local/cpu-network-soa origin/master +``` + +--- + +## Decision record (2026-07-08) + +- **Pause** ringtest GPU network buffers (Stage 2 codegen, Stage 3 wire-up). +- **Discard** unstaged Stage 2 WIP on `gpu-native-qualification` (reverted). +- **Adopt** CoreNEURON-shaped network SoA on NEURON CPU first, with HOC types as wrappers over `data_handle` / `soa` backing store (same model as nodes/mechanisms). +- **Resume** GPU network buffers after SoA merges to master; gpu-native branch rebases then. + +--- + +## North star + +One CPU backing store for integration-hot network data: + +- `std::vector` columns + permutation indices +- `data_handle` / `owning_handle` stable across permute +- `NrnThread` = lightweight slice (`offset` + count) into contiguous regions — **no duplicate CPU copies** +- HOC extras (`Object*`, recording, …) in sidecars keyed by handle + +CoreNEURON layout is the **integration reference**; NEURON adds live create/delete/permute + interpreter compatibility. + +--- + +## Architecture sketch + +```text +HOC / Python (NetCon, PointProcess, PreSyn API) + │ + ▼ +Wrappers (handle / owning_handle into network SoA) + │ + ▼ +neuron::container::soa<...> (weights, pntproc, netcon, presyn, …) + │ + ▼ +psolve hot path (deliver, threshold, fanout) — index-based + │ + ▼ +(future) GPU upload of same columns after merge with gpu-native track +``` + +--- + +## Phase plan (summary) + +| Phase | Content | Gate | +|-------|---------|------| +| **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec reviewed | +| **1** | `Point_process` + `weights` SoA | handles survive permute | +| **2** | `NetCon` SoA, HOC `weight()` → flat weights | CPU ringtest delivery | +| **3** | `PreSyn` `nc_index`/`nc_cnt` fanout | CPU spike parity @ 100 ms | +| **4** | `SelfEvent` indices, `pnt_receive(weight_index)` | ExpSyn @ 1.025 ms CPU | +| **5** | (gpu-native track) net buffers Stages 2–3 | GPU parity | + +--- + +## Parallel work (not blocking this branch) + +On `local/gpu-native-qualification`: + +- Mechanism GPU gates A–E (ringtest mods). +- `-no-netcon` ringtest + APCount for NEURON vs CoreNEURON perf without `NET_RECEIVE`. + +--- + +## Build (first session) + +```bash +cd ~/neuron/cpu_net_soa +mkdir -p build && cd build +cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install -DNRN_ENABLE_MPI=ON +ninja && ninja install +source ~/neuron/bin/nrnenv nrngpu build-cpu-net-soa # add nrnenv alias if helpful +``` + +Adjust `nrnenv` path to `~/neuron/cpu_net_soa/build` — agent may create `build-cpu-net-soa` on first configure. + +--- + +## Tests (CPU gates) + +```bash +# After Phase 3+ +cd build && ctest -R ringtest --output-on-failure + +# Spike parity (single host) +# mpiexec -n 1 ./path/to/special -nobanner ringtest.hoc +``` + +--- + +## Starting prompt (paste into new `cpu_net_soa` session) + +``` +Read ~/neuron/cpu_net_soa/GROK-NETWORK-SOA.md, AGENTS.md, and doc/network-soa-phase0.md. + +Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa (from origin/master). +Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU track; network buffers paused). + +Task: Write the Phase 0 design spec for network SoA — field tags, handle types, +per-thread slicing, HOC sidecar policy, invalidation rules. Then scaffold Phase 1 +(Point_process + weights container) under src/neuron/container/network/. + +Follow the node/mechanism DataHandle pattern (soa_container.hpp, data_handle.hpp). +PreSyn thvar_ data_handle is the existing network prototype. CoreNEURON multicore.hpp +is the integration layout reference. + +Do not implement GPU net_buf_receive on this branch. +``` + +--- + +## Old GPU-native context + +Stage 1 `NetReceiveBuffer` on gpu-native (`1f05cbc19`) remains valid **pattern**; indexing will align with SoA after merge. See `~/neuron/nrngpu/GROK-GPU-NATIVE.md`. \ No newline at end of file diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md new file mode 100644 index 0000000000..5f3ddc99db --- /dev/null +++ b/doc/network-soa-phase0.md @@ -0,0 +1,235 @@ +# Network SoA — Phase 0 design spec (draft scaffold) + +**Status:** Draft for first implementation session. Expand into reviewable spec before Phase 1 coding. + +**Branch:** `local/cpu-network-soa` +**Base:** `origin/master` +**Blocked downstream:** GPU network buffers on `local/gpu-native-qualification` (Stages 2–3) + +--- + +## 1. Problem + +NEURON network objects (`Point_process`, `NetCon`, `PreSyn`, `SelfEvent`) use pointer-heavy layouts suited to HOC interactivity. CoreNEURON GPU integration assumes flat per-thread arrays (`pntprocs`, `weights`, `netcons`, `presyns`) with integer indices. + +Bridging with per-event shims on the legacy layout does not scale. **One SoA backing store** with HOC wrappers as handles matches the existing node/mechanism container model. + +--- + +## 2. Goals + +- Single CPU copy in `std::vector` columns; permutation-stable `data_handle` / `owning_handle`. +- `NrnThread` holds offsets into contiguous thread slices (no per-thread heap graphs). +- HOC API unchanged at the interpreter boundary (`weight()`, `loc()`, … read/write SoA). +- CPU `psolve` hot path uses indices (CoreNEURON-shaped), not `double* weight_` / `dil_` walks. +- GPU track can upload the same columns after SoA merges (out of scope for Phase 0–4). + +## 3. Non-goals (Phase 0–4) + +- GPU `net_buf_receive` codegen or Stage 3 runtime wire-up. +- Full `InputPreSyn` / MPI multisend redesign (defer to post–Phase 3 milestone). +- Removing HOC `Object*` identity — sidecars remain. + +--- + +## 4. Reference layouts + +### 4.1 CoreNEURON (integration hot) + +| Entity | Key fields | +|--------|------------| +| `Point_process` | `_i_instance`, `_type`, `_tid` | +| `weights` | flat `double[n_weight]` | +| `NetCon` | `target_`, `u.weight_index_`, `delay_`, `active_` | +| `PreSyn` | `nc_index_`, `nc_cnt_`, `thvar_index_`, `threshold_`, `gid_` | +| `SelfEvent` | `target_`, `weight_index_`, `flag_` | + +See `src/coreneuron/sim/multicore.hpp`, `src/coreneuron/network/netcon.hpp`. + +### 4.2 NEURON today (to replace on hot path) + +| Entity | Hot-path problem | +|--------|------------------| +| `Point_process` | `sec`, `node`, `prop`, `ob`, … pointers | +| `NetCon` | `double* weight_` heap per netcon | +| `PreSyn` | `NetConPList dil_`, fat HOC fields | +| `SelfEvent` | `double* weight_` | + +See `src/nrnoc/section_fwd.hpp`, `src/nrncvode/netcon.h`. + +### 4.3 NEURON precedent (already SoA-friendly) + +- Nodes / mechanisms: `src/neuron/container/soa_container.hpp`, `data_handle.hpp` +- `PreSyn::thvar_` as `data_handle`; GPU threshold uses `thvar_row` vs `nt->_node_data_offset` (`src/nrncvode/netcvode.cpp`) + +--- + +## 5. Proposed containers (to decide in Phase 0) + +### 5.1 Storage location + +**Proposal:** extend `neuron::model()` with network storages (or `neuron::container::network::` namespace), analogous to `node_data()` and mechanism storage. + +```cpp +// Illustrative — names TBD in spec +struct Model { + container::Node::storage& node_data(); + container::network::PointProcessStorage& point_processes(); + container::network::WeightStorage& weights(); + container::network::NetConStorage& netcons(); + container::network::PreSynStorage& presyns(); +}; +``` + +### 5.2 Field tags (integration columns) + +**Point_process row** + +| Tag | Type | Notes | +|-----|------|-------| +| `Instance` | `int` | mechanism SoA row (`_i_instance`) | +| `MechType` | `int` | `_type` | +| `ThreadId` | `int` | `_tid` | + +**Weight storage** + +| Tag | Type | Notes | +|-----|------|-------| +| `Value` | `double` | flat array; NetCon owns `weight_index` base | + +**NetCon row** + +| Tag | Type | Notes | +|-----|------|-------| +| `TargetPnt` | `int` | index into point_process SoA | +| `WeightIndex` | `int` | into weights | +| `WeightCount` | `int` | `pnt_receive_size` | +| `Delay` | `double` | | +| `Active` | `bool`/`int` | | +| `PreSynIndex` | `int` | source presyn row, or -1 | + +**PreSyn row** + +| Tag | Type | Notes | +|-----|------|-------| +| `ThVar` | `data_handle` or `int` row | prefer handle if already modern | +| `Threshold` | `double` | | +| `Gid` | `int` | | +| `NcIndex` | `int` | start index in netcon array | +| `NcCount` | `int` | fanout count | + +**SelfEvent row** (if queued structurally; else event pool) + +| Tag | Type | Notes | +|-----|------|-------| +| `TargetPnt` | `int` | | +| `WeightIndex` | `int` | | +| `Flag` | `double` | | + +### 5.3 HOC sidecars (not in integration columns) + +Keyed by `owning_handle` or stable id: + +- `Object* ob` +- `IvocVect* tvec_`, recording state +- `HocCommand* stmt_` +- Cached `Section*` (invalidate on `tree_changed`) + +--- + +## 6. Thread slicing + +Mirror node model: + +```text +Global SoA: [ row 0 | row 1 | ... | row N-1 ] + |← thread 0 →|← thread 1 →| +``` + +Per `NrnThread` (new fields, names TBD): + +- `_pntproc_offset`, `_pntproc_count` +- `_netcon_offset`, `_netcon_count` +- `_presyn_offset`, `_presyn_count` +- `_weight_offset`, `_weight_count` (if weights partitioned per thread) + +Permutation chosen so each thread region stays contiguous after `nrn_threads_create` / cell partitioning. + +--- + +## 7. Handle API (wrappers) + +| HOC-facing type | Wrapper holds | Example accessor | +|-----------------|---------------|------------------| +| `Point_process` | `PointProcess::owning_handle` | `prop()` → resolve via `_type` + `_i_instance` | +| `NetCon` | `NetCon::owning_handle` | `weight(i)` → `weights[weight_index+i]` | +| `PreSyn` | `PreSyn::owning_handle` | `threshold` → SoA column | + +**CPU delivery signature (target):** + +```cpp +pnt_receive[type](pnt_handle, weight_index, flag); +``` + +Align with CoreNEURON; migrate from `(Point_process*, double* _args, double flag)`. + +--- + +## 8. Invalidation and sorting + +- Topology / `define_shape` → `mark_as_unsorted()` on network containers (like `node_data()`). +- `tree_changed` → refresh sidecar caches; optional full network reorder pass. +- `psolve` / integration entry: `nrn_ensure_model_data_are_sorted()` includes network containers when fanout order matters. +- **Freeze policy:** same frozen-token pattern as mechanisms during sorted integration phases (TBD). + +--- + +## 9. Migration strategy + +| Step | Action | +|------|--------| +| M1 | Dual-write: create SoA row when creating legacy object | +| M2 | Dual-read: hot path reads SoA; HOC still works via wrapper | +| M3 | Remove hot-path pointer fields (`weight_`, `dil_` iteration) | +| M4 | Sidecars only for interpreter | + +Order: **weights + Point_process → NetCon → PreSyn → SelfEvent**. + +--- + +## 10. Phase 0 deliverables (checklist) + +- [ ] Final field tag list per container (section 5.2) +- [ ] `Model` ownership and file layout (`src/neuron/container/network/`) +- [ ] Handle typedef names (`PointProcess::handle`, …) +- [ ] Thread offset fields on `NrnThread` / multicore +- [ ] Sidecar map design + invalidation rules +- [ ] `pnt_receive` signature migration plan +- [ ] SaveState / BBSaveState impact (minimal note for Phase 2+) +- [ ] Test plan: unit (permute), integration (ringtest CPU spikes) + +--- + +## 11. Phase 1 first code (after spec sign-off) + +``` +src/neuron/container/network/ + point_process.hpp # field tags, storage typedef, handle API + point_process_data.cpp # storage impl hooks + weights.hpp +``` + +Hook point: point process allocation in existing `nrn_point_process` / mechanism insert path — allocate SoA row alongside today's `Prop`. + +--- + +## 12. Open questions + +1. Global vs per-thread weight pool — CoreNEURON uses per-thread `weights`; confirm for NEURON MPI. +2. When to permute network SoA relative to mechanism permute (`nrn_sort_mech_data`). +3. `NetCon` as `DiscreteEvent` subclass — keep for queue compatibility or separate event index? +4. `InputPreSyn` split — Phase 3 or later MPI milestone? + +--- + +*Expand this document into the authoritative Phase 0 spec in the first `cpu_net_soa` session.* \ No newline at end of file From 6d8fc4e6b3be07daa17233cd2f82612ad79e7e88 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Sun, 12 Jul 2026 07:30:19 -0400 Subject: [PATCH 02/33] Phase 0 network SoA design + Phase 1 PointProcess/Weight scaffold Expand doc/network-soa-phase0.md into the authoritative field-tag, handle, thread-slice, sidecar, and invalidation spec. Add neuron::container::network PointProcess and Weight soa storages on Model (Node-style handles), wire find_container_info/unsorted callbacks, and unit-test that handles survive permutation. --- GROK-NETWORK-SOA.md | 4 +- doc/network-soa-phase0.md | 549 +++++++++++++----- src/neuron/container/memory_usage.hpp | 12 + .../container/network/point_process.hpp | 116 ++++ src/neuron/container/network/weights.hpp | 79 +++ src/neuron/model_data.hpp | 30 + src/nrniv/memory_usage.cpp | 14 +- src/nrnoc/container.cpp | 15 + test/CMakeLists.txt | 1 + test/unit_tests/container/network.cpp | 157 +++++ 10 files changed, 844 insertions(+), 133 deletions(-) create mode 100644 src/neuron/container/network/point_process.hpp create mode 100644 src/neuron/container/network/weights.hpp create mode 100644 test/unit_tests/container/network.cpp diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index eb4789d6b5..40de2e3e10 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -68,8 +68,8 @@ psolve hot path (deliver, threshold, fanout) — index-based | Phase | Content | Gate | |-------|---------|------| -| **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec reviewed | -| **1** | `Point_process` + `weights` SoA | handles survive permute | +| **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec done (authoritative) | +| **1** | `Point_process` + `weights` SoA scaffold under `src/neuron/container/network/` | handles survive permute (unit test) | | **2** | `NetCon` SoA, HOC `weight()` → flat weights | CPU ringtest delivery | | **3** | `PreSyn` `nc_index`/`nc_cnt` fanout | CPU spike parity @ 100 ms | | **4** | `SelfEvent` indices, `pnt_receive(weight_index)` | ExpSyn @ 1.025 ms CPU | diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index 5f3ddc99db..9f22da3f63 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -1,34 +1,48 @@ -# Network SoA — Phase 0 design spec (draft scaffold) +# Network SoA — Phase 0 design spec -**Status:** Draft for first implementation session. Expand into reviewable spec before Phase 1 coding. - -**Branch:** `local/cpu-network-soa` -**Base:** `origin/master` -**Blocked downstream:** GPU network buffers on `local/gpu-native-qualification` (Stages 2–3) +**Status:** Authoritative for `local/cpu-network-soa` (Phase 0 complete → Phase 1 scaffold). +**Branch:** `local/cpu-network-soa` (from `origin/master`) +**Sibling (paused):** `~/neuron/nrngpu` @ `local/gpu-native-qualification` — GPU network buffers Stages 2–3 +**Handoff:** `GROK-NETWORK-SOA.md` --- ## 1. Problem -NEURON network objects (`Point_process`, `NetCon`, `PreSyn`, `SelfEvent`) use pointer-heavy layouts suited to HOC interactivity. CoreNEURON GPU integration assumes flat per-thread arrays (`pntprocs`, `weights`, `netcons`, `presyns`) with integer indices. +NEURON network objects (`Point_process`, `NetCon`, `PreSyn`, `SelfEvent`) use pointer-heavy layouts suited to HOC interactivity: + +| Entity | Hot-path problem | +|--------|------------------| +| `Point_process` | `sec`, `node`, `prop`, `ob`, `presyn_`, `_vnt` pointers | +| `NetCon` | per-NetCon `double* weight_` heap allocation | +| `PreSyn` | `NetConPList dil_` + fat HOC fields (`Object*`, recording, …) | +| `SelfEvent` | `double* weight_` into NetCon heap | + +CoreNEURON GPU / multicore integration assumes **flat per-thread arrays** with integer indices (`pntprocs`, `weights`, `netcons`, `presyns`). See `src/coreneuron/sim/multicore.hpp`, `src/coreneuron/network/netcon.hpp`. + +Bridging with per-event shims on the legacy layout does not scale. **One SoA backing store** with HOC wrappers as permutation-stable handles matches the existing node/mechanism container model (`src/neuron/container/soa_container.hpp`, `data_handle.hpp`). -Bridging with per-event shims on the legacy layout does not scale. **One SoA backing store** with HOC wrappers as handles matches the existing node/mechanism container model. +**Existing network prototype:** `PreSyn::thvar_` is already `neuron::container::data_handle` (`src/nrncvode/netcon.h`). Threshold evaluation and GPU threshold use stable handles / row offsets relative to `NrnThread::_node_data_offset`. --- ## 2. Goals -- Single CPU copy in `std::vector` columns; permutation-stable `data_handle` / `owning_handle`. -- `NrnThread` holds offsets into contiguous thread slices (no per-thread heap graphs). -- HOC API unchanged at the interpreter boundary (`weight()`, `loc()`, … read/write SoA). -- CPU `psolve` hot path uses indices (CoreNEURON-shaped), not `double* weight_` / `dil_` walks. -- GPU track can upload the same columns after SoA merges (out of scope for Phase 0–4). +1. Single CPU copy of integration-hot network data in `std::vector` columns. +2. Permutation-stable `data_handle` / `owning_handle` (same lifetime model as `Node`). +3. `NrnThread` (and `neuron::cache::Thread`) hold **offsets + counts** into contiguous global slices — no per-thread duplicate heap graphs. +4. HOC / Python API unchanged at the interpreter boundary (`weight()`, `loc()`, `NetCon.active`, …). +5. CPU `psolve` hot path becomes index-based (CoreNEURON-shaped), not `double* weight_` / `dil_` walks. +6. After SoA merges to master, the GPU track can upload the same columns (out of scope here). -## 3. Non-goals (Phase 0–4) +## 3. Non-goals (Phases 0–4 on this branch) -- GPU `net_buf_receive` codegen or Stage 3 runtime wire-up. -- Full `InputPreSyn` / MPI multisend redesign (defer to post–Phase 3 milestone). -- Removing HOC `Object*` identity — sidecars remain. +| Out of scope | Resume when | +|--------------|-------------| +| GPU `net_buf_receive` codegen / Stage 3 wire-up | SoA on master; rebase `gpu-native-qualification` | +| Full `InputPreSyn` / MPI multisend redesign | Post–Phase 3 MPI milestone | +| Removing HOC `Object*` identity | Never — sidecars remain | +| Changing public HOC/Python method signatures | Not required | --- @@ -36,200 +50,475 @@ Bridging with per-event shims on the legacy layout does not scale. **One SoA bac ### 4.1 CoreNEURON (integration hot) +From `coreneuron::NrnThread` and network headers: + | Entity | Key fields | |--------|------------| | `Point_process` | `_i_instance`, `_type`, `_tid` | -| `weights` | flat `double[n_weight]` | +| `weights` | flat `double[n_weight]`; `NetCon.u.weight_index_` base | | `NetCon` | `target_`, `u.weight_index_`, `delay_`, `active_` | | `PreSyn` | `nc_index_`, `nc_cnt_`, `thvar_index_`, `threshold_`, `gid_` | | `SelfEvent` | `target_`, `weight_index_`, `flag_` | -See `src/coreneuron/sim/multicore.hpp`, `src/coreneuron/network/netcon.hpp`. +Weights for one NetCon are **contiguous** at `[weight_index, weight_index + pnt_receive_size)`. -### 4.2 NEURON today (to replace on hot path) +### 4.2 NEURON node/mechanism precedent -| Entity | Hot-path problem | -|--------|------------------| -| `Point_process` | `sec`, `node`, `prop`, `ob`, … pointers | -| `NetCon` | `double* weight_` heap per netcon | -| `PreSyn` | `NetConPList dil_`, fat HOC fields | -| `SelfEvent` | `double* weight_` | +```text +owning_handle → owns a row; destructor frees row +handle → non-owning; stable across permute; dies when owner dies +data_handle → erases field identity; used for pointers into columns +``` -See `src/nrnoc/section_fwd.hpp`, `src/nrncvode/netcon.h`. +Storage lives under `neuron::model()` (`Model::node_data()`, `Model::mechanism_data(type)`). +Thread slice: `NrnThread::_node_data_offset` + `neuron::cache::Thread::{node,mechanism}_offset`. +Sort gate: `nrn_ensure_model_data_are_sorted()` freezes tokens, permutes, rebuilds `neuron::cache::model`. -### 4.3 NEURON precedent (already SoA-friendly) +### 4.3 PreSyn `thvar_` prototype -- Nodes / mechanisms: `src/neuron/container/soa_container.hpp`, `data_handle.hpp` -- `PreSyn::thvar_` as `data_handle`; GPU threshold uses `thvar_row` vs `nt->_node_data_offset` (`src/nrncvode/netcvode.cpp`) +- Construction takes `data_handle` for the watched voltage (or related variable). +- `notify_when_handle_dies` disconnects when the underlying node dies. +- Do **not** re-introduce raw `double*` for threshold sources in new code. --- -## 5. Proposed containers (to decide in Phase 0) +## 5. Containers and field tags + +### 5.1 Ownership and file layout -### 5.1 Storage location +**Decision:** Network storages are members of `neuron::Model`, analogous to `m_node_data`. -**Proposal:** extend `neuron::model()` with network storages (or `neuron::container::network::` namespace), analogous to `node_data()` and mechanism storage. +``` +src/neuron/container/network/ + point_process.hpp # Phase 1 — tags, storage, handle API + weights.hpp # Phase 1 + netcon.hpp # Phase 2 + presyn.hpp # Phase 3 + self_event.hpp # Phase 4 (if queued structurally) +``` ```cpp -// Illustrative — names TBD in spec -struct Model { - container::Node::storage& node_data(); - container::network::PointProcessStorage& point_processes(); - container::network::WeightStorage& weights(); - container::network::NetConStorage& netcons(); - container::network::PreSynStorage& presyns(); -}; +// neuron::Model (illustrative) +container::network::PointProcess::storage& point_processes(); +container::network::Weight::storage& weights(); +container::network::NetCon::storage& netcons(); // Phase 2 +container::network::PreSyn::storage& presyns(); // Phase 3 +``` + +Namespace root: `neuron::container::network`. +Each entity is a nested namespace with `field::*` tags, `storage`, `handle`, `owning_handle` — same shape as `neuron::container::Node`. + +### 5.2 Point_process (Phase 1) + +**Integration columns** (CoreNEURON-compatible): + +| Tag | C++ type | Default | Meaning | +|-----|----------|---------|---------| +| `field::Instance` | `int` | `-1` | Mechanism SoA row (`_i_instance`) | +| `field::MechType` | `int` | `-1` | Mechanism type (`_type`) | +| `field::ThreadId` | `int` | `-1` | Owning `NrnThread` id (`_tid`) | + +**Handle typedefs:** + +| Name | Type | +|------|------| +| `PointProcess::handle` | `handle_interface>` | +| `PointProcess::owning_handle` | `handle_interface>` | + +**Accessors on `handle_interface`:** `instance()`, `mech_type()`, `thread_id()`, plus `*_handle()` returning `data_handle` where useful for debugging / HOC bridges. + +**Not in SoA (sidecars / dual-write legacy fields):** see §7. + +### 5.3 Weights (Phase 1) + +**Integration columns:** + +| Tag | C++ type | Default | Meaning | +|-----|----------|---------|---------| +| `field::Value` | `double` | `0.0` | One weight scalar | + +One **row** = one scalar weight entry. A NetCon with `cnt = pnt_receive_size[type]` owns a **contiguous block** of `cnt` rows. + +| Name | Type | +|------|------| +| `Weight::handle` / `Weight::owning_handle` | same pattern as Node | + +**Contiguity invariant (CoreNEURON-compatible):** + +```text +weights[weight_index + k] for k in [0, weight_count) ``` -### 5.2 Field tags (integration columns) +- Allocation API (Phase 1→2) must allocate **blocks** of size `weight_count` as consecutive rows. +- Permutation of the weight container must either: + 1. **Block-permute** (move whole NetCon groups and rewrite `NetCon::WeightIndex`), or + 2. **Repack** at sort time into contiguous groups (preferred at `nrn_ensure_model_data_are_sorted`). +- Freeing a NetCon frees its entire block (swap-with-last **block** or hole + repack). + +**Why not one SoA row per NetCon with runtime array dim?** Weight counts differ by target mechanism type (`pnt_receive_size`). A flat pool matches CoreNEURON export (`nrncore_write`) and GPU upload. -**Point_process row** +**Hot-path access:** integer `weight_index` base (current row of first weight in the group), **not** a stable identifier, so delivery stays CoreNEURON-shaped. After any weight permute/repack, all `NetCon` weight indices are remapped. HOC `weight(i)` may use `data_handle` obtained from the base row + array shift once the group is a single logical array, or `weights.get(weight_index + i)`. -| Tag | Type | Notes | -|-----|------|-------| -| `Instance` | `int` | mechanism SoA row (`_i_instance`) | -| `MechType` | `int` | `_type` | -| `ThreadId` | `int` | `_tid` | +### 5.4 NetCon (Phase 2 — tags fixed now) -**Weight storage** +| Tag | C++ type | Default | Meaning | +|-----|----------|---------|---------| +| `field::Target` | `int` | `-1` | Row in `PointProcess` storage (or sentinel) | +| `field::WeightIndex` | `int` | `-1` | Base row in `Weight` storage | +| `field::WeightCount` | `int` | `0` | `pnt_receive_size` for target type | +| `field::Delay` | `double` | `1.0` | Delivery delay (ms) | +| `field::Active` | `int` | `1` | `bool` as `int` (GPU-friendly, CoreNEURON style) | +| `field::SrcPreSyn` | `int` | `-1` | Source PreSyn row, or `-1` if none | -| Tag | Type | Notes | -|-----|------|-------| -| `Value` | `double` | flat array; NetCon owns `weight_index` base | +**Optional later:** `field::ObjectId` only if HOC identity must be recovered from a row without a sidecar map. -**NetCon row** +**Event-queue identity:** `NetCon` remains a `DiscreteEvent` subclass for queue compatibility through Phase 2–3. The SoA row is the data plane; the C++ object is the control/HOC plane during dual-write. Long term the queued payload may shrink to `(type, row)` — not required for Phase 2 gate. -| Tag | Type | Notes | -|-----|------|-------| -| `TargetPnt` | `int` | index into point_process SoA | -| `WeightIndex` | `int` | into weights | -| `WeightCount` | `int` | `pnt_receive_size` | -| `Delay` | `double` | | -| `Active` | `bool`/`int` | | -| `PreSynIndex` | `int` | source presyn row, or -1 | +### 5.5 PreSyn (Phase 3 — tags fixed now) -**PreSyn row** +| Tag | C++ type | Default | Meaning | +|-----|----------|---------|---------| +| `field::Threshold` | `double` | `10.0` | Spike threshold | +| `field::Gid` | `int` | `-1` | Output gid or −1 | +| `field::NcIndex` | `int` | `-1` | Start of fanout range in NetCon order array | +| `field::NcCount` | `int` | `0` | Fanout count | +| `field::OutputIndex` | `int` | `0` | CoreNEURON `output_index_` | +| `field::ThVarRow` | `int` | `-1` | Optional denormalized node-voltage row for threshold scan; **canonical source remains `data_handle` in sidecar / dual-write `thvar_`** | +| `field::ThreadId` | `int` | `-1` | Owning thread | -| Tag | Type | Notes | -|-----|------|-------| -| `ThVar` | `data_handle` or `int` row | prefer handle if already modern | -| `Threshold` | `double` | | -| `Gid` | `int` | | -| `NcIndex` | `int` | start index in netcon array | -| `NcCount` | `int` | fanout count | +**Fanout model (replaces `dil_`):** + +```text +Global NetCon order array (or NetCon SoA already sorted by source): + netcons[NcIndex .. NcIndex+NcCount) + +On spike: for i in [0, NcCount): deliver netcons[NcIndex+i] +``` -**SelfEvent row** (if queued structurally; else event pool) +Matches CoreNEURON `nc_index_` / `nc_cnt_` into `netcon_in_presyn_order_`. At sort time, NetCons are (re)ordered by source PreSyn so ranges are contiguous. -| Tag | Type | Notes | -|-----|------|-------| -| `TargetPnt` | `int` | | -| `WeightIndex` | `int` | | -| `Flag` | `double` | | +**Threshold source:** keep `data_handle` (today’s `thvar_`) in the dual-write / sidecar plane. Optionally cache `ThVarRow` when the handle refers to node voltage for vectorized threshold checks. -### 5.3 HOC sidecars (not in integration columns) +### 5.6 SelfEvent (Phase 4) -Keyed by `owning_handle` or stable id: +SelfEvents are short-lived queue items. Prefer **not** a long-lived SoA of all SelfEvents; instead: -- `Object* ob` -- `IvocVect* tvec_`, recording state -- `HocCommand* stmt_` -- Cached `Section*` (invalidate on `tree_changed`) +| Field on event / pool object | Type | Meaning | +|------------------------------|------|---------| +| `target_pnt` | `int` | PointProcess row | +| `weight_index` | `int` | Into weights (same base as NetCon or NULL weights) | +| `flag` | `double` | NET_RECEIVE flag | + +If a structural pool is useful later, tags match the above. Gate is `pnt_receive(weight_index)` migration (§9). + +### 5.7 Tag summary by phase + +| Phase | Container | Gate | +|-------|-----------|------| +| **0** | Spec (this doc) | Spec reviewed | +| **1** | `PointProcess` + `Weight` | Handles survive permute; unit test | +| **2** | `NetCon` SoA; HOC `weight()` → flat weights | CPU ringtest delivery | +| **3** | `PreSyn` fanout `NcIndex`/`NcCount` | CPU spike parity @ 100 ms | +| **4** | SelfEvent indices; `pnt_receive` by weight index | ExpSyn @ 1.025 ms CPU | +| **5** | (gpu-native track) net buffers | GPU parity — **not this branch** | --- ## 6. Thread slicing -Mirror node model: +### 6.1 Layout + +Mirror nodes: ```text -Global SoA: [ row 0 | row 1 | ... | row N-1 ] - |← thread 0 →|← thread 1 →| +Global PointProcess SoA: [ thread0 rows | thread1 rows | ... ] +Global Weight SoA: [ thread0 block | thread1 block | ... ] +Global NetCon SoA: [ thread0 rows | thread1 rows | ... ] +Global PreSyn SoA: [ thread0 rows | thread1 rows | ... ] ``` -Per `NrnThread` (new fields, names TBD): +### 6.2 Fields + +**On `NrnThread` (or `neuron::cache::Thread` — prefer cache for offsets that only matter when sorted):** + +| Field | Meaning | +|-------|---------| +| `_pntproc_offset` / `pntproc_count` | Slice of PointProcess storage | +| `_weight_offset` / `weight_count` | Slice of Weight storage | +| `_netcon_offset` / `netcon_count` | Slice of NetCon storage | +| `_presyn_offset` / `presyn_count` | Slice of PreSyn storage | + +**Decision (open Q1 resolved):** Weights are **partitioned per thread** (CoreNEURON `nt.weights` / `nt.n_weight`). Cross-thread NetCon is already restricted; weight blocks live on the **target** thread of the Point_process. + +**Hot-path address:** + +```cpp +double* w = weight_storage.get_field_data() + nt.weight_offset; +// delivery uses absolute weight_index into global storage, or local index +// local = absolute - nt.weight_offset +``` + +Absolute global indices simplify dual-write; local indices match CoreNEURON. Prefer **absolute** during dual-write, document conversion at nrncore export. + +### 6.3 When slices are computed + +During `nrn_ensure_model_data_are_sorted()` / network sort pass: + +1. Assign each PointProcess to a thread (from cell partition / existing `_vnt`). +2. Permute PointProcess so thread regions are contiguous; set offsets. +3. Order NetCons by `(src_presyn, …)`; place on target thread of `Target`. +4. Pack weight blocks per NetCon into the target thread’s weight region; rewrite `WeightIndex`. +5. Order PreSyn by thread; build `NcIndex`/`NcCount` into the NetCon order. + +--- + +## 7. HOC sidecar policy + +### 7.1 Principle + +Integration columns are the **only** data the hot path and (future) GPU need. +Interpreter / recording / identity live in **sidecars** keyed by stable id, **not** a second pointer graph that owns network topology. + +### 7.2 Keying -- `_pntproc_offset`, `_pntproc_count` -- `_netcon_offset`, `_netcon_count` -- `_presyn_offset`, `_presyn_count` -- `_weight_offset`, `_weight_count` (if weights partitioned per thread) +| Key | Use | +|-----|-----| +| `owning_handle` / `non_owning_identifier_without_container` | Primary: survives permute; dies with row | +| HOC `Object*` | External identity; points at wrapper that holds `owning_handle` | -Permutation chosen so each thread region stays contiguous after `nrn_threads_create` / cell partitioning. +Do **not** key sidecars by current SoA row integer alone without a stable id. + +### 7.3 Sidecar contents by entity + +**Point_process** + +| Field | Notes | +|-------|-------| +| `Object* ob` | HOC object | +| `Section* sec`, `Node* node` | Location; invalidate on topology change | +| `Prop* prop` | Mechanism instance bridge during dual-write | +| `void* presyn_`, `nvi_`, `_vnt` | Until indices replace them | + +During Phase 1 dual-write, the existing `struct Point_process` **is** the sidecar + legacy shell; SoA holds Instance/MechType/ThreadId. Migration moves fields out of the struct into maps/`owning_handle` wrappers over time. + +**NetCon** + +| Field | Notes | +|-------|-------| +| `Object* obj_` | HOC | +| `PreSyn* src_` | Dual-write until `SrcPreSyn` index | +| `Point_process* target_` | Dual-write until `Target` index | +| `double* weight_` | Dual-write alias into Weight SoA (then removed) | + +**PreSyn** + +| Field | Notes | +|-------|-------| +| `data_handle thvar_` | **Keep** as canonical threshold source | +| `Object* osrc_`, `Section* ssrc_` | Source identity | +| `IvocVect* tvec_`, `idvec_`, `HocCommand* stmt_` | Recording | +| `NrnThread* nt_`, `hoc_Item* hi_th_` | Thread / threshold list | +| `NetConPList dil_` | Dual-write until `NcIndex`/`NcCount` | +| MPI / MUSIC / multisend unions | Sidecar or later phase | + +### 7.4 What must never be duplicated as “second graph” + +- Do not keep a parallel `std::vector` fanout **and** `NcIndex`/`NcCount` as equally authoritative after Phase 3. One source of truth: SoA ranges. +- Do not allocate per-NetCon `new double[cnt]` after weight migration completes. --- -## 7. Handle API (wrappers) +## 8. Invalidation and sorting + +### 8.1 Unsorted triggers + +| Event | Action | +|-------|--------| +| Create/destroy PointProcess, NetCon, PreSyn | `mark_as_unsorted()` on affected network container(s) | +| NetCon retarget / weight count change | weights + netcons unsorted | +| Topology / `define_shape` / `tree_changed` | Invalidate sidecar caches (`Section*`, node location); mark network unsorted if thread membership may change | +| `nrn_threads_create` / repartition | Full network reorder | +| Mechanism permute that changes Instance rows | Update PointProcess `Instance` column (or mark unsorted and fix at sort) | + +Each network `storage` registers `set_unsorted_callback` → `neuron::cache::model.reset()` (same as nodes). + +### 8.2 Sort / freeze policy -| HOC-facing type | Wrapper holds | Example accessor | -|-----------------|---------------|------------------| -| `Point_process` | `PointProcess::owning_handle` | `prop()` → resolve via `_type` + `_i_instance` | -| `NetCon` | `NetCon::owning_handle` | `weight(i)` → `weights[weight_index+i]` | -| `PreSyn` | `PreSyn::owning_handle` | `threshold` → SoA column | +Extend `nrn_ensure_model_data_are_sorted()`: + +1. Issue frozen tokens for node, mechanisms, **and** network containers. +2. If any network container is unsorted, run network permute/repack (§6.3). +3. Hold frozen tokens through integration like mechanisms. + +**Freeze:** same `frozen_token_type` pattern — no row insert/delete while sorted token is held for that container. + +### 8.3 Handle death + +| Situation | Behavior | +|-----------|----------| +| `owning_handle` destroyed | Row freed; non-owning handles / `data_handle`s to that row become invalid (`operator bool` false) | +| PointProcess destroyed while NetCons target it | Existing NetCon disconnect rules; `Target = -1` | +| Weight block freed | NetCon weight fields cleared; HOC weight access errors | +| Node voltage dies under `thvar_` | Existing `notify_when_handle_dies` path | + +### 8.4 Permutation vs mechanism sort (open Q2 resolved) + +**Order inside global sort:** + +1. Nodes (existing). +2. Mechanisms (existing) — Instance rows stable for this step’s PointProcess fixup. +3. **Network:** PointProcess (by thread), Weights (repack by NetCon), NetCon (by src PreSyn / thread), PreSyn (by thread) + fanout ranges. + +PointProcess `Instance` is rewritten if mechanism permute moved rows (stable mech handles preferred when available). + +--- + +## 9. `pnt_receive` signature migration + +### 9.1 Today + +```cpp +typedef void (*pnt_receive_t)(Point_process*, double* weight, double flag); +// POINT_RECEIVE(type, tar, w, f) (*pnt_receive[type])(tar, w, f) +``` -**CPU delivery signature (target):** +### 9.2 Target (align CoreNEURON) ```cpp -pnt_receive[type](pnt_handle, weight_index, flag); +// Conceptual end state +void pnt_receive(int type, int pnt_instance /* or pnt row */, int weight_index, double flag); +// body loads weights via global/thread weight base + weight_index ``` -Align with CoreNEURON; migrate from `(Point_process*, double* _args, double flag)`. +### 9.3 Migration steps + +| Step | Action | +|------|--------| +| M1 | Dual-write weights into SoA; `NetCon::weight_` points into SoA (`data()` + index) so existing `pnt_receive` still gets `double*` | +| M2 | Delivery uses weight_index internally; still materializes `double*` for generated code | +| M3 | Codegen / MOD translation accepts weight_index (Phase 4); legacy pointer form kept until mods regenerated | +| M4 | Drop per-NetCon heap weights | + +Phase 1 only needs M1 scaffolding (container + optional pointer into SoA). **No MOD codegen changes in Phase 1.** --- -## 8. Invalidation and sorting +## 10. SaveState / BBSaveState (note for Phase 2+) + +| Concern | Impact | +|---------|--------| +| `NetConSave` weight pointer tables | Today maps `double* → NetCon*`. After SoA, key by `weight_index` or stable NetCon id. Call `NetConSave::invalid()` on weight repack. | +| `PreSynSave` / `hi_index_` | Keep index tables; rebuild on unsorted. | +| BBSaveState fanout | Uses `PreSyn::fanout`; switch to `NcIndex`/`NcCount` in Phase 3. | +| Binary layout | Do not freeze on-disk format on intermediate dual-write; document stable index fields when dual-write ends. | + +Minimal Phase 1 impact: none if PointProcess/Weight SoA is not yet referenced by SaveState. -- Topology / `define_shape` → `mark_as_unsorted()` on network containers (like `node_data()`). -- `tree_changed` → refresh sidecar caches; optional full network reorder pass. -- `psolve` / integration entry: `nrn_ensure_model_data_are_sorted()` includes network containers when fanout order matters. -- **Freeze policy:** same frozen-token pattern as mechanisms during sorted integration phases (TBD). +--- + +## 11. Handle API and HOC wrappers + +| HOC-facing type | Wrapper holds | Example | +|-----------------|---------------|---------| +| Point process object | `PointProcess::owning_handle` (+ legacy `Point_process*` during dual-write) | `instance()` → mech row | +| `NetCon` | `NetCon::owning_handle` | `weight(i)` → `weights[WeightIndex+i]` | +| `PreSyn` | `PreSyn::owning_handle` | `threshold` ↔ SoA column; `thvar_` stays `data_handle` | + +**CPU delivery target:** + +```cpp +// After Phase 4 +pnt_receive[type](/* target identity */, weight_index, flag); +``` --- -## 9. Migration strategy +## 12. Migration strategy (execution order) | Step | Action | |------|--------| -| M1 | Dual-write: create SoA row when creating legacy object | -| M2 | Dual-read: hot path reads SoA; HOC still works via wrapper | -| M3 | Remove hot-path pointer fields (`weight_`, `dil_` iteration) | -| M4 | Sidecars only for interpreter | +| M1 dual-write | Allocate SoA row when creating legacy object | +| M2 dual-read | Hot path prefers SoA; HOC via wrapper/legacy | +| M3 remove hot-path pointers | Drop `weight_` heap, `dil_` iteration | +| M4 sidecars only | Interpreter extras only in sidecars | -Order: **weights + Point_process → NetCon → PreSyn → SelfEvent**. +**Order:** weights + Point_process → NetCon → PreSyn → SelfEvent. --- -## 10. Phase 0 deliverables (checklist) +## 13. Test plan -- [ ] Final field tag list per container (section 5.2) -- [ ] `Model` ownership and file layout (`src/neuron/container/network/`) -- [ ] Handle typedef names (`PointProcess::handle`, …) -- [ ] Thread offset fields on `NrnThread` / multicore -- [ ] Sidecar map design + invalidation rules -- [ ] `pnt_receive` signature migration plan -- [ ] SaveState / BBSaveState impact (minimal note for Phase 2+) -- [ ] Test plan: unit (permute), integration (ringtest CPU spikes) +### 13.1 Unit (Phase 1) + +| Test | Expectation | +|------|-------------| +| Create N PointProcess rows; set Instance/MechType/ThreadId | Round-trip via handles | +| `apply_reverse_permutation` on PointProcess storage | Handle logical values unchanged; storage order may change | +| Create weight block of size K; permute | Values via handles stable | +| Destroy `owning_handle` | Non-owning handle / data_handle invalid | +| `find_container_info` / `data_handle` promote | Weight `Value` and PointProcess fields discoverable | + +### 13.2 Integration (Phase 2+) + +| Gate | Command / check | +|------|-----------------| +| Phase 2 | CPU ringtest delivery (NetCon → ExpSyn) | +| Phase 3 | CPU spike parity @ 100 ms vs baseline | +| Phase 4 | ExpSyn receive @ 1.025 ms CPU | +| Regression | `ctest` network-related; SaveState smoke if touched | + +### 13.3 Non-tests on this branch + +- GPU `net_buf_receive`, ringtest GPU network buffers. + +--- + +## 14. Phase 0 deliverables checklist + +- [x] Final field tag list per container (§5.2–5.6) +- [x] `Model` ownership and file layout (`src/neuron/container/network/`) +- [x] Handle typedef names (`PointProcess::handle`, `Weight::owning_handle`, …) +- [x] Thread offset fields on `NrnThread` / cache (§6) +- [x] Sidecar map design + invalidation rules (§7–§8) +- [x] `pnt_receive` signature migration plan (§9) +- [x] SaveState / BBSaveState impact note (§10) +- [x] Test plan (§13) --- -## 11. Phase 1 first code (after spec sign-off) +## 15. Phase 1 code scaffold ``` src/neuron/container/network/ - point_process.hpp # field tags, storage typedef, handle API - point_process_data.cpp # storage impl hooks - weights.hpp + point_process.hpp # field tags, storage, handle API + weights.hpp # field tags, storage, handle API ``` -Hook point: point process allocation in existing `nrn_point_process` / mechanism insert path — allocate SoA row alongside today's `Prop`. +Hook points (implementation after scaffold): + +1. `Model` members + accessors + `find_container_info` + unsorted callbacks. +2. Unit test: permute survival. +3. Later: `nrn_point_process` / NetCon ctor dual-write (not required for empty scaffold compile). + +--- + +## 16. Resolved design questions + +| # | Question | Decision | +|---|----------|----------| +| 1 | Global vs per-thread weight pool | **Per-thread slices** of one global SoA (CoreNEURON-compatible); absolute indices during dual-write OK | +| 2 | Network permute vs mechanism permute | **After** nodes + mechanisms inside `nrn_ensure_model_data_are_sorted` | +| 3 | NetCon as `DiscreteEvent` | **Keep subclass** through Phase 2–3 for queue compatibility | +| 4 | `InputPreSyn` | **Defer** to post–Phase 3 MPI milestone | +| 5 | Weight contiguity under permute | **Repack at sort** into contiguous per-NetCon blocks; rewrite `WeightIndex` | +| 6 | PreSyn threshold | **Keep `data_handle`**; optional `ThVarRow` cache for scans | --- -## 12. Open questions +## 17. Out-of-scope reminder -1. Global vs per-thread weight pool — CoreNEURON uses per-thread `weights`; confirm for NEURON MPI. -2. When to permute network SoA relative to mechanism permute (`nrn_sort_mech_data`). -3. `NetCon` as `DiscreteEvent` subclass — keep for queue compatibility or separate event index? -4. `InputPreSyn` split — Phase 3 or later MPI milestone? +Do **not** implement GPU `net_buf_receive` on `local/cpu-network-soa`. Stage 1 `NetReceiveBuffer` on the gpu-native branch remains a valid pattern; indexing will align after SoA merges. --- -*Expand this document into the authoritative Phase 0 spec in the first `cpu_net_soa` session.* \ No newline at end of file +*This document is the Phase 0 gate. Phase 1 implements PointProcess + Weight containers and proves handles survive permutation.* diff --git a/src/neuron/container/memory_usage.hpp b/src/neuron/container/memory_usage.hpp index 33c3cfa5db..a152ded36b 100644 --- a/src/neuron/container/memory_usage.hpp +++ b/src/neuron/container/memory_usage.hpp @@ -76,9 +76,17 @@ struct ModelMemoryUsage { /// @brief The memory usage of all mechanisms. StorageMemoryUsage mechanisms{}; + /// @brief Network SoA: Point_process integration rows. + StorageMemoryUsage point_processes{}; + + /// @brief Network SoA: flat weight pool. + StorageMemoryUsage weights{}; + const ModelMemoryUsage& operator+=(const ModelMemoryUsage& other) { nodes += other.nodes; mechanisms += other.mechanisms; + point_processes += other.point_processes; + weights += other.weights; return *this; } @@ -86,6 +94,8 @@ struct ModelMemoryUsage { VectorMemoryUsage compute_total() const { auto total = nodes.compute_total(); total += mechanisms.compute_total(); + total += point_processes.compute_total(); + total += weights.compute_total(); return total; } @@ -187,6 +197,8 @@ struct MemoryUsageSummary { void add(const ModelMemoryUsage& model) { add(model.nodes); add(model.mechanisms); + add(model.point_processes); + add(model.weights); } void add(const cache::ModelMemoryUsage& model) { diff --git a/src/neuron/container/network/point_process.hpp b/src/neuron/container/network/point_process.hpp new file mode 100644 index 0000000000..805b85132c --- /dev/null +++ b/src/neuron/container/network/point_process.hpp @@ -0,0 +1,116 @@ +#pragma once +/** + * @file network/point_process.hpp + * @brief SoA storage for integration-hot Point_process fields. + * + * CoreNEURON reference: coreneuron::Point_process {_i_instance, _type, _tid}. + * Design: doc/network-soa-phase0.md §5.2. + * + * HOC location / Object* / Prop* remain on the legacy Point_process shell or + * sidecars during dual-write (Phase 1 does not replace interpreter objects). + */ +#include "neuron/container/data_handle.hpp" +#include "neuron/container/soa_container.hpp" +#include "neuron/container/view_utils.hpp" + +#include +#include + +namespace neuron::container::network::PointProcess { +namespace field { + +/** @brief Mechanism SoA instance row (_i_instance). */ +struct Instance { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Mechanism type (_type). */ +struct MechType { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Owning NrnThread id (_tid). */ +struct ThreadId { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +} // namespace field + +/** + * @brief Public API for PointProcess handles (owning and non-owning). + * @tparam Identifier owning_identifier or non_owning_identifier for storage. + */ +template +struct handle_interface: handle_base { + using base_type = handle_base; + using base_type::base_type; + + [[nodiscard]] field::Instance::type& instance() { + return this->template get(); + } + [[nodiscard]] field::Instance::type const& instance() const { + return this->template get(); + } + [[nodiscard]] data_handle instance_handle() { + return this->template get_handle(); + } + + [[nodiscard]] field::MechType::type& mech_type() { + return this->template get(); + } + [[nodiscard]] field::MechType::type const& mech_type() const { + return this->template get(); + } + [[nodiscard]] data_handle mech_type_handle() { + return this->template get_handle(); + } + + [[nodiscard]] field::ThreadId::type& thread_id() { + return this->template get(); + } + [[nodiscard]] field::ThreadId::type const& thread_id() const { + return this->template get(); + } + [[nodiscard]] data_handle thread_id_handle() { + return this->template get_handle(); + } + + friend std::ostream& operator<<(std::ostream& os, handle_interface const& handle) { + if (handle.id()) { + return os << "PointProcess{" << handle.id() << '/' << handle.underlying_storage().size() + << " instance=" << handle.instance() << " mech_type=" << handle.mech_type() + << " thread_id=" << handle.thread_id() << '}'; + } + return os << "PointProcess{null}"; + } +}; + +/** @brief Underlying storage for all PointProcess integration rows. */ +struct storage: soa { + [[nodiscard]] std::string_view name() const { + return "network::PointProcess"; + } +}; + +/** @brief Non-owning handle; stable across permute; invalid after owner dies. */ +using handle = handle_interface>; + +/** @brief Owning handle; destroys the storage row on destruction. */ +struct owning_handle: handle_interface> { + using base_type = handle_interface>; + using base_type::base_type; + + [[nodiscard]] handle non_owning_handle() { + return non_owning_identifier{&underlying_storage(), id()}; + } +}; +} // namespace neuron::container::network::PointProcess diff --git a/src/neuron/container/network/weights.hpp b/src/neuron/container/network/weights.hpp new file mode 100644 index 0000000000..eaf0433d3c --- /dev/null +++ b/src/neuron/container/network/weights.hpp @@ -0,0 +1,79 @@ +#pragma once +/** + * @file network/weights.hpp + * @brief SoA storage for the flat NetCon weight pool. + * + * CoreNEURON reference: NrnThread::weights[n_weight]; NetCon.u.weight_index_ base. + * Design: doc/network-soa-phase0.md §5.3. + * + * Contiguity: a NetCon with weight_count = K owns K consecutive rows. + * Pack/repack at sort time; WeightIndex on NetCon is rewritten after permute. + */ +#include "neuron/container/data_handle.hpp" +#include "neuron/container/soa_container.hpp" +#include "neuron/container/view_utils.hpp" + +#include +#include + +namespace neuron::container::network::Weight { +namespace field { + +/** @brief One scalar weight entry in the flat pool. */ +struct Value { + using type = double; + constexpr type default_value() const { + return 0.; + } +}; + +} // namespace field + +/** + * @brief Public API for Weight handles (owning and non-owning). + * @tparam Identifier owning_identifier or non_owning_identifier for storage. + */ +template +struct handle_interface: handle_base { + using base_type = handle_base; + using base_type::base_type; + + [[nodiscard]] field::Value::type& value() { + return this->template get(); + } + [[nodiscard]] field::Value::type const& value() const { + return this->template get(); + } + [[nodiscard]] data_handle value_handle() { + return this->template get_handle(); + } + + friend std::ostream& operator<<(std::ostream& os, handle_interface const& handle) { + if (handle.id()) { + return os << "Weight{" << handle.id() << '/' << handle.underlying_storage().size() + << " value=" << handle.value() << '}'; + } + return os << "Weight{null}"; + } +}; + +/** @brief Underlying storage for all weight scalars. */ +struct storage: soa { + [[nodiscard]] std::string_view name() const { + return "network::Weight"; + } +}; + +/** @brief Non-owning handle; stable across permute; invalid after owner dies. */ +using handle = handle_interface>; + +/** @brief Owning handle; destroys the storage row on destruction. */ +struct owning_handle: handle_interface> { + using base_type = handle_interface>; + using base_type::base_type; + + [[nodiscard]] handle non_owning_handle() { + return non_owning_identifier{&underlying_storage(), id()}; + } +}; +} // namespace neuron::container::network::Weight diff --git a/src/neuron/model_data.hpp b/src/neuron/model_data.hpp index 4f35a64d5f..6284287d33 100644 --- a/src/neuron/model_data.hpp +++ b/src/neuron/model_data.hpp @@ -2,6 +2,8 @@ #include "neuron/cache/model_data.hpp" #include "neuron/container/mechanism_data.hpp" #include "neuron/container/memory_usage.hpp" +#include "neuron/container/network/point_process.hpp" +#include "neuron/container/network/weights.hpp" #include "neuron/container/node_data.hpp" #include "neuron/model_data_fwd.hpp" @@ -31,6 +33,26 @@ struct Model { return m_node_data; } + /** @brief Access SoA storage for Point_process integration fields. + * @see doc/network-soa-phase0.md + */ + container::network::PointProcess::storage& point_processes() { + return m_point_processes; + } + container::network::PointProcess::storage const& point_processes() const { + return m_point_processes; + } + + /** @brief Access SoA storage for the flat NetCon weight pool. + * @see doc/network-soa-phase0.md + */ + container::network::Weight::storage& weights() { + return m_weights; + } + container::network::Weight::storage const& weights() const { + return m_weights; + } + /** @brief Apply a function to each non-null Mechanism. */ template @@ -120,6 +142,8 @@ struct Model { void shrink_to_fit() { m_node_data.shrink_to_fit(); apply_to_mechanisms([](auto& mech_data) { mech_data.shrink_to_fit(); }); + m_point_processes.shrink_to_fit(); + m_weights.shrink_to_fit(); } private: @@ -151,6 +175,12 @@ struct Model { */ std::vector> m_mech_data{}; + /** @brief Network SoA: Point_process integration rows (Phase 1). */ + container::network::PointProcess::storage m_point_processes{}; + + /** @brief Network SoA: flat weight pool (Phase 1). */ + container::network::Weight::storage m_weights{}; + /** * @brief Backing storage for defer_delete helper. */ diff --git a/src/nrniv/memory_usage.cpp b/src/nrniv/memory_usage.cpp index 49361bbd67..17439d6521 100644 --- a/src/nrniv/memory_usage.cpp +++ b/src/nrniv/memory_usage.cpp @@ -13,7 +13,10 @@ ModelMemoryUsage memory_usage(const Model& model) { auto mechanisms = StorageMemoryUsage(); model.apply_to_mechanisms([&mechanisms](const auto& md) { mechanisms += memory_usage(md); }); - return {nodes, mechanisms}; + auto point_processes = memory_usage(model.point_processes()); + auto weights = memory_usage(model.weights()); + + return {nodes, mechanisms, point_processes, weights}; } cache::ModelMemoryUsage memory_usage(const std::optional& model) { @@ -117,6 +120,15 @@ std::string format_memory_usage(const MemoryUsage& usage) { os << " data " << format_memory_usage(model.mechanisms.heavy_data) << "\n"; os << " stable_identifiers " << format_memory_usage(model.mechanisms.stable_identifiers) << "\n"; + os << " network::PointProcess \n"; + os << " data " << format_memory_usage(model.point_processes.heavy_data) + << "\n"; + os << " stable_identifiers " + << format_memory_usage(model.point_processes.stable_identifiers) << "\n"; + os << " network::Weight \n"; + os << " data " << format_memory_usage(model.weights.heavy_data) << "\n"; + os << " stable_identifiers " << format_memory_usage(model.weights.stable_identifiers) + << "\n"; os << "cache::Model \n"; os << " threads " << format_memory_usage(cache_model.threads) << "\n"; os << " mechanisms " << format_memory_usage(cache_model.mechanisms) << "\n"; diff --git a/src/nrnoc/container.cpp b/src/nrnoc/container.cpp index a493a2f5b5..84cbc92bb2 100644 --- a/src/nrnoc/container.cpp +++ b/src/nrnoc/container.cpp @@ -15,6 +15,8 @@ void invalidate_cache() { namespace neuron { Model::Model() { m_node_data.set_unsorted_callback(invalidate_cache); + m_point_processes.set_unsorted_callback(invalidate_cache); + m_weights.set_unsorted_callback(invalidate_cache); // needs some re-organisation if we ever want to support multiple Model instances assert(!container::detail::defer_delete_storage); container::detail::defer_delete_storage = &m_ptrs_for_deferred_deletion; @@ -38,6 +40,12 @@ std::unique_ptr Model::find_container_info(void return maybe_info; } } + if (auto maybe_info = m_point_processes.find_container_info(cont); maybe_info) { + return maybe_info; + } + if (auto maybe_info = m_weights.find_container_info(cont); maybe_info) { + return maybe_info; + } return {}; } @@ -143,6 +151,13 @@ generic_data_handle promote_or_clear(generic_data_handle gdh) { if (done) { return gdh; } + if (auto h = model.point_processes().find_data_handle(gdh); + h.refers_to_a_modern_data_structure()) { + return h; + } + if (auto h = model.weights().find_data_handle(gdh); h.refers_to_a_modern_data_structure()) { + return h; + } return {}; } } // namespace detail diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index ffa28da9b2..b161e9c925 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -21,6 +21,7 @@ add_executable( unit_tests/container/container.cpp unit_tests/container/generic_data_handle.cpp unit_tests/container/mechanism.cpp + unit_tests/container/network.cpp unit_tests/container/node.cpp unit_tests/node_order_optim/permutations.cpp unit_tests/utils/enumerate.cpp diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp new file mode 100644 index 0000000000..bd8eff8a27 --- /dev/null +++ b/test/unit_tests/container/network.cpp @@ -0,0 +1,157 @@ +#include "neuron/container/network/point_process.hpp" +#include "neuron/container/network/weights.hpp" +#include "neuron/model_data.hpp" +#include "section.h" + +#include + +#include +#include +#include +#include + +using namespace neuron::container::network; + +TEST_CASE("SOA-backed PointProcess structure", "[Neuron][data_structures][network][point_process]") { + auto& storage = neuron::model().point_processes(); + REQUIRE(storage.size() == 0); + + GIVEN("Default-constructed owning handles") { + PointProcess::owning_handle a{storage}; + PointProcess::owning_handle b{storage}; + THEN("Defaults match field tags") { + REQUIRE(a.instance() == -1); + REQUIRE(a.mech_type() == -1); + REQUIRE(a.thread_id() == -1); + REQUIRE(storage.size() == 2); + } + THEN("Fields can be set and read back via non-owning handles") { + a.instance() = 7; + a.mech_type() = 42; + a.thread_id() = 1; + auto ha = a.non_owning_handle(); + REQUIRE(ha.instance() == 7); + REQUIRE(ha.mech_type() == 42); + REQUIRE(ha.thread_id() == 1); + } + THEN("Destroying an owner invalidates non-owning handles to that row") { + auto ha = a.non_owning_handle(); + REQUIRE(ha.id()); + { + PointProcess::owning_handle tmp{std::move(a)}; + REQUIRE(ha.id()); + // tmp destroyed here + } + REQUIRE_FALSE(ha.id()); + REQUIRE(storage.size() == 1); // only b remains + } + } + + GIVEN("Several rows and a reverse permutation") { + constexpr int n = 8; + std::vector rows; + rows.reserve(n); + std::vector ref_instance(n); + for (int i = 0; i < n; ++i) { + rows.emplace_back(storage); + rows.back().instance() = 100 + i; + rows.back().mech_type() = i; + rows.back().thread_id() = i % 3; + ref_instance[i] = 100 + i; + } + { + auto token = storage.issue_frozen_token(); + storage.mark_as_sorted(token); + } + REQUIRE(storage.is_sorted()); + + WHEN("A rotate reverse-permutation is applied") { + std::vector perm(n); + std::iota(perm.begin(), perm.end(), 0); + std::rotate(perm.begin(), std::next(perm.begin()), perm.end()); + storage.apply_reverse_permutation(std::move(perm)); + + THEN("Handles still report the logical field values") { + for (int i = 0; i < n; ++i) { + REQUIRE(rows[i].instance() == ref_instance[i]); + REQUIRE(rows[i].mech_type() == i); + REQUIRE(rows[i].thread_id() == i % 3); + } + } + THEN("Underlying storage order differs from original creation order") { + bool same = true; + for (int i = 0; i < n; ++i) { + if (storage.get(i) != ref_instance[i]) { + same = false; + break; + } + } + REQUIRE_FALSE(same); + } + } + + WHEN("A random reverse-permutation is applied") { + std::vector perm(n); + std::iota(perm.begin(), perm.end(), 0); + std::mt19937 g{42}; + std::shuffle(perm.begin(), perm.end(), g); + storage.apply_reverse_permutation(std::move(perm)); + THEN("Handles survive and keep logical values") { + for (int i = 0; i < n; ++i) { + REQUIRE(rows[i].instance() == ref_instance[i]); + } + } + } + } +} + +TEST_CASE("SOA-backed Weight structure", "[Neuron][data_structures][network][weights]") { + auto& storage = neuron::model().weights(); + REQUIRE(storage.size() == 0); + + GIVEN("A contiguous weight block (NetCon-style)") { + constexpr int weight_count = 5; + std::vector block; + block.reserve(weight_count); + for (int k = 0; k < weight_count; ++k) { + block.emplace_back(storage); + block.back().value() = static_cast(k + 1); // exact binary values + } + REQUIRE(storage.size() == static_cast(weight_count)); + + THEN("data_handle to Value is modern and yields correct values") { + auto dh = block[2].value_handle(); + REQUIRE(dh); + REQUIRE(dh.refers_to_a_modern_data_structure()); + REQUIRE(dh.refers_to(storage)); + REQUIRE(*dh == 3.0); + } + + WHEN("The weight storage is reverse-permuted") { + std::vector perm(weight_count); + std::iota(perm.begin(), perm.end(), 0); + std::rotate(perm.begin(), std::next(perm.begin()), perm.end()); + storage.apply_reverse_permutation(std::move(perm)); + + THEN("Handles still report the logical values") { + for (int k = 0; k < weight_count; ++k) { + REQUIRE(block[k].value() == static_cast(k + 1)); + } + } + THEN("data_handle still works after permute") { + auto dh = block[0].value_handle(); + REQUIRE(*dh == 1.0); + } + } + + WHEN("The owning block is destroyed") { + auto dh = block[0].value_handle(); + REQUIRE(dh); + block.clear(); + THEN("data_handles become invalid and storage is empty") { + REQUIRE_FALSE(dh); + REQUIRE(storage.size() == 0); + } + } + } +} From 47480bb14af79e8c86af1b786e1f9ecda350394c Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 08:26:18 -0400 Subject: [PATCH 03/33] Phase 0: document NetCon linkage and remote PreSyn roles MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add §5.4.1 (target_/src_ as indices/handles, not data_handle) and §5.5.1 (InputPreSyn deferred; thin gid→fanout after cpu-net-soa). --- doc/network-soa-phase0.md | 96 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 91 insertions(+), 5 deletions(-) diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index 9f22da3f63..70fe08c0b8 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -40,7 +40,7 @@ Bridging with per-event shims on the legacy layout does not scale. **One SoA bac | Out of scope | Resume when | |--------------|-------------| | GPU `net_buf_receive` codegen / Stage 3 wire-up | SoA on master; rebase `gpu-native-qualification` | -| Full `InputPreSyn` / MPI multisend redesign | Post–Phase 3 MPI milestone | +| Remote input / `InputPreSyn` split; MPI multisend redesign | After cpu-net-soa (see §5.5.1) | | Removing HOC `Object*` identity | Never — sidecars remain | | Changing public HOC/Python method signatures | Not required | @@ -168,12 +168,61 @@ weights[weight_index + k] for k in [0, weight_count) | `field::WeightCount` | `int` | `0` | `pnt_receive_size` for target type | | `field::Delay` | `double` | `1.0` | Delivery delay (ms) | | `field::Active` | `int` | `1` | `bool` as `int` (GPU-friendly, CoreNEURON style) | -| `field::SrcPreSyn` | `int` | `-1` | Source PreSyn row, or `-1` if none | +| `field::SrcPreSyn` | `int` | `-1` | Source PreSyn row, or `-1` if none (management / reverse edge; see §5.4.1) | **Optional later:** `field::ObjectId` only if HOC identity must be recovered from a row without a sidecar map. **Event-queue identity:** `NetCon` remains a `DiscreteEvent` subclass for queue compatibility through Phase 2–3. The SoA row is the data plane; the C++ object is the control/HOC plane during dual-write. Long term the queued payload may shrink to `(type, row)` — not required for Phase 2 gate. +#### 5.4.1 NetCon linkage: `target_` and `src_` (pointers vs handles vs indices) + +Today NEURON keeps a **mutual pointer graph**: + +```text +PreSyn ──dil_──► NetCon* ──src_──► PreSyn + │ + └──target_──► Point_process* +``` + +CoreNEURON’s integration layout does **not** retain that graph: `NetCon` has `target_` (struct pointer into `pntprocs`) and `weight_index_`; source is implicit because the NetCon sits in a PreSyn’s fanout range (`nc_index_` / `nc_cnt_`). There is no `src_` on the hot path. + +**Decision for NEURON SoA — three layers, not one type for all uses:** + +| Concern | Representation | Why | +|---------|----------------|-----| +| **Spike / delivery hot path** | Integer **SoA row indices** (`Target`, `WeightIndex`; fanout via PreSyn `NcIndex`/`NcCount`) | CoreNEURON-shaped, GPU-uploadable, no pointer chasing, contiguous scans | +| **Stable identity across permute / dual-write / HOC** | `PointProcess::handle` / `PreSyn::handle` (or `non_owning_identifier`) on the wrapper object | Same model as `Node::handle` — tracks row through permute without remapping by hand on every HOC touch | +| **Legacy shell (transitional)** | Keep `Point_process* target_` and `PreSyn* src_` on the C++ `NetCon` during dual-write | Existing disconnect / HOC / queue code keeps working until dual-read is complete | + +**Do not use `data_handle` for `target_` / `src_`.** +`data_handle` is a stable reference to a **scalar field in a column** (e.g. `PreSyn::thvar_` → a `double` voltage). A NetCon’s target is a **row of another entity**, not a double. The correct SoA tools are: + +- **row index `int`** after sort (integration), +- **`handle` / `owning_handle` / `non_owning_identifier`** when a stable cross-reference is needed outside the frozen sorted window. + +**Directionality of the assembly (important):** + +```text +Hot path (spike → targets): + PreSyn ──NcIndex/NcCount──► NetCon rows ──Target index──► PointProcess row + │ + └── WeightIndex ──► weight block + +Reverse edge (management / HOC only): + NetCon ──SrcPreSyn index or PreSyn::handle──► PreSyn +``` + +- **Forward fanout is authoritative** for delivery (replaces walking `dil_` as a pointer list). +- **`src_` / `SrcPreSyn` is a reverse edge**: disconnect, `replace_src`, mindelay queries, interpreter inspection. It is **not** required for CoreNEURON-shaped deliver. It may remain on the HOC wrapper / dual-write shell longer than `target_`, or stay as a SoA `int` column for convenience when destroying a PreSyn and sweeping its NetCons (until ranges alone suffice). + +**Why not keep pointer mutuality as the long-term design?** + +- Pointer graphs fight permutation, thread packing, and GPU mirrors. +- Two-way raw pointers duplicate topology already expressed by fanout ranges + `Target`. +- Mutuality is valuable for **interactive** NEURON; express it with **handles + indices**, not with a second heap of cross-pointers that own the topology. + +**Phase timing:** Phase 2 dual-writes `Target` / weights while leaving `target_`/`src_` pointers valid; Phase 3 builds fanout ranges; hot path drops `dil_` walks and pointer fanout; pointer fields become optional wrappers over handles/indices and can shrink later without another layout break. + ### 5.5 PreSyn (Phase 3 — tags fixed now) | Tag | C++ type | Default | Meaning | @@ -199,6 +248,42 @@ Matches CoreNEURON `nc_index_` / `nc_cnt_` into `netcon_in_presyn_order_`. At so **Threshold source:** keep `data_handle` (today’s `thvar_`) in the dual-write / sidecar plane. Optionally cache `ThVarRow` when the handle refers to node voltage for vectorized threshold checks. +#### 5.5.1 PreSyn vs remote input (`InputPreSyn`) + +Two **roles** appear on a rank; CoreNEURON splits them into types, NEURON historically does not. + +| Role | Job on this rank | Threshold / `thvar` / HOC recording? | Fanout (`NcIndex`/`NcCount`)? | +|------|------------------|--------------------------------------|-------------------------------| +| **Local / output PreSyn** | Detect spike, optional gid send, fan out local NetCons | Yes | Yes | +| **Remote / input source** | Map received `(gid, t)` to local NetCons | No | Yes | + +CoreNEURON’s `InputPreSyn` is the second role reduced to `nc_index_` / `nc_cnt_` (plus optional multisend bookkeeping). NEURON today stores **full `PreSyn*`** in both `gid2out_` and `gid2in_` (`netpar.cpp`), so remote sources pay for `ConditionEvent`, threshold fields, and sidecars they never use. That cost matters when connectivity is dense and `gid2in` is large — the common case where **MPI_Allgather** (or compressed collectives) remains near-optimal because almost every rank needs almost every spike. Point-to-point / multisend sophistication (BlueGene-era) is orthogonal: it changes the **wire** path, not whether a remote source needs threshold state. + +**Is the role still useful?** Yes. Remote delivery only needs: + +```text +gid → (NcIndex, NcCount) → NetCon rows → targets / weights +``` + +**Must NEURON clone CoreNEURON’s dual class tree?** No. Prefer a **data-plane** split when the time comes: + +1. **Local PreSyn SoA** — threshold columns + gid/output + fanout range + HOC sidecars. +2. **Remote inputs** — thin **`gid → fanout range`** (dedicated small SoA or map), not a fat PreSyn and not a HOC-facing type. +3. Spike exchange resolves gid → range → enqueue NetCons (or one lightweight input event holding the range + time). + +That preserves CoreNEURON’s memory and export shape without a second interactive pointer graph. Optional fields on one “spike source” SoA are a weaker fit (mixed sort keys, threshold code special-cases). + +**Timing (explicitly out of scope for this branch’s Phase 0–4):** + +| When | What | +|------|------| +| Phases 1–2 | Ignore remote split | +| Phase 3 | Local fanout on existing PreSyn; `gid2in_` may stay fat PreSyn | +| **After** cpu-net-soa finishes (post–Phase 3/4 MPI milestone) | Thin remote representation; align `nrncore` / CoreNEURON `gid2in` | +| GPU net buffers | Upload the same fanout ranges; role split reduces payload | + +**Design constraint for Phase 3:** build fanout so a later remote-input table can own the same style of `(NcIndex, NcCount)` ranges **without** restating NetCon layout. Do not implement `InputPreSyn` work on `local/cpu-network-soa`. + ### 5.6 SelfEvent (Phase 4) SelfEvents are short-lived queue items. Prefer **not** a long-lived SoA of all SelfEvents; instead: @@ -306,8 +391,8 @@ During Phase 1 dual-write, the existing `struct Point_process` **is** the sideca | Field | Notes | |-------|-------| | `Object* obj_` | HOC | -| `PreSyn* src_` | Dual-write until `SrcPreSyn` index | -| `Point_process* target_` | Dual-write until `Target` index | +| `PreSyn* src_` | Dual-write reverse edge; long-term `SrcPreSyn` index and/or `PreSyn::handle` (§5.4.1) — not hot-path delivery | +| `Point_process* target_` | Dual-write; long-term `Target` index (+ optional `PointProcess::handle` on wrapper) | | `double* weight_` | Dual-write alias into Weight SoA (then removed) | **PreSyn** @@ -509,9 +594,10 @@ Hook points (implementation after scaffold): | 1 | Global vs per-thread weight pool | **Per-thread slices** of one global SoA (CoreNEURON-compatible); absolute indices during dual-write OK | | 2 | Network permute vs mechanism permute | **After** nodes + mechanisms inside `nrn_ensure_model_data_are_sorted` | | 3 | NetCon as `DiscreteEvent` | **Keep subclass** through Phase 2–3 for queue compatibility | -| 4 | `InputPreSyn` | **Defer** to post–Phase 3 MPI milestone | +| 4 | `InputPreSyn` | **Role yes, dual class optional**; thin gid→fanout after cpu-net-soa — §5.5.1 | | 5 | Weight contiguity under permute | **Repack at sort** into contiguous per-NetCon blocks; rewrite `WeightIndex` | | 6 | PreSyn threshold | **Keep `data_handle`**; optional `ThVarRow` cache for scans | +| 7 | NetCon `target_` / `src_` | **Indices on hot path**; **handles** for stable/HOC refs; **not** `data_handle`; pointers dual-write only — §5.4.1 | --- From 8e32e0a3a9e8b51767e5f52b1d596f305ca8d591 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 08:32:30 -0400 Subject: [PATCH 04/33] Phase 1 dual-write: PointProcess and Weight SoA on create/destroy Embed PointProcess::owning_handle on Point_process and sync Instance/ MechType/ThreadId when prop or _vnt changes. NetCon allocates parallel Weight SoA rows (heap weight_ stays delivery primary). Unit tests cover dual-write helpers and PP shell lifetime. --- GROK-NETWORK-SOA.md | 2 +- doc/network-soa-phase0.md | 2 +- src/neuron/container/network/weight_block.hpp | 48 +++++++++++++++++++ src/nrncvode/netcon.h | 3 ++ src/nrncvode/netcvode.cpp | 8 ++++ src/nrnoc/multicore.cpp | 2 + src/nrnoc/point.cpp | 25 ++++++++++ src/nrnoc/section_fwd.hpp | 10 ++++ src/nrnoc/treeset.cpp | 1 + test/unit_tests/container/network.cpp | 40 ++++++++++++++++ 10 files changed, 139 insertions(+), 2 deletions(-) create mode 100644 src/neuron/container/network/weight_block.hpp diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 40de2e3e10..0a2c725b10 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -69,7 +69,7 @@ psolve hot path (deliver, threshold, fanout) — index-based | Phase | Content | Gate | |-------|---------|------| | **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec done (authoritative) | -| **1** | `Point_process` + `weights` SoA scaffold under `src/neuron/container/network/` | handles survive permute (unit test) | +| **1** | `Point_process` + `weights` SoA + dual-write create/destroy | handles survive permute; PP/NetCon dual-write | | **2** | `NetCon` SoA, HOC `weight()` → flat weights | CPU ringtest delivery | | **3** | `PreSyn` `nc_index`/`nc_cnt` fanout | CPU spike parity @ 100 ms | | **4** | `SelfEvent` indices, `pnt_receive(weight_index)` | ExpSyn @ 1.025 ms CPU | diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index 70fe08c0b8..18eebc0788 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -301,7 +301,7 @@ If a structural pool is useful later, tags match the above. Gate is `pnt_receive | Phase | Container | Gate | |-------|-----------|------| | **0** | Spec (this doc) | Spec reviewed | -| **1** | `PointProcess` + `Weight` | Handles survive permute; unit test | +| **1** | `PointProcess` + `Weight` | Handles survive permute; dual-write on create/destroy | | **2** | `NetCon` SoA; HOC `weight()` → flat weights | CPU ringtest delivery | | **3** | `PreSyn` fanout `NcIndex`/`NcCount` | CPU spike parity @ 100 ms | | **4** | SelfEvent indices; `pnt_receive` by weight index | ExpSyn @ 1.025 ms CPU | diff --git a/src/neuron/container/network/weight_block.hpp b/src/neuron/container/network/weight_block.hpp new file mode 100644 index 0000000000..15c70a9315 --- /dev/null +++ b/src/neuron/container/network/weight_block.hpp @@ -0,0 +1,48 @@ +#pragma once +/** + * @file network/weight_block.hpp + * @brief Helpers to dual-write NetCon weight blocks into Weight SoA. + * + * Kept separate from weights.hpp so model_data.hpp can include Weight storage + * without a circular include. + * + * Phase 1: heap `double* weight_` remains the delivery primary; SoA rows are + * owned in parallel for layout readiness (Phase 2 WeightIndex). + */ +#include "neuron/container/network/weights.hpp" +#include "neuron/model_data.hpp" + +#include +#include + +namespace neuron::container::network::Weight { + +/** + * @brief Allocate @p n weight SoA rows, optionally mirroring heap values. + */ +inline std::vector allocate_weight_rows(int n, double const* mirror = nullptr) { + std::vector rows; + if (n <= 0) { + return rows; + } + auto& store = neuron::model().weights(); + rows.reserve(static_cast(n)); + for (int i = 0; i < n; ++i) { + rows.emplace_back(store); + rows.back().value() = mirror ? mirror[i] : 0.; + } + return rows; +} + +/** @brief Copy heap weight values into already-allocated SoA rows. */ +inline void mirror_weights_to_soa(std::vector& rows, double const* heap, int n) { + if (!heap) { + return; + } + auto const m = std::min(static_cast(rows.size()), n); + for (int i = 0; i < m; ++i) { + rows[i].value() = heap[i]; + } +} + +} // namespace neuron::container::network::Weight diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index 17c49eee2b..ddb49ee5ef 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -3,6 +3,7 @@ #undef check #include "neuron/container/data_handle.hpp" +#include "neuron/container/network/weight_block.hpp" #include "nrnmpi.h" #include "nrnneosm.h" #include "pool.hpp" @@ -117,6 +118,8 @@ class NetCon: public DiscreteEvent { Object* obj_; int cnt_; bool active_; + /** @brief Phase 1 dual-write owners for Weight SoA rows (see weight_block.hpp). */ + std::vector weight_soa_{}; static unsigned long netcon_send_active_; static unsigned long netcon_send_inactive_; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 18b2b5fdd4..3a055a3882 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -645,6 +645,11 @@ static double nc_setpost(void* v) { if (d->cnt_ != cnt) { d->cnt_ = cnt; delete[] std::exchange(d->weight_, new double[d->cnt_]); + for (int i = 0; i < d->cnt_; ++i) { + d->weight_[i] = 0.0; + } + d->weight_soa_ = + neuron::container::network::Weight::allocate_weight_rows(d->cnt_, d->weight_); } return 0.; } @@ -4714,6 +4719,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { cnt_ = 1; weight_ = new double[cnt_]; weight_[0] = 0.0; + weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(cnt_, weight_); return; } target_ = ob2pntproc(target); @@ -4731,6 +4737,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { for (int i = 0; i < cnt_; ++i) { weight_[i] = 0.0; } + weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(cnt_, weight_); } } @@ -4741,6 +4748,7 @@ NetCon::~NetCon() { if (cnt_) { delete[] weight_; } + weight_soa_.clear(); #if DISCRETE_EVENT_OBSERVER if (target_) { ObjObservable::Detach(target_->ob, this); diff --git a/src/nrnoc/multicore.cpp b/src/nrnoc/multicore.cpp index 10286a6bc0..a71d144eff 100644 --- a/src/nrnoc/multicore.cpp +++ b/src/nrnoc/multicore.cpp @@ -616,11 +616,13 @@ printf("thread_memblist_setup %lx v_node_count=%d ncell=%d end=%d\n", (long)nth, for (int i = 0; i < tml->ml->nodecount; ++i) { auto* pnt = tml->ml->prop[i]->dparam[1].get(); pnt->_vnt = _nt; + nrn_point_process_soa_sync(pnt); } } else { for (int i = 0; i < tml->ml->nodecount; ++i) { auto* pnt = tml->ml->pdata[i][1].get(); pnt->_vnt = _nt; + nrn_point_process_soa_sync(pnt); } } } diff --git a/src/nrnoc/point.cpp b/src/nrnoc/point.cpp index b822181d5b..6a3f9f1985 100644 --- a/src/nrnoc/point.cpp +++ b/src/nrnoc/point.cpp @@ -30,6 +30,26 @@ Prop* nrn_point_prop_; void (*nrnpy_o2loc_p_)(Object*, Section**, double*); void (*nrnpy_o2loc2_p_)(Object*, Section**, double*); +void nrn_point_process_soa_sync(Point_process* pnt) { + if (!pnt) { + return; + } + auto& h = pnt->_soa; + if (pnt->prop) { + h.mech_type() = pnt->prop->_type; + // Prop for a point process always owns a mechanism SoA row. + h.instance() = static_cast(pnt->prop->current_row()); + } else { + h.mech_type() = -1; + h.instance() = -1; + } + if (pnt->_vnt) { + h.thread_id() = static_cast(pnt->_vnt)->id; + } else { + h.thread_id() = -1; + } +} + void* create_point_process(int pointtype, Object* ho) { auto* const pp = new Point_process{}; pp->ob = ho; @@ -40,6 +60,7 @@ void* create_point_process(int pointtype, Object* ho) { if (ho && ho->ctemplate->steer && ifarg(1)) { loc_point_process(pointtype, (void*) pp); } + nrn_point_process_soa_sync(pp); return pp; } @@ -132,6 +153,7 @@ void nrn_loc_point_process(int pointtype, Point_process* pnt, Section* sec, Node hoc_template_notify(pnt->ob, 2); } } + nrn_point_process_soa_sync(pnt); } static void create_artcell_prop(Point_process* pnt, short type) { @@ -148,6 +170,7 @@ static void create_artcell_prop(Point_process* pnt, short type) { hoc_template_notify(pnt->ob, 2); } } + nrn_point_process_soa_sync(pnt); } void nrn_relocate_old_points(Section* oldsec, Node* oldnode, Section* sec, Node* node) { @@ -310,6 +333,7 @@ void connect_point_process_pointer(void) { static void free_one_point(Point_process* pnt) { auto* p = pnt->prop; if (!p) { + nrn_point_process_soa_sync(pnt); return; } if (!nrn_is_artificial_[p->_type]) { @@ -341,6 +365,7 @@ static void free_one_point(Point_process* pnt) { section_unref(pnt->sec); } pnt->sec = (Section*) 0; + nrn_point_process_soa_sync(pnt); } // called from prop_free diff --git a/src/nrnoc/section_fwd.hpp b/src/nrnoc/section_fwd.hpp index fe1a955f6c..b2d8ea2043 100644 --- a/src/nrnoc/section_fwd.hpp +++ b/src/nrnoc/section_fwd.hpp @@ -1,6 +1,8 @@ #pragma once #include "multicore.h" #include "neuron/container/generic_data_handle.hpp" +#include "neuron/container/network/point_process.hpp" +#include "neuron/model_data.hpp" #include "nrnredef.h" /** * @file section_fwd.hpp @@ -83,4 +85,12 @@ struct Point_process { void* presyn_{}; /* non-threshold presynapse for NetCon */ void* nvi_{}; /* NrnVarIntegrator (for local step method) */ void* _vnt{}; /* NrnThread* (for NET_RECEIVE and multicore) */ + /** @brief Phase 1 dual-write: SoA row (Instance, MechType, ThreadId). + * @see doc/network-soa-phase0.md §5.2 + */ + neuron::container::network::PointProcess::owning_handle _soa{ + neuron::model().point_processes()}; }; + +/** @brief Sync legacy Point_process fields into the network SoA row. */ +void nrn_point_process_soa_sync(Point_process* pnt); diff --git a/src/nrnoc/treeset.cpp b/src/nrnoc/treeset.cpp index e24a45b196..1d20b001ba 100644 --- a/src/nrnoc/treeset.cpp +++ b/src/nrnoc/treeset.cpp @@ -1694,6 +1694,7 @@ void v_setup_vectors(void) { pnt->_vnt = nrn_threads + nti; nti = (nti + 1) % nrn_nthread; } + nrn_point_process_soa_sync(pnt); auto const tid = static_cast(pnt->_vnt)->id; ++thread_counts[tid]; // pnt->_i_instance = j; diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index bd8eff8a27..314a754c90 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -1,4 +1,5 @@ #include "neuron/container/network/point_process.hpp" +#include "neuron/container/network/weight_block.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/model_data.hpp" #include "section.h" @@ -105,6 +106,28 @@ TEST_CASE("SOA-backed PointProcess structure", "[Neuron][data_structures][networ } } +TEST_CASE("Point_process dual-write into network SoA", + "[Neuron][data_structures][network][point_process][dualwrite]") { + auto& storage = neuron::model().point_processes(); + auto const before = storage.size(); + GIVEN("A default-constructed Point_process shell") { + // Allocates an SoA row via Point_process::_soa (Phase 1 dual-write). + auto* pp = new Point_process{}; + THEN("SoA size grows by one and fields are defaults until prop is set") { + REQUIRE(storage.size() == before + 1); + REQUIRE(pp->_soa.mech_type() == -1); + REQUIRE(pp->_soa.instance() == -1); + REQUIRE(pp->_soa.thread_id() == -1); + } + WHEN("The Point_process is destroyed") { + delete pp; + THEN("The SoA row is released") { + REQUIRE(storage.size() == before); + } + } + } +} + TEST_CASE("SOA-backed Weight structure", "[Neuron][data_structures][network][weights]") { auto& storage = neuron::model().weights(); REQUIRE(storage.size() == 0); @@ -154,4 +177,21 @@ TEST_CASE("SOA-backed Weight structure", "[Neuron][data_structures][network][wei } } } + + GIVEN("allocate_weight_rows dual-write helper") { + double heap[3] = {1.5, 2.5, 3.5}; + auto rows = Weight::allocate_weight_rows(3, heap); + THEN("SoA mirrors the heap values") { + REQUIRE(rows.size() == 3); + REQUIRE(storage.size() == 3); + REQUIRE(rows[0].value() == 1.5); + REQUIRE(rows[1].value() == 2.5); + REQUIRE(rows[2].value() == 3.5); + } + WHEN("heap is updated and remirrored") { + heap[1] = 9.0; + Weight::mirror_weights_to_soa(rows, heap, 3); + REQUIRE(rows[1].value() == 9.0); + } + } } From 69ceb738e1f6624a1d454b45fc9eca753c03ff6d Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 10:59:06 -0400 Subject: [PATCH 05/33] Phase 2: NetCon SoA dual-write and HOC weight via Weight SoA MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add network::NetCon storage (Target, WeightIndex, WeightCount, Delay, Active, SrcPreSyn) on Model. NetCon owns an SoA row and syncs fields on create/update. HOC weight() steers into Weight SoA data_handles; deliver mirrors SoA↔heap around pnt_receive so NET_RECEIVE still sees double*. --- GROK-NETWORK-SOA.md | 4 +- doc/network-soa-phase0.md | 2 +- src/neuron/container/memory_usage.hpp | 6 + src/neuron/container/network/netcon.hpp | 162 ++++++++++++++++++++++++ src/neuron/model_data.hpp | 15 +++ src/nrncvode/netcon.h | 11 ++ src/nrncvode/netcvode.cpp | 66 +++++++++- src/nrniv/memory_usage.cpp | 7 +- src/nrnoc/container.cpp | 7 + test/unit_tests/container/network.cpp | 57 +++++++++ 10 files changed, 331 insertions(+), 6 deletions(-) create mode 100644 src/neuron/container/network/netcon.hpp diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 0a2c725b10..20561a236b 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -69,8 +69,8 @@ psolve hot path (deliver, threshold, fanout) — index-based | Phase | Content | Gate | |-------|---------|------| | **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec done (authoritative) | -| **1** | `Point_process` + `weights` SoA + dual-write create/destroy | handles survive permute; PP/NetCon dual-write | -| **2** | `NetCon` SoA, HOC `weight()` → flat weights | CPU ringtest delivery | +| **1** | `Point_process` + `weights` SoA + dual-write create/destroy | handles survive permute; PP dual-write | +| **2** | `NetCon` SoA, HOC `weight()` → flat Weight SoA | dual-write; unit tests; CPU ringtest when available | | **3** | `PreSyn` `nc_index`/`nc_cnt` fanout | CPU spike parity @ 100 ms | | **4** | `SelfEvent` indices, `pnt_receive(weight_index)` | ExpSyn @ 1.025 ms CPU | | **5** | (gpu-native track) net buffers Stages 2–3 | GPU parity | diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index 18eebc0788..969a223325 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -302,7 +302,7 @@ If a structural pool is useful later, tags match the above. Gate is `pnt_receive |-------|-----------|------| | **0** | Spec (this doc) | Spec reviewed | | **1** | `PointProcess` + `Weight` | Handles survive permute; dual-write on create/destroy | -| **2** | `NetCon` SoA; HOC `weight()` → flat weights | CPU ringtest delivery | +| **2** | `NetCon` SoA; HOC `weight()` → flat Weight SoA | dual-write; handles survive permute | | **3** | `PreSyn` fanout `NcIndex`/`NcCount` | CPU spike parity @ 100 ms | | **4** | SelfEvent indices; `pnt_receive` by weight index | ExpSyn @ 1.025 ms CPU | | **5** | (gpu-native track) net buffers | GPU parity — **not this branch** | diff --git a/src/neuron/container/memory_usage.hpp b/src/neuron/container/memory_usage.hpp index a152ded36b..db1cf6a14d 100644 --- a/src/neuron/container/memory_usage.hpp +++ b/src/neuron/container/memory_usage.hpp @@ -82,11 +82,15 @@ struct ModelMemoryUsage { /// @brief Network SoA: flat weight pool. StorageMemoryUsage weights{}; + /// @brief Network SoA: NetCon integration rows. + StorageMemoryUsage netcons{}; + const ModelMemoryUsage& operator+=(const ModelMemoryUsage& other) { nodes += other.nodes; mechanisms += other.mechanisms; point_processes += other.point_processes; weights += other.weights; + netcons += other.netcons; return *this; } @@ -96,6 +100,7 @@ struct ModelMemoryUsage { total += mechanisms.compute_total(); total += point_processes.compute_total(); total += weights.compute_total(); + total += netcons.compute_total(); return total; } @@ -199,6 +204,7 @@ struct MemoryUsageSummary { add(model.mechanisms); add(model.point_processes); add(model.weights); + add(model.netcons); } void add(const cache::ModelMemoryUsage& model) { diff --git a/src/neuron/container/network/netcon.hpp b/src/neuron/container/network/netcon.hpp new file mode 100644 index 0000000000..d626ff457e --- /dev/null +++ b/src/neuron/container/network/netcon.hpp @@ -0,0 +1,162 @@ +#pragma once +/** + * @file network/netcon.hpp + * @brief SoA storage for integration-hot NetCon fields. + * + * CoreNEURON reference: target_, u.weight_index_, delay_, active_. + * Design: doc/network-soa-phase0.md §5.4, §5.4.1. + * + * Phase 2 dual-write: C++ NetCon keeps DiscreteEvent + pointers; this SoA holds + * the CoreNEURON-shaped columns. HOC weight() steers into Weight SoA when + * weight_soa_ is populated (see netcvode.cpp). + */ +#include "neuron/container/data_handle.hpp" +#include "neuron/container/soa_container.hpp" +#include "neuron/container/view_utils.hpp" + +#include +#include + +namespace neuron::container::network::NetCon { +namespace field { + +/** @brief Row in PointProcess storage (CoreNEURON target). */ +struct Target { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Base row in Weight storage for this NetCon's weight block. */ +struct WeightIndex { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Number of weights (pnt_receive_size). */ +struct WeightCount { + using type = int; + constexpr type default_value() const { + return 0; + } +}; + +/** @brief Delivery delay (ms). */ +struct Delay { + using type = double; + constexpr type default_value() const { + return 1.0; + } +}; + +/** @brief Active flag as int (GPU-friendly). */ +struct Active { + using type = int; + constexpr type default_value() const { + return 1; + } +}; + +/** + * @brief Source PreSyn row, or -1. + * + * Phase 2: remains -1 until PreSyn SoA exists (Phase 3). Reverse edge only. + */ +struct SrcPreSyn { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +} // namespace field + +/** + * @brief Public API for NetCon handles (owning and non-owning). + * @tparam Identifier owning_identifier or non_owning_identifier for storage. + */ +template +struct handle_interface: handle_base { + using base_type = handle_base; + using base_type::base_type; + + [[nodiscard]] field::Target::type& target() { + return this->template get(); + } + [[nodiscard]] field::Target::type const& target() const { + return this->template get(); + } + + [[nodiscard]] field::WeightIndex::type& weight_index() { + return this->template get(); + } + [[nodiscard]] field::WeightIndex::type const& weight_index() const { + return this->template get(); + } + + [[nodiscard]] field::WeightCount::type& weight_count() { + return this->template get(); + } + [[nodiscard]] field::WeightCount::type const& weight_count() const { + return this->template get(); + } + + [[nodiscard]] field::Delay::type& delay() { + return this->template get(); + } + [[nodiscard]] field::Delay::type const& delay() const { + return this->template get(); + } + + [[nodiscard]] field::Active::type& active() { + return this->template get(); + } + [[nodiscard]] field::Active::type const& active() const { + return this->template get(); + } + + [[nodiscard]] field::SrcPreSyn::type& src_presyn() { + return this->template get(); + } + [[nodiscard]] field::SrcPreSyn::type const& src_presyn() const { + return this->template get(); + } + + friend std::ostream& operator<<(std::ostream& os, handle_interface const& handle) { + if (handle.id()) { + return os << "NetCon{" << handle.id() << '/' << handle.underlying_storage().size() + << " target=" << handle.target() << " widx=" << handle.weight_index() + << " wcnt=" << handle.weight_count() << " delay=" << handle.delay() + << " active=" << handle.active() << " src=" << handle.src_presyn() << '}'; + } + return os << "NetCon{null}"; + } +}; + +/** @brief Underlying storage for all NetCon integration rows. */ +struct storage: soa { + [[nodiscard]] std::string_view name() const { + return "network::NetCon"; + } +}; + +using handle = handle_interface>; + +struct owning_handle: handle_interface> { + using base_type = handle_interface>; + using base_type::base_type; + + [[nodiscard]] handle non_owning_handle() { + return non_owning_identifier{&underlying_storage(), id()}; + } +}; +} // namespace neuron::container::network::NetCon diff --git a/src/neuron/model_data.hpp b/src/neuron/model_data.hpp index 6284287d33..69ea00c5d7 100644 --- a/src/neuron/model_data.hpp +++ b/src/neuron/model_data.hpp @@ -2,6 +2,7 @@ #include "neuron/cache/model_data.hpp" #include "neuron/container/mechanism_data.hpp" #include "neuron/container/memory_usage.hpp" +#include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/container/node_data.hpp" @@ -53,6 +54,16 @@ struct Model { return m_weights; } + /** @brief Access SoA storage for NetCon integration fields (Phase 2). + * @see doc/network-soa-phase0.md §5.4 + */ + container::network::NetCon::storage& netcons() { + return m_netcons; + } + container::network::NetCon::storage const& netcons() const { + return m_netcons; + } + /** @brief Apply a function to each non-null Mechanism. */ template @@ -144,6 +155,7 @@ struct Model { apply_to_mechanisms([](auto& mech_data) { mech_data.shrink_to_fit(); }); m_point_processes.shrink_to_fit(); m_weights.shrink_to_fit(); + m_netcons.shrink_to_fit(); } private: @@ -181,6 +193,9 @@ struct Model { /** @brief Network SoA: flat weight pool (Phase 1). */ container::network::Weight::storage m_weights{}; + /** @brief Network SoA: NetCon integration rows (Phase 2). */ + container::network::NetCon::storage m_netcons{}; + /** * @brief Backing storage for defer_delete helper. */ diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index ddb49ee5ef..b8d58f3b6e 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -3,7 +3,9 @@ #undef check #include "neuron/container/data_handle.hpp" +#include "neuron/container/network/netcon.hpp" #include "neuron/container/network/weight_block.hpp" +#include "neuron/model_data.hpp" #include "nrnmpi.h" #include "nrnneosm.h" #include "pool.hpp" @@ -111,6 +113,13 @@ class NetCon: public DiscreteEvent { void replace_src(PreSyn*); virtual void disconnect(Observable*); + /** @brief Sync legacy NetCon fields into NetCon + Weight SoA (Phase 2). */ + void soa_sync(); + /** @brief Copy Weight SoA values → heap weight_ (before pnt_receive). */ + void weights_soa_to_heap(); + /** @brief Copy heap weight_ → Weight SoA (after pnt_receive / MOD writes). */ + void weights_heap_to_soa(); + double delay_; PreSyn* src_; Point_process* target_; @@ -120,6 +129,8 @@ class NetCon: public DiscreteEvent { bool active_; /** @brief Phase 1 dual-write owners for Weight SoA rows (see weight_block.hpp). */ std::vector weight_soa_{}; + /** @brief Phase 2 dual-write: NetCon integration row. */ + neuron::container::network::NetCon::owning_handle _soa{neuron::model().netcons()}; static unsigned long netcon_send_active_; static unsigned long netcon_send_inactive_; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 3a055a3882..b41c9adb31 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -118,8 +118,12 @@ double nrn_netcon_get_delay(NetCon* nc) { } void nrn_netcon_set_delay(NetCon* nc, double d) { nc->delay_ = d; + nc->soa_sync(); } int nrn_netcon_weight(NetCon* nc, double** pw) { + // SoA is HOC source of truth when weight_soa_ is populated; keep heap current + // for MOD code that writes through the returned pointer. + nc->weights_soa_to_heap(); *pw = nc->weight_; return nc->cnt_; } @@ -158,6 +162,7 @@ int nrn_netcon_info(NetCon* nc, double** pw, Point_process** target, double** th *target = (nc->target_) ? nc->target_ : (Point_process*) 0; *th = (nc->src_) ? &(nc->src_->threshold_) : (double*) 0; *del = &nc->delay_; + nc->weights_soa_to_heap(); *pw = nc->weight_; return nc->cnt_; } @@ -644,13 +649,15 @@ static double nc_setpost(void* v) { } if (d->cnt_ != cnt) { d->cnt_ = cnt; - delete[] std::exchange(d->weight_, new double[d->cnt_]); + delete[] d->weight_; + d->weight_ = d->cnt_ ? new double[d->cnt_] : nullptr; for (int i = 0; i < d->cnt_; ++i) { d->weight_[i] = 0.0; } d->weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(d->cnt_, d->weight_); } + d->soa_sync(); return 0.; } @@ -668,6 +675,7 @@ static double nc_active(void* v) { bool a = d->active_; if (d->target_ && ifarg(1)) { d->active_ = bool(chkarg(1, 0, 1)); + d->soa_sync(); } hoc_return_type_code = HocReturnType::boolean; return double(a); @@ -780,6 +788,7 @@ static void steer_val(void* v) { Symbol* s = hoc_spop(); if (strcmp(s->name, "delay") == 0) { d->chksrc(); + // HOC writes delay_; soa_sync mirrors into NetCon SoA. hoc_pushpx(&d->delay_); d->src_->use_min_delay_ = 0; } else if (strcmp(s->name, "weight") == 0) { @@ -788,7 +797,12 @@ static void steer_val(void* v) { s->arayinfo->sub[0] = d->cnt_; index = hoc_araypt(s, SYMBOL); } - hoc_pushpx(d->weight_ + index); + // Phase 2: HOC weight() steers into flat Weight SoA when dual-write rows exist. + if (index >= 0 && index < static_cast(d->weight_soa_.size())) { + hoc_push(d->weight_soa_[index].value_handle()); + } else { + hoc_pushpx(d->weight_ + index); + } } else if (strcmp(s->name, "x") == 0) { static double dummy = 0.; d->chksrc(); @@ -2984,7 +2998,11 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); + // Weight SoA is HOC source of truth; heap is the pnt_receive buffer. + weights_soa_to_heap(); POINT_RECEIVE(type, target_, weight_, 0); + // Capture any weight mutations by NET_RECEIVE / MOD code. + weights_heap_to_soa(); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3000,7 +3018,9 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { assert(target_); int type = target_->prop->_type; STATISTICS(netcon_deliver_); + weights_soa_to_heap(); POINT_RECEIVE(type, target_, weight_, 0); + weights_heap_to_soa(); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -4704,6 +4724,43 @@ void DiscreteEvent::savestate_write(FILE* f) { fprintf(f, "%d\n", DiscreteEventType); } +void NetCon::weights_soa_to_heap() { + if (!weight_ || weight_soa_.empty()) { + return; + } + auto const n = std::min(cnt_, static_cast(weight_soa_.size())); + for (int i = 0; i < n; ++i) { + weight_[i] = weight_soa_[i].value(); + } +} + +void NetCon::weights_heap_to_soa() { + if (!weight_ || weight_soa_.empty()) { + return; + } + neuron::container::network::Weight::mirror_weights_to_soa(weight_soa_, weight_, cnt_); +} + +void NetCon::soa_sync() { + _soa.delay() = delay_; + _soa.active() = active_ ? 1 : 0; + _soa.weight_count() = cnt_; + if (target_) { + // Point_process dual-write row (Phase 1). + _soa.target() = static_cast(target_->_soa.current_row()); + } else { + _soa.target() = -1; + } + // PreSyn SoA is Phase 3; reverse edge stays -1 until then. + _soa.src_presyn() = -1; + if (!weight_soa_.empty()) { + _soa.weight_index() = static_cast(weight_soa_.front().current_row()); + weights_heap_to_soa(); + } else { + _soa.weight_index() = -1; + } +} + NetCon::NetCon(PreSyn* src, Object* target) { NetConSave::invalid(); obj_ = nullptr; @@ -4720,6 +4777,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { weight_ = new double[cnt_]; weight_[0] = 0.0; weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(cnt_, weight_); + soa_sync(); return; } target_ = ob2pntproc(target); @@ -4739,6 +4797,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { } weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(cnt_, weight_); } + soa_sync(); } NetCon::~NetCon() { @@ -4749,6 +4808,7 @@ NetCon::~NetCon() { delete[] weight_; } weight_soa_.clear(); + // _soa owning_handle frees the NetCon SoA row. #if DISCRETE_EVENT_OBSERVER if (target_) { ObjObservable::Detach(target_->ob, this); @@ -4771,6 +4831,7 @@ void NetCon::rmsrc() { } } src_ = nullptr; + soa_sync(); } void NetCon::replace_src(PreSyn* p) { @@ -4780,6 +4841,7 @@ void NetCon::replace_src(PreSyn* p) { src_->dil_.push_back(this); src_->use_min_delay_ = 0; } + soa_sync(); } DiscreteEvent* NetCon::savestate_save() { diff --git a/src/nrniv/memory_usage.cpp b/src/nrniv/memory_usage.cpp index 17439d6521..2289aa2ecd 100644 --- a/src/nrniv/memory_usage.cpp +++ b/src/nrniv/memory_usage.cpp @@ -15,8 +15,9 @@ ModelMemoryUsage memory_usage(const Model& model) { auto point_processes = memory_usage(model.point_processes()); auto weights = memory_usage(model.weights()); + auto netcons = memory_usage(model.netcons()); - return {nodes, mechanisms, point_processes, weights}; + return {nodes, mechanisms, point_processes, weights, netcons}; } cache::ModelMemoryUsage memory_usage(const std::optional& model) { @@ -129,6 +130,10 @@ std::string format_memory_usage(const MemoryUsage& usage) { os << " data " << format_memory_usage(model.weights.heavy_data) << "\n"; os << " stable_identifiers " << format_memory_usage(model.weights.stable_identifiers) << "\n"; + os << " network::NetCon \n"; + os << " data " << format_memory_usage(model.netcons.heavy_data) << "\n"; + os << " stable_identifiers " << format_memory_usage(model.netcons.stable_identifiers) + << "\n"; os << "cache::Model \n"; os << " threads " << format_memory_usage(cache_model.threads) << "\n"; os << " mechanisms " << format_memory_usage(cache_model.mechanisms) << "\n"; diff --git a/src/nrnoc/container.cpp b/src/nrnoc/container.cpp index 84cbc92bb2..1b1c61a58c 100644 --- a/src/nrnoc/container.cpp +++ b/src/nrnoc/container.cpp @@ -17,6 +17,7 @@ Model::Model() { m_node_data.set_unsorted_callback(invalidate_cache); m_point_processes.set_unsorted_callback(invalidate_cache); m_weights.set_unsorted_callback(invalidate_cache); + m_netcons.set_unsorted_callback(invalidate_cache); // needs some re-organisation if we ever want to support multiple Model instances assert(!container::detail::defer_delete_storage); container::detail::defer_delete_storage = &m_ptrs_for_deferred_deletion; @@ -46,6 +47,9 @@ std::unique_ptr Model::find_container_info(void if (auto maybe_info = m_weights.find_container_info(cont); maybe_info) { return maybe_info; } + if (auto maybe_info = m_netcons.find_container_info(cont); maybe_info) { + return maybe_info; + } return {}; } @@ -158,6 +162,9 @@ generic_data_handle promote_or_clear(generic_data_handle gdh) { if (auto h = model.weights().find_data_handle(gdh); h.refers_to_a_modern_data_structure()) { return h; } + if (auto h = model.netcons().find_data_handle(gdh); h.refers_to_a_modern_data_structure()) { + return h; + } return {}; } } // namespace detail diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index 314a754c90..3a3d499710 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -1,3 +1,4 @@ +#include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/container/network/weights.hpp" @@ -195,3 +196,59 @@ TEST_CASE("SOA-backed Weight structure", "[Neuron][data_structures][network][wei } } } + +TEST_CASE("SOA-backed NetCon structure", "[Neuron][data_structures][network][netcon]") { + auto& storage = neuron::model().netcons(); + auto& wstore = neuron::model().weights(); + REQUIRE(storage.size() == 0); + + GIVEN("Default-constructed NetCon SoA rows") { + NetCon::owning_handle a{storage}; + NetCon::owning_handle b{storage}; + THEN("Defaults match field tags") { + REQUIRE(a.target() == -1); + REQUIRE(a.weight_index() == -1); + REQUIRE(a.weight_count() == 0); + REQUIRE(a.delay() == 1.0); + REQUIRE(a.active() == 1); + REQUIRE(a.src_presyn() == -1); + REQUIRE(storage.size() == 2); + } + THEN("Fields round-trip and survive reverse permutation") { + a.target() = 3; + a.weight_index() = 10; + a.weight_count() = 2; + a.delay() = 0.5; + a.active() = 0; + b.target() = 7; + b.delay() = 2.0; + { + auto token = storage.issue_frozen_token(); + storage.mark_as_sorted(token); + } + std::vector perm{0, 1}; + std::rotate(perm.begin(), std::next(perm.begin()), perm.end()); + storage.apply_reverse_permutation(std::move(perm)); + REQUIRE(a.target() == 3); + REQUIRE(a.weight_index() == 10); + REQUIRE(a.weight_count() == 2); + REQUIRE(a.delay() == 0.5); + REQUIRE(a.active() == 0); + REQUIRE(b.target() == 7); + REQUIRE(b.delay() == 2.0); + } + } + + GIVEN("Weight block linked like Phase 2 dual-write") { + auto wrows = Weight::allocate_weight_rows(2, nullptr); + wrows[0].value() = 0.1; + wrows[1].value() = 0.2; + NetCon::owning_handle nc{storage}; + nc.weight_index() = static_cast(wrows[0].current_row()); + nc.weight_count() = 2; + THEN("WeightIndex points at first SoA weight row") { + REQUIRE(wstore.get(nc.weight_index()) == 0.1); + REQUIRE(wstore.get(nc.weight_index() + 1) == 0.2); + } + } +} From 46eaa27e1db4047b0b55a751b576360e99da7f4c Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 11:44:07 -0400 Subject: [PATCH 06/33] Phase 3: PreSyn SoA and NcIndex/NcCount fanout dual-write Add network::PreSyn storage on Model and embed owning handles on PreSyn. Rebuild a global NetCon* fanout order from dil_ (CoreNEURON-shaped ranges); send/deliver/fanout iterate that order with dil_ fallback. NetCon::SrcPreSyn points at the source PreSyn SoA row. --- GROK-NETWORK-SOA.md | 2 +- doc/network-soa-phase0.md | 2 +- src/neuron/container/memory_usage.hpp | 6 + src/neuron/container/network/presyn.hpp | 173 ++++++++++++++++++++++++ src/neuron/model_data.hpp | 15 ++ src/nrncvode/netcon.h | 10 ++ src/nrncvode/netcvode.cpp | 100 ++++++++++++-- src/nrniv/memory_usage.cpp | 7 +- src/nrniv/netpar.cpp | 2 + src/nrnoc/container.cpp | 7 + test/unit_tests/container/network.cpp | 46 +++++++ 11 files changed, 357 insertions(+), 13 deletions(-) create mode 100644 src/neuron/container/network/presyn.hpp diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 20561a236b..3c29c6c1dc 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -71,7 +71,7 @@ psolve hot path (deliver, threshold, fanout) — index-based | **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec done (authoritative) | | **1** | `Point_process` + `weights` SoA + dual-write create/destroy | handles survive permute; PP dual-write | | **2** | `NetCon` SoA, HOC `weight()` → flat Weight SoA | dual-write; unit tests; CPU ringtest when available | -| **3** | `PreSyn` `nc_index`/`nc_cnt` fanout | CPU spike parity @ 100 ms | +| **3** | `PreSyn` SoA + `NcIndex`/`NcCount` fanout order | dual-write; send/deliver use fanout | | **4** | `SelfEvent` indices, `pnt_receive(weight_index)` | ExpSyn @ 1.025 ms CPU | | **5** | (gpu-native track) net buffers Stages 2–3 | GPU parity | diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index 969a223325..d69cb98f2e 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -303,7 +303,7 @@ If a structural pool is useful later, tags match the above. Gate is `pnt_receive | **0** | Spec (this doc) | Spec reviewed | | **1** | `PointProcess` + `Weight` | Handles survive permute; dual-write on create/destroy | | **2** | `NetCon` SoA; HOC `weight()` → flat Weight SoA | dual-write; handles survive permute | -| **3** | `PreSyn` fanout `NcIndex`/`NcCount` | CPU spike parity @ 100 ms | +| **3** | `PreSyn` SoA + fanout `NcIndex`/`NcCount` | dual-write; hot path uses fanout order | | **4** | SelfEvent indices; `pnt_receive` by weight index | ExpSyn @ 1.025 ms CPU | | **5** | (gpu-native track) net buffers | GPU parity — **not this branch** | diff --git a/src/neuron/container/memory_usage.hpp b/src/neuron/container/memory_usage.hpp index db1cf6a14d..6e7c83afbb 100644 --- a/src/neuron/container/memory_usage.hpp +++ b/src/neuron/container/memory_usage.hpp @@ -85,12 +85,16 @@ struct ModelMemoryUsage { /// @brief Network SoA: NetCon integration rows. StorageMemoryUsage netcons{}; + /// @brief Network SoA: PreSyn integration rows. + StorageMemoryUsage presyns{}; + const ModelMemoryUsage& operator+=(const ModelMemoryUsage& other) { nodes += other.nodes; mechanisms += other.mechanisms; point_processes += other.point_processes; weights += other.weights; netcons += other.netcons; + presyns += other.presyns; return *this; } @@ -101,6 +105,7 @@ struct ModelMemoryUsage { total += point_processes.compute_total(); total += weights.compute_total(); total += netcons.compute_total(); + total += presyns.compute_total(); return total; } @@ -205,6 +210,7 @@ struct MemoryUsageSummary { add(model.point_processes); add(model.weights); add(model.netcons); + add(model.presyns); } void add(const cache::ModelMemoryUsage& model) { diff --git a/src/neuron/container/network/presyn.hpp b/src/neuron/container/network/presyn.hpp new file mode 100644 index 0000000000..7fc9c68212 --- /dev/null +++ b/src/neuron/container/network/presyn.hpp @@ -0,0 +1,173 @@ +#pragma once +/** + * @file network/presyn.hpp + * @brief SoA storage for integration-hot PreSyn fields. + * + * CoreNEURON reference: nc_index_, nc_cnt_, thvar_index_, threshold_, gid_. + * Design: doc/network-soa-phase0.md §5.5. + * + * Phase 3 dual-write: legacy PreSyn keeps dil_ and thvar_; this SoA holds + * CoreNEURON-shaped columns. Fanout order is rebuilt into a global NetCon* + * table (see netcvode.cpp); NcIndex/NcCount describe ranges in that table. + */ +#include "neuron/container/data_handle.hpp" +#include "neuron/container/soa_container.hpp" +#include "neuron/container/view_utils.hpp" + +#include +#include + +namespace neuron::container::network::PreSyn { +namespace field { + +/** @brief Spike threshold. */ +struct Threshold { + using type = double; + constexpr type default_value() const { + return 10.; + } +}; + +/** @brief Output gid, or -1. */ +struct Gid { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Start index into global fanout order (NetCon*). */ +struct NcIndex { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Fanout count (replaces dil_.size() on hot path when sorted). */ +struct NcCount { + using type = int; + constexpr type default_value() const { + return 0; + } +}; + +/** @brief CoreNEURON output_index_ (MPI / spike compression). */ +struct OutputIndex { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** + * @brief Optional denormalized node-voltage row for threshold scans. + * + * Canonical threshold source remains PreSyn::thvar_ (data_handle). -1 if unknown. + */ +struct ThVarRow { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +/** @brief Owning NrnThread id. */ +struct ThreadId { + using type = int; + constexpr type default_value() const { + return -1; + } +}; + +} // namespace field + +template +struct handle_interface: handle_base { + using base_type = handle_base; + using base_type::base_type; + + [[nodiscard]] field::Threshold::type& threshold() { + return this->template get(); + } + [[nodiscard]] field::Threshold::type const& threshold() const { + return this->template get(); + } + + [[nodiscard]] field::Gid::type& gid() { + return this->template get(); + } + [[nodiscard]] field::Gid::type const& gid() const { + return this->template get(); + } + + [[nodiscard]] field::NcIndex::type& nc_index() { + return this->template get(); + } + [[nodiscard]] field::NcIndex::type const& nc_index() const { + return this->template get(); + } + + [[nodiscard]] field::NcCount::type& nc_count() { + return this->template get(); + } + [[nodiscard]] field::NcCount::type const& nc_count() const { + return this->template get(); + } + + [[nodiscard]] field::OutputIndex::type& output_index() { + return this->template get(); + } + [[nodiscard]] field::OutputIndex::type const& output_index() const { + return this->template get(); + } + + [[nodiscard]] field::ThVarRow::type& thvar_row() { + return this->template get(); + } + [[nodiscard]] field::ThVarRow::type const& thvar_row() const { + return this->template get(); + } + + [[nodiscard]] field::ThreadId::type& thread_id() { + return this->template get(); + } + [[nodiscard]] field::ThreadId::type const& thread_id() const { + return this->template get(); + } + + friend std::ostream& operator<<(std::ostream& os, handle_interface const& handle) { + if (handle.id()) { + return os << "PreSyn{" << handle.id() << '/' << handle.underlying_storage().size() + << " thr=" << handle.threshold() << " gid=" << handle.gid() + << " nc=[" << handle.nc_index() << "," << handle.nc_count() << ")" + << " out=" << handle.output_index() << " tid=" << handle.thread_id() << '}'; + } + return os << "PreSyn{null}"; + } +}; + +struct storage: soa { + [[nodiscard]] std::string_view name() const { + return "network::PreSyn"; + } +}; + +using handle = handle_interface>; + +struct owning_handle: handle_interface> { + using base_type = handle_interface>; + using base_type::base_type; + + [[nodiscard]] handle non_owning_handle() { + return non_owning_identifier{&underlying_storage(), id()}; + } +}; +} // namespace neuron::container::network::PreSyn diff --git a/src/neuron/model_data.hpp b/src/neuron/model_data.hpp index 69ea00c5d7..9174ebf651 100644 --- a/src/neuron/model_data.hpp +++ b/src/neuron/model_data.hpp @@ -4,6 +4,7 @@ #include "neuron/container/memory_usage.hpp" #include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" +#include "neuron/container/network/presyn.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/container/node_data.hpp" #include "neuron/model_data_fwd.hpp" @@ -64,6 +65,16 @@ struct Model { return m_netcons; } + /** @brief Access SoA storage for PreSyn integration fields (Phase 3). + * @see doc/network-soa-phase0.md §5.5 + */ + container::network::PreSyn::storage& presyns() { + return m_presyns; + } + container::network::PreSyn::storage const& presyns() const { + return m_presyns; + } + /** @brief Apply a function to each non-null Mechanism. */ template @@ -156,6 +167,7 @@ struct Model { m_point_processes.shrink_to_fit(); m_weights.shrink_to_fit(); m_netcons.shrink_to_fit(); + m_presyns.shrink_to_fit(); } private: @@ -196,6 +208,9 @@ struct Model { /** @brief Network SoA: NetCon integration rows (Phase 2). */ container::network::NetCon::storage m_netcons{}; + /** @brief Network SoA: PreSyn integration rows (Phase 3). */ + container::network::PreSyn::storage m_presyns{}; + /** * @brief Backing storage for defer_delete helper. */ diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index b8d58f3b6e..f50e455ab6 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -4,6 +4,7 @@ #include "neuron/container/data_handle.hpp" #include "neuron/container/network/netcon.hpp" +#include "neuron/container/network/presyn.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/model_data.hpp" #include "nrnmpi.h" @@ -305,6 +306,13 @@ class PreSyn: public ConditionEvent { double mindelay(); void fanout(double, NetCvode*, NrnThread*); // used by bbsavestate + /** @brief Sync legacy PreSyn fields into PreSyn SoA (Phase 3). */ + void soa_sync(); + /** @brief Mark global NetCon fanout order dirty (dil_ changed). */ + static void mark_fanout_unsorted(); + /** @brief Rebuild global fanout order from all PreSyn dil_ lists. */ + static void ensure_fanout_order(); + NetConPList dil_; double threshold_; double delay_; @@ -321,6 +329,8 @@ class PreSyn: public ConditionEvent { int rec_id_; int output_index_; int gid_; + /** @brief Phase 3 dual-write: PreSyn integration row. */ + neuron::container::network::PreSyn::owning_handle _soa{neuron::model().presyns()}; #if NRNMPI unsigned char localgid_; // compressed gid for spike transfer #endif diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index b41c9adb31..d016c54b83 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -145,6 +145,7 @@ double nrn_netcon_get_thresh(NetCon* nc) { void nrn_netcon_set_thresh(NetCon* nc, double th) { if (nc->src_) { nc->src_->threshold_ = th; + nc->src_->soa_sync(); } } @@ -3038,6 +3039,72 @@ void NetCon::pr(const char* s, double tt, NetCvode* ns) { Printf(" target=%s %.15g\n", (target_ ? hoc_object_name(target_->ob) : "nullptr"), tt); } +namespace { +// Global CoreNEURON-shaped fanout: NetCon* ranges described by PreSyn NcIndex/NcCount. +std::vector g_network_fanout_order; +bool g_network_fanout_sorted = true; + +template +void for_each_fanout_netcon(PreSyn* ps, F&& fn) { + PreSyn::ensure_fanout_order(); + int const base = ps->_soa.nc_index(); + int const cnt = ps->_soa.nc_count(); + if (base >= 0 && cnt == static_cast(ps->dil_.size()) && + base + cnt <= static_cast(g_network_fanout_order.size())) { + for (int i = 0; i < cnt; ++i) { + fn(g_network_fanout_order[static_cast(base + i)]); + } + } else { + // Fallback while topology is mid-update or PreSyn not in psl_. + for (NetCon* d: ps->dil_) { + fn(d); + } + } +} +} // namespace + +void PreSyn::mark_fanout_unsorted() { + g_network_fanout_sorted = false; +} + +void PreSyn::ensure_fanout_order() { + if (g_network_fanout_sorted) { + return; + } + g_network_fanout_order.clear(); + if (net_cvode_instance && net_cvode_instance->psl_) { + for (PreSyn* ps: *net_cvode_instance->psl_) { + ps->_soa.nc_index() = static_cast(g_network_fanout_order.size()); + for (NetCon* nc: ps->dil_) { + g_network_fanout_order.push_back(nc); + } + ps->_soa.nc_count() = static_cast(ps->dil_.size()); + } + } + g_network_fanout_sorted = true; +} + +void PreSyn::soa_sync() { + _soa.threshold() = threshold_; + _soa.gid() = gid_; + _soa.output_index() = output_index_; + _soa.thread_id() = nt_ ? nt_->id : -1; + _soa.nc_count() = static_cast(dil_.size()); + // NcIndex is owned by ensure_fanout_order when sorted. + if (!g_network_fanout_sorted) { + _soa.nc_index() = -1; + } + if (thvar_ && thvar_.refers_to_a_modern_data_structure()) { + try { + _soa.thvar_row() = static_cast(thvar_.current_row()); + } catch (...) { + _soa.thvar_row() = -1; + } + } else { + _soa.thvar_row() = -1; + } +} + void PreSyn::send(double tt, NetCvode* ns, NrnThread* nt) { int i; record(tt); @@ -3053,7 +3120,8 @@ void PreSyn::send(double tt, NetCvode* ns, NrnThread* nt) { } } else { STATISTICS(presyn_send_direct_); - for (const auto& d: dil_) { + // Phase 3: fanout via NcIndex/NcCount into global order (dual-write with dil_). + for_each_fanout_netcon(this, [&](NetCon* d) { if (d->active_ && d->target_) { NrnThread* n = PP2NT(d->target_); if (nt == n) { @@ -3062,7 +3130,7 @@ void PreSyn::send(double tt, NetCvode* ns, NrnThread* nt) { ns->p[n->id].interthread_send(tt + d->delay_, d, n); } } - } + }); } #endif // ndef USENCS #if USENCS || NRNMPI @@ -3116,7 +3184,7 @@ void PreSyn::deliver(double tt, NetCvode* ns, NrnThread* nt) { } // the thread is the one that owns the targets STATISTICS(presyn_deliver_netcon_); - for (const auto& d: dil_) { + for_each_fanout_netcon(this, [&](NetCon* d) { if (d->active_ && d->target_ && PP2NT(d->target_) == nt) { double dtt = d->delay_ - delay_; if (dtt == 0.) { @@ -3130,19 +3198,19 @@ void PreSyn::deliver(double tt, NetCvode* ns, NrnThread* nt) { ns->event(tt + dtt, d, nt); } } - } + }); } // used by bbsavestate since during restore, some NetCon spikes may // have already been delivered while others need to be delivered in // the future. Not implemented fof qthresh_ case. No statistics. void PreSyn::fanout(double td, NetCvode* ns, NrnThread* nt) { - for (const auto& d: dil_) { + for_each_fanout_netcon(this, [&](NetCon* d) { if (d->active_ && d->target_ && PP2NT(d->target_) == nt) { double dtt = d->delay_ - delay_; ns->bin_event(td + dtt, d, nt); } - } + }); } NrnThread* PreSyn::thread() { @@ -3160,7 +3228,7 @@ void PreSyn::pgvts_deliver(double tt, NetCvode* ns) { return; } STATISTICS(presyn_deliver_netcon_); - for (const auto& d: dil_) { + for_each_fanout_netcon(this, [&](NetCon* d) { if (d->active_ && d->target_) { double dtt = d->delay_ - delay_; if (dtt < 0.) { @@ -3170,7 +3238,7 @@ void PreSyn::pgvts_deliver(double tt, NetCvode* ns) { ns->event(tt + dtt, d, nt); } } - } + }); } void PreSyn::pr(const char* s, double tt, NetCvode* ns) { @@ -4751,8 +4819,12 @@ void NetCon::soa_sync() { } else { _soa.target() = -1; } - // PreSyn SoA is Phase 3; reverse edge stays -1 until then. - _soa.src_presyn() = -1; + // Phase 3: reverse edge into PreSyn SoA. + if (src_) { + _soa.src_presyn() = static_cast(src_->_soa.current_row()); + } else { + _soa.src_presyn() = -1; + } if (!weight_soa_.empty()) { _soa.weight_index() = static_cast(weight_soa_.front().current_row()); weights_heap_to_soa(); @@ -4769,6 +4841,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { if (src_) { src_->dil_.push_back(this); src_->use_min_delay_ = 0; + PreSyn::mark_fanout_unsorted(); } if (target == nullptr) { target_ = nullptr; @@ -4821,6 +4894,7 @@ void NetCon::rmsrc() { for (size_t i = 0; i < src_->dil_.size(); ++i) { if (src_->dil_[i] == this) { src_->dil_.erase(src_->dil_.begin() + i); + PreSyn::mark_fanout_unsorted(); if (src_->dil_.size() == 0 && src_->tvec_ == NULL && src_->idvec_ == NULL) { if (src_->output_index_ == -1) { delete std::exchange(src_, nullptr); @@ -4840,6 +4914,7 @@ void NetCon::replace_src(PreSyn* p) { if (src_) { src_->dil_.push_back(this); src_->use_min_delay_ = 0; + PreSyn::mark_fanout_unsorted(); } soa_sync(); } @@ -4955,6 +5030,7 @@ void NetCvode::ps_thread_link(PreSyn* ps) { } } if (!ps->nt_) { // premature, reorder_secorder() not called yet + ps->soa_sync(); return; } if (ps->thvar_) { @@ -4964,6 +5040,7 @@ void NetCvode::ps_thread_link(PreSyn* ps) { } ps->hi_th_ = hoc_l_insertvoid(p[i].psl_thr_, ps); } + ps->soa_sync(); } void NetCvode::update_ps2nt() { @@ -5040,10 +5117,13 @@ PreSyn::PreSyn(neuron::container::data_handle src, Object* osrc, Section nrn_notify_when_void_freed(osrc_, this); } #endif + soa_sync(); + mark_fanout_unsorted(); } PreSyn::~PreSyn() { PreSynSave::invalid(); + mark_fanout_unsorted(); // printf("~PreSyn %p\n", this); nrn_cleanup_presyn(this); delete std::exchange(stmt_, nullptr); diff --git a/src/nrniv/memory_usage.cpp b/src/nrniv/memory_usage.cpp index 2289aa2ecd..cfef3abc88 100644 --- a/src/nrniv/memory_usage.cpp +++ b/src/nrniv/memory_usage.cpp @@ -16,8 +16,9 @@ ModelMemoryUsage memory_usage(const Model& model) { auto point_processes = memory_usage(model.point_processes()); auto weights = memory_usage(model.weights()); auto netcons = memory_usage(model.netcons()); + auto presyns = memory_usage(model.presyns()); - return {nodes, mechanisms, point_processes, weights, netcons}; + return {nodes, mechanisms, point_processes, weights, netcons, presyns}; } cache::ModelMemoryUsage memory_usage(const std::optional& model) { @@ -134,6 +135,10 @@ std::string format_memory_usage(const MemoryUsage& usage) { os << " data " << format_memory_usage(model.netcons.heavy_data) << "\n"; os << " stable_identifiers " << format_memory_usage(model.netcons.stable_identifiers) << "\n"; + os << " network::PreSyn \n"; + os << " data " << format_memory_usage(model.presyns.heavy_data) << "\n"; + os << " stable_identifiers " << format_memory_usage(model.presyns.stable_identifiers) + << "\n"; os << "cache::Model \n"; os << " threads " << format_memory_usage(cache_model.threads) << "\n"; os << " mechanisms " << format_memory_usage(cache_model.mechanisms) << "\n"; diff --git a/src/nrniv/netpar.cpp b/src/nrniv/netpar.cpp index b274d641b6..54048abb51 100644 --- a/src/nrniv/netpar.cpp +++ b/src/nrniv/netpar.cpp @@ -1099,6 +1099,7 @@ void BBS::cell() { } else { ps->output_index_ = gid; } + ps->soa_sync(); } void BBS::outputcell(int gid) { @@ -1108,6 +1109,7 @@ void BBS::outputcell(int gid) { assert(ps); ps->output_index_ = gid; ps->gid_ = gid; + ps->soa_sync(); } void BBS::spike_record(int gid, IvocVect* spikevec, IvocVect* gidvec) { diff --git a/src/nrnoc/container.cpp b/src/nrnoc/container.cpp index 1b1c61a58c..ab2d26e301 100644 --- a/src/nrnoc/container.cpp +++ b/src/nrnoc/container.cpp @@ -18,6 +18,7 @@ Model::Model() { m_point_processes.set_unsorted_callback(invalidate_cache); m_weights.set_unsorted_callback(invalidate_cache); m_netcons.set_unsorted_callback(invalidate_cache); + m_presyns.set_unsorted_callback(invalidate_cache); // needs some re-organisation if we ever want to support multiple Model instances assert(!container::detail::defer_delete_storage); container::detail::defer_delete_storage = &m_ptrs_for_deferred_deletion; @@ -50,6 +51,9 @@ std::unique_ptr Model::find_container_info(void if (auto maybe_info = m_netcons.find_container_info(cont); maybe_info) { return maybe_info; } + if (auto maybe_info = m_presyns.find_container_info(cont); maybe_info) { + return maybe_info; + } return {}; } @@ -165,6 +169,9 @@ generic_data_handle promote_or_clear(generic_data_handle gdh) { if (auto h = model.netcons().find_data_handle(gdh); h.refers_to_a_modern_data_structure()) { return h; } + if (auto h = model.presyns().find_data_handle(gdh); h.refers_to_a_modern_data_structure()) { + return h; + } return {}; } } // namespace detail diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index 3a3d499710..d8381c0e77 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -1,5 +1,6 @@ #include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" +#include "neuron/container/network/presyn.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/model_data.hpp" @@ -252,3 +253,48 @@ TEST_CASE("SOA-backed NetCon structure", "[Neuron][data_structures][network][net } } } + +TEST_CASE("SOA-backed PreSyn structure and fanout ranges", + "[Neuron][data_structures][network][presyn]") { + auto& storage = neuron::model().presyns(); + REQUIRE(storage.size() == 0); + + GIVEN("Default PreSyn SoA rows") { + PreSyn::owning_handle a{storage}; + PreSyn::owning_handle b{storage}; + THEN("Defaults match field tags") { + REQUIRE(a.threshold() == 10.0); + REQUIRE(a.gid() == -1); + REQUIRE(a.nc_index() == -1); + REQUIRE(a.nc_count() == 0); + REQUIRE(a.output_index() == -1); + REQUIRE(a.thvar_row() == -1); + REQUIRE(a.thread_id() == -1); + REQUIRE(storage.size() == 2); + } + THEN("Fanout range fields survive reverse permutation") { + // Simulate CoreNEURON-style contiguous fanout ranges in a global order. + a.nc_index() = 0; + a.nc_count() = 3; + a.threshold() = -20.; + a.gid() = 7; + b.nc_index() = 3; + b.nc_count() = 2; + b.gid() = 11; + { + auto token = storage.issue_frozen_token(); + storage.mark_as_sorted(token); + } + std::vector perm{0, 1}; + std::rotate(perm.begin(), std::next(perm.begin()), perm.end()); + storage.apply_reverse_permutation(std::move(perm)); + REQUIRE(a.nc_index() == 0); + REQUIRE(a.nc_count() == 3); + REQUIRE(a.threshold() == -20.); + REQUIRE(a.gid() == 7); + REQUIRE(b.nc_index() == 3); + REQUIRE(b.nc_count() == 2); + REQUIRE(b.gid() == 11); + } + } +} From 35ddea4f85114d51a5e0c856a2c51d1c0d281efe Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 13:29:11 -0400 Subject: [PATCH 07/33] Phase 4: SelfEvent weight_index and pnt_receive-by-index dual-write MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SelfEvent records Weight SoA base index and PointProcess row; net_send resolves index from NetCon. Delivery uses nrn_pnt_receive_by_weight_index (M2: SoA→heap→MOD pnt_receive→heap→SoA). NetCon::deliver shares that path. --- GROK-NETWORK-SOA.md | 2 +- doc/network-soa-phase0.md | 2 +- src/neuron/container/network/self_event.hpp | 81 +++++++++++++++ src/nrncvode/netcon.h | 5 + src/nrncvode/netcvode.cpp | 106 ++++++++++++++++++-- src/nrnoc/nrniv_mf.h | 12 +++ test/unit_tests/container/network.cpp | 20 ++++ 7 files changed, 217 insertions(+), 11 deletions(-) create mode 100644 src/neuron/container/network/self_event.hpp diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 3c29c6c1dc..7425a3cb35 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -72,7 +72,7 @@ psolve hot path (deliver, threshold, fanout) — index-based | **1** | `Point_process` + `weights` SoA + dual-write create/destroy | handles survive permute; PP dual-write | | **2** | `NetCon` SoA, HOC `weight()` → flat Weight SoA | dual-write; unit tests; CPU ringtest when available | | **3** | `PreSyn` SoA + `NcIndex`/`NcCount` fanout order | dual-write; send/deliver use fanout | -| **4** | `SelfEvent` indices, `pnt_receive(weight_index)` | ExpSyn @ 1.025 ms CPU | +| **4** | SelfEvent weight_index + `nrn_pnt_receive_by_weight_index` | dual-write M2; ExpSyn delivery | | **5** | (gpu-native track) net buffers Stages 2–3 | GPU parity | --- diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index d69cb98f2e..ae89ac95db 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -304,7 +304,7 @@ If a structural pool is useful later, tags match the above. Gate is `pnt_receive | **1** | `PointProcess` + `Weight` | Handles survive permute; dual-write on create/destroy | | **2** | `NetCon` SoA; HOC `weight()` → flat Weight SoA | dual-write; handles survive permute | | **3** | `PreSyn` SoA + fanout `NcIndex`/`NcCount` | dual-write; hot path uses fanout order | -| **4** | SelfEvent indices; `pnt_receive` by weight index | ExpSyn @ 1.025 ms CPU | +| **4** | SelfEvent weight_index; `nrn_pnt_receive_by_weight_index` | dual-write M2 (double* still for MOD) | | **5** | (gpu-native track) net buffers | GPU parity — **not this branch** | --- diff --git a/src/neuron/container/network/self_event.hpp b/src/neuron/container/network/self_event.hpp new file mode 100644 index 0000000000..6bfd3e9eab --- /dev/null +++ b/src/neuron/container/network/self_event.hpp @@ -0,0 +1,81 @@ +#pragma once +/** + * @file network/self_event.hpp + * @brief Phase 4 SelfEvent index fields (not a long-lived SoA container). + * + * Design: doc/network-soa-phase0.md §5.6, §9. + * + * SelfEvents are short-lived queue/pool objects. Integration-relevant payload: + * - target PointProcess row (optional dual-write) + * - weight_index into Weight SoA (base of the NetCon weight block) + * - flag (NET_RECEIVE flag) + * + * Generated MOD code still receives double* for Phase 4 (M2 dual-write): + * delivery materializes SoA → heap around pnt_receive. + */ +#include "neuron/container/network/weights.hpp" +#include "neuron/model_data.hpp" + +#include + +namespace neuron::container::network::SelfEventFields { + +/** Field tags for documentation / future structural pool (not used as soa<> tags yet). */ +namespace field { +struct TargetPnt { + using type = int; + static constexpr type default_value() { + return -1; + } +}; +struct WeightIndex { + using type = int; + static constexpr type default_value() { + return -1; + } +}; +struct Flag { + using type = double; + static constexpr type default_value() { + return 0.; + } +}; +} // namespace field + +/** + * @brief Copy @p count consecutive Weight SoA rows starting at @p weight_index into @p out. + * + * Contiguity is the dual-write invariant from weight block allocation (Phase 1–2). + * After a full weight repack/sort this remains valid for a NetCon's block. + */ +inline void materialize_weight_block(int weight_index, int count, double* out) { + if (!out || count <= 0 || weight_index < 0) { + return; + } + auto& store = neuron::model().weights(); + auto const n = static_cast(store.size()); + for (int i = 0; i < count; ++i) { + int const row = weight_index + i; + out[i] = (row >= 0 && row < n) ? store.get(static_cast(row)) + : 0.; + } +} + +/** + * @brief Write @p count heap values back into Weight SoA at @p weight_index. + */ +inline void store_weight_block(int weight_index, int count, double const* in) { + if (!in || count <= 0 || weight_index < 0) { + return; + } + auto& store = neuron::model().weights(); + auto const n = static_cast(store.size()); + for (int i = 0; i < count; ++i) { + int const row = weight_index + i; + if (row >= 0 && row < n) { + store.get(static_cast(row)) = in[i]; + } + } +} + +} // namespace neuron::container::network::SelfEventFields diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index f50e455ab6..de0a51f027 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -5,6 +5,7 @@ #include "neuron/container/data_handle.hpp" #include "neuron/container/network/netcon.hpp" #include "neuron/container/network/presyn.hpp" +#include "neuron/container/network/self_event.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/model_data.hpp" #include "nrnmpi.h" @@ -184,6 +185,10 @@ class SelfEvent: public DiscreteEvent { Point_process* target_; double* weight_; Datum* movable_; // pointed-to Datum holds TQItem* + /** @brief Phase 4: base row in Weight SoA (-1 if unknown / null weights). */ + int weight_index_{-1}; + /** @brief Phase 4: PointProcess SoA row of target_ (-1 if unknown). */ + int target_row_{-1}; static unsigned long selfevent_send_; static unsigned long selfevent_move_; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index d016c54b83..8c39872a20 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2272,6 +2272,62 @@ void NetCvode::remove_event(TQItem* q, int tid) { p[tid].tqe_->remove(q); } +namespace { +/** Phase 4: fill SelfEvent dual-write index fields from weight* / Point_process. */ +void selfevent_set_indices(SelfEvent* se, Point_process* pnt, double* weight) { + se->target_row_ = pnt ? static_cast(pnt->_soa.current_row()) : -1; + se->weight_index_ = -1; + if (!weight) { + return; + } + // weight* is almost always a NetCon heap base; resolve Weight SoA index. + if (NetCon* nc = NetConSave::weight2netcon(weight)) { + if (!nc->weight_soa_.empty()) { + se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); + } else { + se->weight_index_ = nc->_soa.weight_index(); + } + } +} +} // namespace + +void nrn_pnt_receive_by_weight_index(Point_process* pnt, + int weight_index, + double flag, + double* weight_heap) { + if (!pnt || !pnt->prop) { + return; + } + int const type = pnt->prop->_type; + if (!pnt_receive[type]) { + return; + } + int const n = pnt_receive_size[type]; + // Prefer NetCon dual-write owners when heap base is a known NetCon weight_. + if (weight_heap) { + if (NetCon* nc = NetConSave::weight2netcon(weight_heap)) { + nc->weights_soa_to_heap(); + POINT_RECEIVE(type, pnt, weight_heap, flag); + nc->weights_heap_to_soa(); + return; + } + } + // Index-only path: materialize a temporary (or fill provided heap) from Weight SoA. + double* buf = weight_heap; + std::vector tmp; + if (!buf && n > 0) { + tmp.resize(static_cast(n), 0.); + buf = tmp.data(); + } + if (weight_index >= 0 && n > 0 && buf) { + neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); + } + POINT_RECEIVE(type, pnt, buf, flag); + if (weight_index >= 0 && n > 0 && buf) { + neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); + } +} + // for threads, revised net_send to use absolute time (in the // mod file we add the thread time when we call it). void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, double flag) { @@ -2282,6 +2338,7 @@ void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, doubl se->flag_ = flag; se->target_ = pnt; se->weight_ = weight; + selfevent_set_indices(se, pnt, weight); se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -2309,6 +2366,7 @@ void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, d se->flag_ = flag; se->target_ = pnt; se->weight_ = weight; + selfevent_set_indices(se, pnt, weight); se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -2999,11 +3057,18 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); - // Weight SoA is HOC source of truth; heap is the pnt_receive buffer. - weights_soa_to_heap(); - POINT_RECEIVE(type, target_, weight_, 0); - // Capture any weight mutations by NET_RECEIVE / MOD code. - weights_heap_to_soa(); + // Phase 4: deliver via weight_index (SoA), materializing double* for MOD. + int widx = _soa.weight_index(); + if (widx < 0 && !weight_soa_.empty()) { + widx = static_cast(weight_soa_.front().current_row()); + } + if (widx >= 0) { + nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); + } else { + weights_soa_to_heap(); + POINT_RECEIVE(type, target_, weight_, 0); + weights_heap_to_soa(); + } if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3019,9 +3084,17 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { assert(target_); int type = target_->prop->_type; STATISTICS(netcon_deliver_); - weights_soa_to_heap(); - POINT_RECEIVE(type, target_, weight_, 0); - weights_heap_to_soa(); + int widx = _soa.weight_index(); + if (widx < 0 && !weight_soa_.empty()) { + widx = static_cast(weight_soa_.front().current_row()); + } + if (widx >= 0) { + nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); + } else { + weights_soa_to_heap(); + POINT_RECEIVE(type, target_, weight_, 0); + weights_heap_to_soa(); + } if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3256,6 +3329,8 @@ DiscreteEvent* SelfEvent::savestate_save() { se->flag_ = flag_; se->target_ = target_; se->weight_ = weight_; + se->weight_index_ = weight_index_; + se->target_row_ = target_row_; se->movable_ = movable_; return se; } @@ -3277,9 +3352,16 @@ DiscreteEvent* SelfEvent::savestate_read(FILE* f) { 6); se->target_ = SelfEvent::index2pp(pptype, ppindex); se->weight_ = nullptr; + se->weight_index_ = -1; + se->target_row_ = se->target_ ? static_cast(se->target_->_soa.current_row()) : -1; if (ncindex >= 0) { NetCon* nc = NetConSave::index2netcon(ncindex); se->weight_ = nc->weight_; + if (!nc->weight_soa_.empty()) { + se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); + } else { + se->weight_index_ = nc->_soa.weight_index(); + } } se->flag_ = flag; se->movable_ = (moff >= 0) ? (se->target_->prop->dparam + moff) : nullptr; @@ -3373,7 +3455,13 @@ void SelfEvent::pgvts_deliver(double tt, NetCvode* ns) { } void SelfEvent::call_net_receive(NetCvode* ns) { STATISTICS(selfevent_deliver_); - POINT_RECEIVE(target_->prop->_type, target_, weight_, flag_); + // Phase 4 M2: prefer weight_index path; still pass double* to generated MOD. + if (weight_index_ >= 0) { + nrn_pnt_receive_by_weight_index(target_, weight_index_, flag_, weight_); + } else { + // Null weights (flag-only self events) or unresolved index: legacy path. + POINT_RECEIVE(target_->prop->_type, target_, weight_, flag_); + } if (errno) { if (nrn_errno_check(target_->prop->_type)) { hoc_warning("errno set during SelfEvent deliver to NET_RECEIVE", (char*) 0); diff --git a/src/nrnoc/nrniv_mf.h b/src/nrnoc/nrniv_mf.h index 3d28a0cbec..b22b18b1b6 100644 --- a/src/nrnoc/nrniv_mf.h +++ b/src/nrnoc/nrniv_mf.h @@ -22,6 +22,18 @@ using ldifusfunc_t = void (*)(ldifusfunc2_t, neuron::model_sorted_token const&, typedef void (*pnt_receive_t)(Point_process*, double*, double); typedef void (*pnt_receive_init_t)(Point_process*, double*, double); +/** + * @brief Phase 4 dual-write: call pnt_receive after materializing Weight SoA → heap buffer. + * + * Generated MOD still uses (Point_process*, double*, double). weight_index is the + * base row in neuron::model().weights(); count is pnt_receive_size[type]. + * If weight_heap is non-null it is used as the buffer (and written back to SoA after). + */ +void nrn_pnt_receive_by_weight_index(Point_process* pnt, + int weight_index, + double flag, + double* weight_heap = nullptr); + extern Prop* need_memb_cl(Symbol*, int*, int*); extern Prop* prop_alloc(Prop**, int, Node*); void prop_update_ion_variables(Prop*, Node*); diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index d8381c0e77..3a8a613c90 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -1,6 +1,7 @@ #include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" #include "neuron/container/network/presyn.hpp" +#include "neuron/container/network/self_event.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/model_data.hpp" @@ -254,6 +255,25 @@ TEST_CASE("SOA-backed NetCon structure", "[Neuron][data_structures][network][net } } +TEST_CASE("Weight index materialize/store (Phase 4 SelfEvent path)", + "[Neuron][data_structures][network][selfevent]") { + auto& store = neuron::model().weights(); + REQUIRE(store.size() == 0); + auto rows = Weight::allocate_weight_rows(3, nullptr); + rows[0].value() = 1.0; + rows[1].value() = 2.0; + rows[2].value() = 3.0; + int const base = static_cast(rows[0].current_row()); + double buf[3]{}; + neuron::container::network::SelfEventFields::materialize_weight_block(base, 3, buf); + REQUIRE(buf[0] == 1.0); + REQUIRE(buf[1] == 2.0); + REQUIRE(buf[2] == 3.0); + buf[1] = 9.0; + neuron::container::network::SelfEventFields::store_weight_block(base, 3, buf); + REQUIRE(rows[1].value() == 9.0); +} + TEST_CASE("SOA-backed PreSyn structure and fanout ranges", "[Neuron][data_structures][network][presyn]") { auto& storage = neuron::model().presyns(); From db6f4debc00260a262dc4e16545278f4e1049396 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 13:56:30 -0400 Subject: [PATCH 08/33] Harden network SoA dual-write: delivery gate, fanout at init, fix weight wipe MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Document Phases 0–4 status and known gaps in GROK-NETWORK-SOA.md. Rebuild PreSyn fanout at NetCvode::init_events (not only on first spike). Stop soa_sync from copying heap weights over SoA (HOC weight writes). Add pytest/ctest network_soa_delivery gate for NetStim→ExpSyn. --- GROK-NETWORK-SOA.md | 154 ++++++++++++----------- doc/network-soa-phase0.md | 2 +- src/nrncvode/netcvode.cpp | 22 +++- test/CMakeLists.txt | 8 ++ test/pytest/test_network_soa_delivery.py | 70 +++++++++++ 5 files changed, 182 insertions(+), 74 deletions(-) create mode 100644 test/pytest/test_network_soa_delivery.py diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 7425a3cb35..09953c941d 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -4,30 +4,41 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s --- -## Why a separate worktree +## Status (2026-07) -Network SoA is a focused CPU/infrastructure PR. It should **not** carry the GPU-native qualification commit stack (`local/gpu-native-qualification`). +| Phase | Status | Notes | +|-------|--------|--------| +| **0** Spec | **Done** | `doc/network-soa-phase0.md` authoritative | +| **1** PointProcess + Weight SoA dual-write | **Done** | Handles + create/destroy | +| **2** NetCon SoA; HOC weight → Weight SoA | **Done** | Delivery materializes `double*` for MOD | +| **3** PreSyn SoA + fanout `NcIndex`/`NcCount` | **Done** | Rebuild at `init_events` + lazy on spike | +| **4** SelfEvent `weight_index` + receive-by-index | **Done** | M2: still `pnt_receive(..., double*, flag)` | +| **5** GPU net buffers | **Out of scope** | `local/gpu-native-qualification` after merge | -| Worktree | Branch | Purpose | -|----------|--------|---------| -| `~/neuron/cpu_net_soa` | `local/cpu-network-soa` | Network SoA → PR to **master** | -| `~/neuron/nrngpu` | `local/gpu-native-qualification` | Mechanism GPU, Stage 1 buffer plumbing; Stages 2–3 **paused** until SoA lands | +**Dual-write complete for the CoreNEURON-shaped data plane.** Legacy pointers/`dil_`/`weight_` heap remain for interpreter, queue, and nocmodl ABI. -Created with: +### Known gaps (do not “just drop heap” yet) -```bash -cd ~/neuron/nrngpu -git worktree add ~/neuron/cpu_net_soa -b local/cpu-network-soa origin/master -``` +- Long-lived `NetCon::weight_` heap still required for generated `pnt_receive` ABI (default **nocmodl**, `NRN_ENABLE_NMODL=OFF`). +- Network containers not yet in `nrn_ensure_model_data_are_sorted()` (no full network permute/repack pass). +- `InputPreSyn` / thin remote gid→fanout deferred (see phase0 §5.5.1). +- SaveState still largely keyed by weight pointers; SoA repack would need map invalidation. +- Full `ctest -j N` needs a complete build/install; judge network work with filters below first. + +### Default codegen + +**nocmodl** (not NMODL). SoA weights work via materialize-around-`pnt_receive`. Native `weight_index` in MOD requires a separate nocmodl/ABI project. --- -## Decision record (2026-07-08) +## Why a separate worktree -- **Pause** ringtest GPU network buffers (Stage 2 codegen, Stage 3 wire-up). -- **Discard** unstaged Stage 2 WIP on `gpu-native-qualification` (reverted). -- **Adopt** CoreNEURON-shaped network SoA on NEURON CPU first, with HOC types as wrappers over `data_handle` / `soa` backing store (same model as nodes/mechanisms). -- **Resume** GPU network buffers after SoA merges to master; gpu-native branch rebases then. +Network SoA is a focused CPU/infrastructure PR. It should **not** carry the GPU-native qualification commit stack (`local/gpu-native-qualification`). + +| Worktree | Branch | Purpose | +|----------|--------|---------| +| `~/neuron/cpu_net_soa` | `local/cpu-network-soa` | Network SoA → PR to **master** | +| `~/neuron/nrngpu` | `local/gpu-native-qualification` | Mechanism GPU, Stage 1 buffer plumbing; Stages 2–3 **paused** until SoA lands | --- @@ -37,102 +48,101 @@ One CPU backing store for integration-hot network data: - `std::vector` columns + permutation indices - `data_handle` / `owning_handle` stable across permute -- `NrnThread` = lightweight slice (`offset` + count) into contiguous regions — **no duplicate CPU copies** -- HOC extras (`Object*`, recording, …) in sidecars keyed by handle +- `NrnThread` = lightweight slice (`offset` + count) — **no duplicate CPU copies** +- HOC extras in sidecars keyed by handle -CoreNEURON layout is the **integration reference**; NEURON adds live create/delete/permute + interpreter compatibility. +CoreNEURON layout is the **integration reference**. --- -## Architecture sketch +## Architecture (current dual-write) ```text -HOC / Python (NetCon, PointProcess, PreSyn API) - │ - ▼ -Wrappers (handle / owning_handle into network SoA) - │ - ▼ -neuron::container::soa<...> (weights, pntproc, netcon, presyn, …) - │ - ▼ -psolve hot path (deliver, threshold, fanout) — index-based - │ - ▼ -(future) GPU upload of same columns after merge with gpu-native track +HOC / Python + │ + ▼ +Legacy shells (Point_process*, NetCon, PreSyn) + owning_handle _soa + │ + ▼ +neuron::container::network::{PointProcess,Weight,NetCon,PreSyn} + │ + ▼ +Hot path: fanout order + weight_index → materialize → pnt_receive(double*) ``` ---- - -## Phase plan (summary) +Key paths: -| Phase | Content | Gate | -|-------|---------|------| -| **0** | Field tags, handle typedefs, thread slicing — `doc/network-soa-phase0.md` | Spec done (authoritative) | -| **1** | `Point_process` + `weights` SoA + dual-write create/destroy | handles survive permute; PP dual-write | -| **2** | `NetCon` SoA, HOC `weight()` → flat Weight SoA | dual-write; unit tests; CPU ringtest when available | -| **3** | `PreSyn` SoA + `NcIndex`/`NcCount` fanout order | dual-write; send/deliver use fanout | -| **4** | SelfEvent weight_index + `nrn_pnt_receive_by_weight_index` | dual-write M2; ExpSyn delivery | -| **5** | (gpu-native track) net buffers Stages 2–3 | GPU parity | +| Path | File / API | +|------|------------| +| Containers | `src/neuron/container/network/*.hpp` | +| Model ownership | `src/neuron/model_data.hpp` | +| PP dual-write | `Point_process::_soa`, `nrn_point_process_soa_sync` | +| NetCon / weights | `NetCon::_soa`, `weight_soa_`, `soa_sync` | +| Fanout | `PreSyn::ensure_fanout_order`, global NetCon* order | +| Receive by index | `nrn_pnt_receive_by_weight_index` | --- -## Parallel work (not blocking this branch) +## Recommended next work (after dual-write) -On `local/gpu-native-qualification`: - -- Mechanism GPU gates A–E (ringtest mods). -- `-no-netcon` ringtest + APCount for NEURON vs CoreNEURON perf without `NET_RECEIVE`. +1. ~~Handoff status + automated delivery gate~~ (this session if present). +2. Wire network storages into `nrn_ensure_model_data_are_sorted` (thread slices, weight repack). +3. Policy: short-lived materialize vs nocmodl `weight_index` ABI; then drop long-lived heap `weight_`. +4. SaveState / `weight2netcon` → id or weight_index keys. +5. Merge toward master; GPU track rebases for Phase 5-style buffers. --- -## Build (first session) +## Build ```bash cd ~/neuron/cpu_net_soa mkdir -p build && cd build -cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install -DNRN_ENABLE_MPI=ON -ninja && ninja install -source ~/neuron/bin/nrnenv nrngpu build-cpu-net-soa # add nrnenv alias if helpful +cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install \ + -DNRN_ENABLE_MPI=ON -DNRN_ENABLE_TESTS=ON \ + -DNRN_ENABLE_CORENEURON=OFF -DNRN_ENABLE_NMODL=OFF +cmake --build . --parallel +# Prefer install for integration tests that need share/hoc +cmake --build . --target install ``` -Adjust `nrnenv` path to `~/neuron/cpu_net_soa/build` — agent may create `build-cpu-net-soa` on first configure. - --- ## Tests (CPU gates) ```bash -# After Phase 3+ -cd build && ctest -R ringtest --output-on-failure +cd build + +# Unit: SoA containers + dual-write helpers +./bin/test/testneuron '[network]' --reporter compact +./bin/test/testneuron '[data_structures]' --reporter compact -# Spike parity (single host) -# mpiexec -n 1 ./path/to/special -nobanner ringtest.hoc +# Integration delivery gate (also under ctest pytest group) +# Requires PYTHONPATH / build env for neuron +python -m pytest ../test/pytest/test_network_soa_delivery.py -v + +# Broader (env-sensitive) +ctest -j 4 -R 'unit_tests::testneuron|network_soa|pytest' --output-on-failure +# Full ctest: complete install first; CoreNEURON jobs skipped when disabled ``` --- -## Starting prompt (paste into new `cpu_net_soa` session) +## Starting prompt (new session) ``` Read ~/neuron/cpu_net_soa/GROK-NETWORK-SOA.md, AGENTS.md, and doc/network-soa-phase0.md. -Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa (from origin/master). -Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU track; network buffers paused). - -Task: Write the Phase 0 design spec for network SoA — field tags, handle types, -per-thread slicing, HOC sidecar policy, invalidation rules. Then scaffold Phase 1 -(Point_process + weights container) under src/neuron/container/network/. - -Follow the node/mechanism DataHandle pattern (soa_container.hpp, data_handle.hpp). -PreSyn thvar_ data_handle is the existing network prototype. CoreNEURON multicore.hpp -is the integration layout reference. +Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa. +Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU network buffers paused). -Do not implement GPU net_buf_receive on this branch. +Phases 0–4 dual-write are done. Next: wire network SoA into +nrn_ensure_model_data_are_sorted (thread slices, weight repack), or +SaveState index keys — not GPU net_buf_receive on this branch. ``` --- ## Old GPU-native context -Stage 1 `NetReceiveBuffer` on gpu-native (`1f05cbc19`) remains valid **pattern**; indexing will align with SoA after merge. See `~/neuron/nrngpu/GROK-GPU-NATIVE.md`. \ No newline at end of file +Stage 1 `NetReceiveBuffer` on gpu-native remains a valid **pattern**; indexing aligns with SoA after merge. See `~/neuron/nrngpu/GROK-GPU-NATIVE.md`. diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index ae89ac95db..7778679343 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -1,6 +1,6 @@ # Network SoA — Phase 0 design spec -**Status:** Authoritative for `local/cpu-network-soa` (Phase 0 complete → Phase 1 scaffold). +**Status:** Authoritative for `local/cpu-network-soa`. Phases 0–4 dual-write implemented; see `GROK-NETWORK-SOA.md` for status and gaps. **Branch:** `local/cpu-network-soa` (from `origin/master`) **Sibling (paused):** `~/neuron/nrngpu` @ `local/gpu-native-qualification` — GPU network buffers Stages 2–3 **Handoff:** `GROK-NETWORK-SOA.md` diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 8c39872a20..eae8757fc4 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2858,6 +2858,9 @@ void NetCvode::init_events() { } } } + // Rebuild CoreNEURON-shaped fanout ranges before the run (not only on first spike). + PreSyn::mark_fanout_unsorted(); + PreSyn::ensure_fanout_order(); } // iterate over all NetCon in creation order to call // NETRECEIVE INITIAL blocks. @@ -3147,9 +3150,25 @@ void PreSyn::ensure_fanout_order() { g_network_fanout_order.clear(); if (net_cvode_instance && net_cvode_instance->psl_) { for (PreSyn* ps: *net_cvode_instance->psl_) { + // Refresh dual-write scalars (threshold may have been HOC-steered via double*). + ps->_soa.threshold() = ps->threshold_; + ps->_soa.gid() = ps->gid_; + ps->_soa.output_index() = ps->output_index_; + ps->_soa.thread_id() = ps->nt_ ? ps->nt_->id : -1; + if (ps->thvar_ && ps->thvar_.refers_to_a_modern_data_structure()) { + try { + ps->_soa.thvar_row() = static_cast(ps->thvar_.current_row()); + } catch (...) { + ps->_soa.thvar_row() = -1; + } + } else { + ps->_soa.thvar_row() = -1; + } ps->_soa.nc_index() = static_cast(g_network_fanout_order.size()); for (NetCon* nc: ps->dil_) { g_network_fanout_order.push_back(nc); + // Keep NetCon reverse edge / delay dual-write current before sim. + nc->soa_sync(); } ps->_soa.nc_count() = static_cast(ps->dil_.size()); } @@ -4915,7 +4934,8 @@ void NetCon::soa_sync() { } if (!weight_soa_.empty()) { _soa.weight_index() = static_cast(weight_soa_.front().current_row()); - weights_heap_to_soa(); + // Do not mirror heap → SoA here: HOC weight() writes SoA first; heap is only + // a pnt_receive buffer. heap→SoA after MOD runs (weights_heap_to_soa). } else { _soa.weight_index() = -1; } diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index b161e9c925..8826430f8f 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -246,6 +246,14 @@ if(NRN_ENABLE_PYTHON) COMMAND "${exe}" ${pytest} "./test/${group}" SCRIPT_PATTERNS "test/${group}/*.json" "test/${group}/*.py") endforeach() + # Focused CPU gate for network SoA dual-write (NetStim→ExpSyn). + nrn_add_test( + GROUP pytest + NAME network_soa_delivery_py${pyver} + PRELOAD_SANITIZER + ENVIRONMENT "CC=${CMAKE_C_COMPILER}" + COMMAND "${exe}" ${pytest} "./test/pytest/test_network_soa_delivery.py" + SCRIPT_PATTERNS "test/pytest/test_network_soa_delivery.py") endforeach() # Add some tests that are specifically aimed at NEURON - Python integration, covering different diff --git a/test/pytest/test_network_soa_delivery.py b/test/pytest/test_network_soa_delivery.py new file mode 100644 index 0000000000..b2d29e44c3 --- /dev/null +++ b/test/pytest/test_network_soa_delivery.py @@ -0,0 +1,70 @@ +""" +CPU delivery gate for network SoA dual-write (local/cpu-network-soa). + +NetStim -> NetCon -> ExpSyn exercises: + - Weight SoA dual-write and HOC weight[] + - nrn_pnt_receive_by_weight_index around pnt_receive + - PreSyn fanout order rebuild at init_events +""" +from neuron import h + + +def test_netstim_expsyn_delivery_near_1_025_ms(): + """Spike at t=1, delay 0.025 → receive ~1.025; g should peak soon after.""" + s = h.Section(name="soma") + s.insert("pas") + s.L = s.diam = 10 + + syn = h.ExpSyn(s(0.5)) + syn.tau = 1 + syn.e = 0 + + ns = h.NetStim() + ns.start = 1.0 + ns.number = 1 + ns.noise = 0 + + nc = h.NetCon(ns, syn) + # Write weight through HOC (Weight SoA dual-write path when available). + nc.weight[0] = 0.05 + nc.delay = 0.025 + + h.dt = 0.025 + h.finitialize(-65) + + g_max = 0.0 + t_gmax = 0.0 + while h.t < 3.0 - h.dt / 2: + h.fadvance() + g = float(syn.g) + if g > g_max: + g_max = g + t_gmax = float(h.t) + + assert g_max > 0.04, f"expected ExpSyn conductance peak, got g_max={g_max}" + assert 1.0 <= t_gmax <= 1.2, f"expected peak near 1.025 ms window, got t_gmax={t_gmax}" + + +def test_netcon_weight_hoc_roundtrip(): + """HOC weight set is visible to delivery (SoA dual-write + materialize).""" + s = h.Section(name="soma2") + s.insert("pas") + s.L = s.diam = 10 + syn = h.ExpSyn(s(0.5)) + ns = h.NetStim() + ns.start = 0.5 + ns.number = 1 + ns.noise = 0 + nc = h.NetCon(ns, syn) + nc.weight[0] = 0.12 + nc.delay = 0.0 + assert abs(nc.weight[0] - 0.12) < 1e-12 + + h.dt = 0.025 + h.finitialize(-65) + g_max = 0.0 + while h.t < 2.0 - h.dt / 2: + h.fadvance() + g_max = max(g_max, float(syn.g)) + # Larger weight → larger g peak than default-zero would give + assert g_max > 0.1, f"weight 0.12 should produce larger g, g_max={g_max}" From 26fcf128969426c10429c075aa73675692caf1ea Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Mon, 13 Jul 2026 15:44:06 -0400 Subject: [PATCH 09/33] Keep section_fwd MOD-light: Point_process SoA ownership off-header MOD/nrnivmodl builds only see build/include. Embedding PointProcess owning_handle forced network/model_data headers that are not installed there, breaking demo release compiles. Store a non-owning _soa_id on Point_process; own the SoA row via a map in point.cpp. Add point_process_access.hpp / nrn_point_process_soa_row for library code. --- .../network/point_process_access.hpp | 30 ++++++++++++++++++ src/nrncvode/netcvode.cpp | 6 ++-- src/nrnoc/point.cpp | 31 +++++++++++++++++-- src/nrnoc/section_fwd.hpp | 27 ++++++++++++---- test/unit_tests/container/network.cpp | 12 ++++--- 5 files changed, 91 insertions(+), 15 deletions(-) create mode 100644 src/neuron/container/network/point_process_access.hpp diff --git a/src/neuron/container/network/point_process_access.hpp b/src/neuron/container/network/point_process_access.hpp new file mode 100644 index 0000000000..1d6bd7f261 --- /dev/null +++ b/src/neuron/container/network/point_process_access.hpp @@ -0,0 +1,30 @@ +#pragma once +/** + * @file point_process_access.hpp + * @brief Internal helper to get a PointProcess SoA handle from Point_process*. + * + * Not for MOD files / section_fwd.hpp — only NEURON library TUs that already + * link model_data. + */ +#include "neuron/container/network/point_process.hpp" +#include "neuron/model_data.hpp" +#include "section_fwd.hpp" + +#include + +namespace neuron::container::network { + +/** @brief Non-owning handle to the dual-write SoA row for @p pnt. */ +inline PointProcess::handle point_process_soa(Point_process* pnt) { + assert(pnt); + assert(pnt->_soa_id); + return PointProcess::handle{ + non_owning_identifier{&neuron::model().point_processes(), + pnt->_soa_id}}; +} + +inline PointProcess::handle point_process_soa(Point_process const* pnt) { + return point_process_soa(const_cast(pnt)); +} + +} // namespace neuron::container::network diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index eae8757fc4..714276bcb4 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2275,7 +2275,7 @@ void NetCvode::remove_event(TQItem* q, int tid) { namespace { /** Phase 4: fill SelfEvent dual-write index fields from weight* / Point_process. */ void selfevent_set_indices(SelfEvent* se, Point_process* pnt, double* weight) { - se->target_row_ = pnt ? static_cast(pnt->_soa.current_row()) : -1; + se->target_row_ = nrn_point_process_soa_row(pnt); se->weight_index_ = -1; if (!weight) { return; @@ -3372,7 +3372,7 @@ DiscreteEvent* SelfEvent::savestate_read(FILE* f) { se->target_ = SelfEvent::index2pp(pptype, ppindex); se->weight_ = nullptr; se->weight_index_ = -1; - se->target_row_ = se->target_ ? static_cast(se->target_->_soa.current_row()) : -1; + se->target_row_ = nrn_point_process_soa_row(se->target_); if (ncindex >= 0) { NetCon* nc = NetConSave::index2netcon(ncindex); se->weight_ = nc->weight_; @@ -4922,7 +4922,7 @@ void NetCon::soa_sync() { _soa.weight_count() = cnt_; if (target_) { // Point_process dual-write row (Phase 1). - _soa.target() = static_cast(target_->_soa.current_row()); + _soa.target() = nrn_point_process_soa_row(target_); } else { _soa.target() = -1; } diff --git a/src/nrnoc/point.cpp b/src/nrnoc/point.cpp index 6a3f9f1985..04fbdd6965 100644 --- a/src/nrnoc/point.cpp +++ b/src/nrnoc/point.cpp @@ -7,11 +7,13 @@ saves the pointtype as later argument to create and loc */ #include #include "membfunc.h" +#include "neuron/container/network/point_process_access.hpp" #include "nrniv_mf.h" #include "ocnotify.h" #include "parse_with_deps.hpp" #include "section.h" +#include extern char* pnt_map; extern Symbol** pointsym; /*list of variable symbols in s->u.ppsym[k] @@ -30,11 +32,36 @@ Prop* nrn_point_prop_; void (*nrnpy_o2loc_p_)(Object*, Section**, double*); void (*nrnpy_o2loc2_p_)(Object*, Section**, double*); +namespace { +// Owning SoA rows keyed by Point_process* so section_fwd.hpp stays MOD-light. +std::unordered_map + g_point_process_soa_owners; +} // namespace + +Point_process::Point_process() { + auto& owner = g_point_process_soa_owners + .emplace(this, neuron::model().point_processes()) + .first->second; + _soa_id = owner.id(); +} + +Point_process::~Point_process() { + g_point_process_soa_owners.erase(this); + _soa_id = {}; +} + +int nrn_point_process_soa_row(Point_process const* pnt) { + if (!pnt || !pnt->_soa_id) { + return -1; + } + return static_cast(pnt->_soa_id.current_row()); +} + void nrn_point_process_soa_sync(Point_process* pnt) { - if (!pnt) { + if (!pnt || !pnt->_soa_id) { return; } - auto& h = pnt->_soa; + auto h = neuron::container::network::point_process_soa(pnt); if (pnt->prop) { h.mech_type() = pnt->prop->_type; // Prop for a point process always owns a mechanism SoA row. diff --git a/src/nrnoc/section_fwd.hpp b/src/nrnoc/section_fwd.hpp index b2d8ea2043..7fbd01e36e 100644 --- a/src/nrnoc/section_fwd.hpp +++ b/src/nrnoc/section_fwd.hpp @@ -1,12 +1,15 @@ #pragma once #include "multicore.h" #include "neuron/container/generic_data_handle.hpp" -#include "neuron/container/network/point_process.hpp" -#include "neuron/model_data.hpp" +#include "neuron/container/non_owning_soa_identifier.hpp" #include "nrnredef.h" /** * @file section_fwd.hpp * @brief Forward declarations of Section, Node etc. to be included in translated MOD files. + * + * Keep this header light: MOD / nrnivmodl compile against build/include only. + * Network SoA ownership lives in point.cpp (see nrn_point_process_soa_*), not + * as a heavy owning_handle member here. */ struct Node; struct Prop; @@ -85,12 +88,24 @@ struct Point_process { void* presyn_{}; /* non-threshold presynapse for NetCon */ void* nvi_{}; /* NrnVarIntegrator (for local step method) */ void* _vnt{}; /* NrnThread* (for NET_RECEIVE and multicore) */ - /** @brief Phase 1 dual-write: SoA row (Instance, MechType, ThreadId). - * @see doc/network-soa-phase0.md §5.2 + /** + * @brief Non-owning id of the network::PointProcess SoA row (Phase 1 dual-write). + * + * Owning lifetime is managed in point.cpp so this header stays free of + * model_data / network SoA includes (required for nrnivmodl / demo MOD builds). */ - neuron::container::network::PointProcess::owning_handle _soa{ - neuron::model().point_processes()}; + neuron::container::non_owning_identifier_without_container _soa_id{}; + + Point_process(); + ~Point_process(); + Point_process(Point_process const&) = delete; + Point_process& operator=(Point_process const&) = delete; + Point_process(Point_process&&) = delete; + Point_process& operator=(Point_process&&) = delete; }; /** @brief Sync legacy Point_process fields into the network SoA row. */ void nrn_point_process_soa_sync(Point_process* pnt); + +/** @brief Current SoA row of a Point_process dual-write entry (-1 if none). */ +int nrn_point_process_soa_row(Point_process const* pnt); diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index 3a8a613c90..d9671a8480 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -1,5 +1,6 @@ #include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" +#include "neuron/container/network/point_process_access.hpp" #include "neuron/container/network/presyn.hpp" #include "neuron/container/network/self_event.hpp" #include "neuron/container/network/weight_block.hpp" @@ -114,13 +115,16 @@ TEST_CASE("Point_process dual-write into network SoA", auto& storage = neuron::model().point_processes(); auto const before = storage.size(); GIVEN("A default-constructed Point_process shell") { - // Allocates an SoA row via Point_process::_soa (Phase 1 dual-write). + // Allocates an SoA row via Point_process ctor dual-write map. auto* pp = new Point_process{}; THEN("SoA size grows by one and fields are defaults until prop is set") { REQUIRE(storage.size() == before + 1); - REQUIRE(pp->_soa.mech_type() == -1); - REQUIRE(pp->_soa.instance() == -1); - REQUIRE(pp->_soa.thread_id() == -1); + REQUIRE(pp->_soa_id); + REQUIRE(nrn_point_process_soa_row(pp) >= 0); + auto h = neuron::container::network::point_process_soa(pp); + REQUIRE(h.mech_type() == -1); + REQUIRE(h.instance() == -1); + REQUIRE(h.thread_id() == -1); } WHEN("The Point_process is destroyed") { delete pp; From 93cf18c68de2ab10a6f9506bb0c361cff0f87d96 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Tue, 14 Jul 2026 06:58:22 -0400 Subject: [PATCH 10/33] Fix dual-write for NET_RECEIVE INITIAL and FOR_NETCONS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit HOC weight[] is SoA-primary; pnt_receive_init and FOR_NETCONS mutate the heap. Sync SoA↔heap around INITIAL, and for FOR_NETCONS targets sync all NetCons sharing the target (including WatchCondition flag delivery). Fixes hoctests::test_netrec_init_py and external_nrntest FOR_NETCONS/stdp1. --- src/nrncvode/netcvode.cpp | 72 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 714276bcb4..122b21d917 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2291,6 +2291,39 @@ void selfevent_set_indices(SelfEvent* se, Point_process* pnt, double* weight) { } } // namespace +namespace { +bool type_has_fornetcon(int type) { + for (int i = 0; i < nrn_fornetcon_cnt_; ++i) { + if (nrn_fornetcon_type_[i] == type) { + return true; + } + } + return false; +} + +/** Sync all NetCon weight heaps ↔ SoA that share a target (FOR_NETCONS mutates them). */ +void sync_netcon_weights_for_target(Point_process* pnt, bool soa_to_heap) { + if (!pnt) { + return; + } + Symbol* sym = hoc_lookup("NetCon"); + if (!sym || !sym->u.ctemplate || !sym->u.ctemplate->olist) { + return; + } + hoc_Item* q = nullptr; + ITERATE(q, sym->u.ctemplate->olist) { + auto* nc = static_cast(OBJ(q)->u.this_pointer); + if (nc && nc->target_ == pnt) { + if (soa_to_heap) { + nc->weights_soa_to_heap(); + } else { + nc->weights_heap_to_soa(); + } + } + } +} +} // namespace + void nrn_pnt_receive_by_weight_index(Point_process* pnt, int weight_index, double flag, @@ -2303,16 +2336,30 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, return; } int const n = pnt_receive_size[type]; + bool const fornet = type_has_fornetcon(type); + // Prefer NetCon dual-write owners when heap base is a known NetCon weight_. if (weight_heap) { if (NetCon* nc = NetConSave::weight2netcon(weight_heap)) { - nc->weights_soa_to_heap(); + if (fornet) { + // FOR_NETCONS walks weight_ of all NetCons with this target. + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); + } else { + nc->weights_soa_to_heap(); + } POINT_RECEIVE(type, pnt, weight_heap, flag); - nc->weights_heap_to_soa(); + if (fornet) { + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); + } else { + nc->weights_heap_to_soa(); + } return; } } // Index-only path: materialize a temporary (or fill provided heap) from Weight SoA. + if (fornet) { + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); + } double* buf = weight_heap; std::vector tmp; if (!buf && n > 0) { @@ -2326,6 +2373,9 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, if (weight_index >= 0 && n > 0 && buf) { neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); } + if (fornet) { + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); + } } // for threads, revised net_send to use absolute time (in the @@ -2875,6 +2925,9 @@ void NetCvode::init_events() { auto* d = static_cast(obj->u.this_pointer); if (d->target_) { int type = d->target_->prop->_type; // somehow prop is non-deterministically-null here + // Dual-write: INITIAL and HOC weight[] must share one value stream. + // SoA is HOC-primary; heap is the buffer for generated pnt_receive_init. + d->weights_soa_to_heap(); if (pnt_receive_init[type]) { (*pnt_receive_init[type])(d->target_, d->weight_, 0); } else { @@ -2883,6 +2936,7 @@ void NetCvode::init_events() { d->weight_[j] = 0.; } } + d->weights_heap_to_soa(); } } if (gcv_) { @@ -5600,7 +5654,15 @@ void WatchCondition::deliver(double tt, NetCvode* ns, NrnThread* nt) { PP2t(pnt_) = tt; } STATISTICS(watch_deliver_); + // WATCH-driven NET_RECEIVE (e.g. flag=2) may run FOR_NETCONS, which mutates + // every NetCon weight_ heap sharing this target — keep SoA dual-write coherent. + if (type_has_fornetcon(type)) { + sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ true); + } POINT_RECEIVE(type, pnt_, nullptr, nrflag_); + if (type_has_fornetcon(type)) { + sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ false); + } if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during WatchCondition deliver to NET_RECEIVE", (char*) 0); @@ -5691,7 +5753,13 @@ void WatchCondition::pgvts_deliver(double tt, NetCvode* ns) { } int type = pnt_->prop->_type; STATISTICS(watch_deliver_); + if (type_has_fornetcon(type)) { + sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ true); + } POINT_RECEIVE(type, pnt_, nullptr, nrflag_); + if (type_has_fornetcon(type)) { + sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ false); + } if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during WatchCondition deliver to NET_RECEIVE", (char*) 0); From 0b00a0736c227fe7329b8afb6e18a18f2d934400 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Tue, 21 Jul 2026 17:09:37 -0400 Subject: [PATCH 11/33] Wire network SoA into model sort; short-lived weight materialize on deliver Include PointProcess, Weight, NetCon, and PreSyn containers in nrn_ensure_model_data_are_sorted (thread partition, weight-block repack, fanout rebuild, cache offsets). On NetCon deliver without FOR_NETCONS, pass a temporary buffer into pnt_receive instead of the long-lived weight_ heap so SoA stays the source of truth while the heap remains for SaveState and FOR_NETCONS. --- GROK-NETWORK-SOA.md | 53 +++- cmake/NeuronFileLists.cmake | 1 + doc/network-soa-phase0.md | 2 +- src/neuron/cache/model_data.hpp | 8 + src/neuron/container/network/sort.hpp | 34 +++ src/neuron/model_data.hpp | 10 + src/nrncvode/netcvode.cpp | 10 +- src/nrncvode/network_soa_sort.cpp | 356 ++++++++++++++++++++++++++ src/nrnoc/treeset.cpp | 28 ++ test/unit_tests/container/network.cpp | 91 +++++++ 10 files changed, 577 insertions(+), 16 deletions(-) create mode 100644 src/neuron/container/network/sort.hpp create mode 100644 src/nrncvode/network_soa_sort.cpp diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 09953c941d..1eccbe52cf 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -13,16 +13,18 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s | **2** NetCon SoA; HOC weight → Weight SoA | **Done** | Delivery materializes `double*` for MOD | | **3** PreSyn SoA + fanout `NcIndex`/`NcCount` | **Done** | Rebuild at `init_events` + lazy on spike | | **4** SelfEvent `weight_index` + receive-by-index | **Done** | M2: still `pnt_receive(..., double*, flag)` | +| **Sort wiring** | **Done** | Network in `nrn_ensure_model_data_are_sorted` (thread slices, weight repack) | +| **Heap-drop policy** | **In progress** | Short-lived materialize on deliver; heap kept for FOR_NETCONS / SaveState / INITIAL | | **5** GPU net buffers | **Out of scope** | `local/gpu-native-qualification` after merge | -**Dual-write complete for the CoreNEURON-shaped data plane.** Legacy pointers/`dil_`/`weight_` heap remain for interpreter, queue, and nocmodl ABI. +**Dual-write complete for the CoreNEURON-shaped data plane.** Network SoA participates in the global sort gate. Legacy pointers/`dil_`/`weight_` heap remain for interpreter, queue, and nocmodl ABI where required. ### Known gaps (do not “just drop heap” yet) -- Long-lived `NetCon::weight_` heap still required for generated `pnt_receive` ABI (default **nocmodl**, `NRN_ENABLE_NMODL=OFF`). -- Network containers not yet in `nrn_ensure_model_data_are_sorted()` (no full network permute/repack pass). +- Long-lived `NetCon::weight_` heap still allocated; hot-path deliver uses **short-lived materialize** unless the target type has `FOR_NETCONS` (needs stable heap bases). Full drop needs SaveState index keys + INITIAL/FOR_NETCONS migration. +- Default codegen is **nocmodl** (`NRN_ENABLE_NMODL=OFF`); native `weight_index` MOD ABI is a separate project. - `InputPreSyn` / thin remote gid→fanout deferred (see phase0 §5.5.1). -- SaveState still largely keyed by weight pointers; SoA repack would need map invalidation. +- SaveState still largely keyed by weight pointers; weight SoA permute keeps handles stable but pointer map is independent. - Full `ctest -j N` needs a complete build/install; judge network work with filters below first. ### Default codegen @@ -79,16 +81,42 @@ Key paths: | PP dual-write | `Point_process::_soa`, `nrn_point_process_soa_sync` | | NetCon / weights | `NetCon::_soa`, `weight_soa_`, `soa_sync` | | Fanout | `PreSyn::ensure_fanout_order`, global NetCon* order | +| Sort / repack | `network_soa_sort.cpp`, `sort_network_data` in ensure_sorted | | Receive by index | `nrn_pnt_receive_by_weight_index` | --- -## Recommended next work (after dual-write) +## Sort wiring (implemented) -1. ~~Handoff status + automated delivery gate~~ (this session if present). -2. Wire network storages into `nrn_ensure_model_data_are_sorted` (thread slices, weight repack). -3. Policy: short-lived materialize vs nocmodl `weight_index` ABI; then drop long-lived heap `weight_`. -4. SaveState / `weight2netcon` → id or weight_index keys. +`nrn_ensure_model_data_are_sorted()` now freezes and sorts network containers after nodes + mechanisms: + +1. **PointProcess** — partition by `ThreadId`; set `cache.thread[i].point_process_offset` +2. **Weight** — repack contiguous per-NetCon blocks ordered by target thread; set `weight_offset` +3. **NetCon** — same order as weight packing; set `netcon_offset`; refresh dual-write indices +4. **PreSyn** — partition by thread; rebuild `NcIndex`/`NcCount` fanout; set `presyn_offset` + +Implementation: `src/nrncvode/network_soa_sort.cpp`, `neuron/container/network/sort.hpp`. + +## Heap-drop policy (decision) + +| Path | Weight source | Notes | +|------|---------------|--------| +| NetCon deliver (no FOR_NETCONS) | SoA → **temp buffer** → `pnt_receive` | Hot path; heap not required for content | +| NetCon deliver (FOR_NETCONS) | SoA → long-lived `weight_` heaps | MOD walks other NetCon `weight_` pointers | +| `pnt_receive_init` / HOC INITIAL | heap buffer + SoA sync | Keep until INITIAL uses index | +| SaveState / `weight2netcon` | heap pointer map | Migrate to `weight_index` / NetCon id before free | +| HOC `weight[i]` | SoA `data_handle` | Already SoA-primary | + +**Do not free `weight_` until** FOR_NETCONS, SaveState, and INITIAL no longer need stable heap bases. Prefer short-lived materialize everywhere else (current default for simple deliver). + +Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep materialize around generated `pnt_receive(double*)`. + +## Recommended next work + +1. ~~Wire network into `nrn_ensure_model_data_are_sorted`~~ (done). +2. ~~Heap-drop policy + short-lived materialize on simple deliver~~ (done; full free deferred). +3. SaveState / `weight2netcon` → id or weight_index keys (unblocks full heap free). +4. FOR_NETCONS / INITIAL without long-lived heap; then drop `weight_` allocation. 5. Merge toward master; GPU track rebases for Phase 5-style buffers. --- @@ -136,9 +164,10 @@ Read ~/neuron/cpu_net_soa/GROK-NETWORK-SOA.md, AGENTS.md, and doc/network-soa-ph Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa. Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU network buffers paused). -Phases 0–4 dual-write are done. Next: wire network SoA into -nrn_ensure_model_data_are_sorted (thread slices, weight repack), or -SaveState index keys — not GPU net_buf_receive on this branch. +Phases 0–4 dual-write + network sort wiring are done. Heap-drop policy: +short-lived materialize on simple deliver; keep weight_ for FOR_NETCONS / +SaveState / INITIAL. Next: SaveState weight_index keys, or full heap free +preconditions — not GPU net_buf_receive on this branch. ``` --- diff --git a/cmake/NeuronFileLists.cmake b/cmake/NeuronFileLists.cmake index ca60833d2e..e506417a23 100644 --- a/cmake/NeuronFileLists.cmake +++ b/cmake/NeuronFileLists.cmake @@ -262,6 +262,7 @@ set(NRNCVODE_FILE_LIST cvtrset.cpp htlist.cpp netcvode.cpp + network_soa_sort.cpp nrndaspk.cpp occvode.cpp tqueue.cpp) diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index 7778679343..a1fc4d33e9 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -1,6 +1,6 @@ # Network SoA — Phase 0 design spec -**Status:** Authoritative for `local/cpu-network-soa`. Phases 0–4 dual-write implemented; see `GROK-NETWORK-SOA.md` for status and gaps. +**Status:** Authoritative for `local/cpu-network-soa`. Phases 0–4 dual-write + network sort wiring implemented; see `GROK-NETWORK-SOA.md` for status and gaps. **Branch:** `local/cpu-network-soa` (from `origin/master`) **Sibling (paused):** `~/neuron/nrngpu` @ `local/gpu-native-qualification` — GPU network buffers Stages 2–3 **Handoff:** `GROK-NETWORK-SOA.md` diff --git a/src/neuron/cache/model_data.hpp b/src/neuron/cache/model_data.hpp index 675b9b4097..14d410aa6f 100644 --- a/src/neuron/cache/model_data.hpp +++ b/src/neuron/cache/model_data.hpp @@ -32,6 +32,14 @@ struct Thread { * @brief Offsets into global mechanism storage for this thread (one per mechanism) */ std::vector mechanism_offset{}; + /** + * @brief Offsets into global network SoA storages for this thread. + * @see doc/network-soa-phase0.md §6 + */ + std::size_t point_process_offset{}; + std::size_t weight_offset{}; + std::size_t netcon_offset{}; + std::size_t presyn_offset{}; }; struct Model { std::vector thread{}; diff --git a/src/neuron/container/network/sort.hpp b/src/neuron/container/network/sort.hpp new file mode 100644 index 0000000000..fdaab21898 --- /dev/null +++ b/src/neuron/container/network/sort.hpp @@ -0,0 +1,34 @@ +#pragma once +/** + * @file network/sort.hpp + * @brief Sort / repack network SoA containers inside nrn_ensure_model_data_are_sorted. + * + * Design: doc/network-soa-phase0.md §6, §8.2–§8.4. + */ +#include "neuron/cache/model_data.hpp" +#include "neuron/model_data.hpp" + +namespace neuron::container::network { + +/** + * @brief Partition and repack network SoA for integration. + * + * Order (after nodes + mechanisms are already sorted): + * 1. PointProcess by thread + * 2. Weight blocks contiguous per NetCon, packed by target thread + * 3. NetCon by (target thread, src PreSyn) + * 4. PreSyn by thread + rebuild NcIndex/NcCount fanout ranges + * + * @param cache Working model cache (thread offsets filled here). + * @param pp_token Sole frozen token for PointProcess storage. + * @param w_token Sole frozen token for Weight storage. + * @param nc_token Sole frozen token for NetCon storage. + * @param ps_token Sole frozen token for PreSyn storage. + */ +void sort_network_data(neuron::cache::Model& cache, + PointProcess::storage::frozen_token_type& pp_token, + Weight::storage::frozen_token_type& w_token, + NetCon::storage::frozen_token_type& nc_token, + PreSyn::storage::frozen_token_type& ps_token); + +} // namespace neuron::container::network diff --git a/src/neuron/model_data.hpp b/src/neuron/model_data.hpp index 9174ebf651..cae73b78ac 100644 --- a/src/neuron/model_data.hpp +++ b/src/neuron/model_data.hpp @@ -245,6 +245,16 @@ struct model_sorted_token { } container::Node::storage::frozen_token_type node_data_token; std::vector mech_data_tokens{}; + /** + * @brief Frozen tokens for network SoA containers (sorted with the model). + * + * Held as vectors (0 or 1 element) because frozen_token_type is not default- + * constructible; same pattern as mech_data_tokens. + */ + std::vector point_process_tokens{}; + std::vector weight_tokens{}; + std::vector netcon_tokens{}; + std::vector presyn_tokens{}; private: std::reference_wrapper m_cache; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 122b21d917..a213a02cad 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -3114,13 +3114,16 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); - // Phase 4: deliver via weight_index (SoA), materializing double* for MOD. + // Phase 4 + heap-drop policy: prefer weight_index → short-lived materialize. + // Pass long-lived weight_ heap only when FOR_NETCONS needs stable bases for all + // NetCons on this target; otherwise a temporary buffer is enough for nocmodl ABI. int widx = _soa.weight_index(); if (widx < 0 && !weight_soa_.empty()) { widx = static_cast(weight_soa_.front().current_row()); } if (widx >= 0) { - nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); + double* heap_for_abi = type_has_fornetcon(type) ? weight_ : nullptr; + nrn_pnt_receive_by_weight_index(target_, widx, 0., heap_for_abi); } else { weights_soa_to_heap(); POINT_RECEIVE(type, target_, weight_, 0); @@ -3146,7 +3149,8 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { widx = static_cast(weight_soa_.front().current_row()); } if (widx >= 0) { - nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); + double* heap_for_abi = type_has_fornetcon(type) ? weight_ : nullptr; + nrn_pnt_receive_by_weight_index(target_, widx, 0., heap_for_abi); } else { weights_soa_to_heap(); POINT_RECEIVE(type, target_, weight_, 0); diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp new file mode 100644 index 0000000000..209f651e80 --- /dev/null +++ b/src/nrncvode/network_soa_sort.cpp @@ -0,0 +1,356 @@ +/** + * @file network_soa_sort.cpp + * @brief Network SoA sort / weight repack for nrn_ensure_model_data_are_sorted. + */ +#include "neuron/container/network/point_process_access.hpp" +#include "neuron/container/network/sort.hpp" +#include "netcon.h" +#include "netcvode.h" +#include "section.h" + +#include +#include +#include +#include + +extern cTemplate** nrn_pnt_template_; +extern int n_memb_func; +extern NetCvode* net_cvode_instance; + +namespace neuron::container::network { +namespace { + +constexpr std::size_t npos = std::numeric_limits::max(); + +/** @brief Sync all Point_process dual-write rows from HOC point-process lists. */ +void sync_all_point_processes() { + if (!nrn_pnt_template_) { + return; + } + for (int type = 0; type < n_memb_func; ++type) { + cTemplate* tmp = nrn_pnt_template_[type]; + if (!tmp || !tmp->olist) { + continue; + } + hoc_Item* q = nullptr; + ITERATE(q, tmp->olist) { + auto* pnt = static_cast(OBJ(q)->u.this_pointer); + nrn_point_process_soa_sync(pnt); + } + } +} + +/** @brief Enumerate live NetCon shells from the HOC template list. */ +std::vector<::NetCon*> all_netcons() { + std::vector<::NetCon*> out; + Symbol* sym = hoc_lookup("NetCon"); + if (!sym || !sym->u.ctemplate || !sym->u.ctemplate->olist) { + return out; + } + out.reserve(static_cast(sym->u.ctemplate->count)); + hoc_Item* q = nullptr; + ITERATE(q, sym->u.ctemplate->olist) { + auto* nc = static_cast<::NetCon*>(OBJ(q)->u.this_pointer); + if (nc) { + out.push_back(nc); + } + } + return out; +} + +/** @brief Enumerate live PreSyn shells (psl_ when available). */ +std::vector<::PreSyn*> all_presyns() { + std::vector<::PreSyn*> out; + if (net_cvode_instance && net_cvode_instance->psl_) { + out.reserve(net_cvode_instance->psl_->size()); + for (::PreSyn* ps: *net_cvode_instance->psl_) { + if (ps) { + out.push_back(ps); + } + } + } + return out; +} + +int target_thread_id(::NetCon* nc) { + if (!nc || !nc->target_) { + return -1; + } + if (nc->target_->_vnt) { + return static_cast(nc->target_->_vnt)->id; + } + // Fall back to dual-write column if present. + int const row = nrn_point_process_soa_row(nc->target_); + if (row >= 0) { + return neuron::model().point_processes().get( + static_cast(row)); + } + return -1; +} + +int src_presyn_row(::NetCon* nc) { + if (!nc || !nc->src_) { + return -1; + } + return static_cast(nc->src_->_soa.current_row()); +} + +/** + * @brief Build perm[old_row] = new_row grouping rows by integer key in [0, nkey). + * @return Number of rows placed in the keyed region (orphans follow). + */ +template +std::vector partition_by_key(std::size_t n, + int nkey, + GetKey&& get_key, + std::vector& key_offsets) { + key_offsets.assign(static_cast(std::max(nkey, 0)), 0); + std::vector perm(n, npos); + std::size_t global_i = 0; + for (int k = 0; k < nkey; ++k) { + key_offsets[static_cast(k)] = global_i; + for (std::size_t row = 0; row < n; ++row) { + if (get_key(row) == k) { + perm[row] = global_i++; + } + } + } + for (std::size_t row = 0; row < n; ++row) { + if (perm[row] == npos) { + perm[row] = global_i++; + } + } + assert(global_i == n); + return perm; +} + +template +void apply_if_needed(Storage& store, Token& token, std::vector perm) { + if (store.size() == 0) { + store.mark_as_sorted(token); + return; + } + // apply_reverse_permutation always leaves the container sorted. + store.apply_reverse_permutation(std::move(perm), token); +} + +} // namespace + +void sort_network_data(neuron::cache::Model& cache, + PointProcess::storage::frozen_token_type& pp_token, + Weight::storage::frozen_token_type& w_token, + NetCon::storage::frozen_token_type& nc_token, + PreSyn::storage::frozen_token_type& ps_token) { + auto& model = neuron::model(); + auto& pp_store = model.point_processes(); + auto& w_store = model.weights(); + auto& nc_store = model.netcons(); + auto& ps_store = model.presyns(); + + // Ensure cache.thread is sized (caller should have done this; be defensive). + if (cache.thread.size() < static_cast(nrn_nthread)) { + cache.thread.resize(static_cast(nrn_nthread)); + } + + // ------------------------------------------------------------------ + // 0. Dual-write sync from legacy shells (values / thread membership). + // ------------------------------------------------------------------ + sync_all_point_processes(); + auto netcons = all_netcons(); + for (::NetCon* nc: netcons) { + // Keep WeightIndex from handles; do not heap→SoA wipe HOC-primary values. + nc->soa_sync(); + } + auto presyns = all_presyns(); + for (::PreSyn* ps: presyns) { + ps->soa_sync(); + } + + // ------------------------------------------------------------------ + // 1. PointProcess: partition by thread_id. + // ------------------------------------------------------------------ + { + std::vector pp_offsets; + auto perm = partition_by_key( + pp_store.size(), + nrn_nthread, + [&](std::size_t row) { + return pp_store.get(row); + }, + pp_offsets); + for (int tid = 0; tid < nrn_nthread; ++tid) { + cache.thread[static_cast(tid)].point_process_offset = + tid < static_cast(pp_offsets.size()) ? pp_offsets[static_cast(tid)] + : 0; + } + apply_if_needed(pp_store, pp_token, std::move(perm)); + } + + // Order NetCons by (target thread, src PreSyn SoA row, NetCon SoA row). + struct NcSortKey { + ::NetCon* nc{}; + int tid{-1}; + int src_row{-1}; + std::size_t soa_row{0}; + }; + std::vector ordered; + ordered.reserve(netcons.size()); + for (::NetCon* nc: netcons) { + NcSortKey k; + k.nc = nc; + k.tid = target_thread_id(nc); + k.src_row = src_presyn_row(nc); + k.soa_row = nc->_soa.current_row(); + ordered.push_back(k); + } + std::stable_sort(ordered.begin(), ordered.end(), [](NcSortKey const& a, NcSortKey const& b) { + if (a.tid != b.tid) { + // Unassigned (-1) after real threads. + int const at = a.tid < 0 ? nrn_nthread : a.tid; + int const bt = b.tid < 0 ? nrn_nthread : b.tid; + if (at != bt) { + return at < bt; + } + } + if (a.src_row != b.src_row) { + return a.src_row < b.src_row; + } + return a.soa_row < b.soa_row; + }); + + // ------------------------------------------------------------------ + // 2. Weight repack: contiguous blocks per NetCon, packed by target thread. + // ------------------------------------------------------------------ + { + std::size_t const wsize = w_store.size(); + std::vector desired; // old rows in new order + desired.reserve(wsize); + std::vector used(wsize, 0); + std::vector first_pos(static_cast(std::max(nrn_nthread, 0)), + npos); + + for (auto const& k: ordered) { + if (k.tid >= 0 && k.tid < nrn_nthread) { + auto& fp = first_pos[static_cast(k.tid)]; + if (fp == npos) { + fp = desired.size(); + } + } + for (auto& wh: k.nc->weight_soa_) { + if (!wh.id()) { + continue; + } + auto const row = wh.current_row(); + if (row < wsize && !used[row]) { + used[row] = 1; + desired.push_back(row); + } + } + } + // Orphan weight rows (not owned by any live NetCon) go at the end. + for (std::size_t row = 0; row < wsize; ++row) { + if (!used[row]) { + desired.push_back(row); + } + } + assert(desired.size() == wsize); + + std::size_t running = 0; + for (int tid = 0; tid < nrn_nthread; ++tid) { + auto const fp = first_pos[static_cast(tid)]; + if (fp != npos) { + running = fp; + } + cache.thread[static_cast(tid)].weight_offset = running; + } + + std::vector perm(wsize, npos); + for (std::size_t new_i = 0; new_i < desired.size(); ++new_i) { + perm[desired[new_i]] = new_i; + } + apply_if_needed(w_store, w_token, std::move(perm)); + } + + // ------------------------------------------------------------------ + // 3. NetCon SoA: same order as weight packing (target thread, src, …). + // ------------------------------------------------------------------ + { + std::size_t const n = nc_store.size(); + std::vector perm(n, npos); + std::vector first_pos(static_cast(nrn_nthread), npos); + std::size_t global_i = 0; + std::vector seen(n, 0); + for (auto const& k: ordered) { + auto const old = k.soa_row; + if (old < n && !seen[old]) { + if (k.tid >= 0 && k.tid < nrn_nthread) { + auto& fp = first_pos[static_cast(k.tid)]; + if (fp == npos) { + fp = global_i; + } + } + perm[old] = global_i++; + seen[old] = 1; + } + } + for (std::size_t row = 0; row < n; ++row) { + if (perm[row] == npos) { + perm[row] = global_i++; + } + } + assert(global_i == n); + std::size_t running = 0; + for (int tid = 0; tid < nrn_nthread; ++tid) { + auto const fp = first_pos[static_cast(tid)]; + if (fp != npos) { + running = fp; + } + cache.thread[static_cast(tid)].netcon_offset = running; + } + apply_if_needed(nc_store, nc_token, std::move(perm)); + } + + // Refresh dual-write indices after PP/weight/NetCon permutes. + for (::NetCon* nc: netcons) { + nc->soa_sync(); + } + + // ------------------------------------------------------------------ + // 4. PreSyn by thread + fanout ranges (NcIndex/NcCount). + // ------------------------------------------------------------------ + { + // Refresh thread_id on PreSyn shells before partition. + for (::PreSyn* ps: presyns) { + ps->soa_sync(); + } + std::vector ps_offsets; + auto perm = partition_by_key( + ps_store.size(), + nrn_nthread, + [&](std::size_t row) { return ps_store.get(row); }, + ps_offsets); + for (int tid = 0; tid < nrn_nthread; ++tid) { + cache.thread[static_cast(tid)].presyn_offset = + tid < static_cast(ps_offsets.size()) ? ps_offsets[static_cast(tid)] + : 0; + } + apply_if_needed(ps_store, ps_token, std::move(perm)); + } + + // Fanout order uses NetCon* ranges; rebuild after topology/sort. + ::PreSyn::mark_fanout_unsorted(); + ::PreSyn::ensure_fanout_order(); + + // Final NetCon reverse-edge refresh (src PreSyn rows may have moved). + for (::NetCon* nc: netcons) { + nc->soa_sync(); + } + + // Ensure all four are marked sorted even if empty / trivial perm paths. + pp_store.mark_as_sorted(pp_token); + w_store.mark_as_sorted(w_token); + nc_store.mark_as_sorted(nc_token); + ps_store.mark_as_sorted(ps_token); +} + +} // namespace neuron::container::network diff --git a/src/nrnoc/treeset.cpp b/src/nrnoc/treeset.cpp index 1d20b001ba..bf00829cc2 100644 --- a/src/nrnoc/treeset.cpp +++ b/src/nrnoc/treeset.cpp @@ -9,6 +9,7 @@ #include "neuron.h" #include "neuron/cache/mechanism_range.hpp" #include "neuron/cache/model_data.hpp" +#include "neuron/container/network/sort.hpp" #include "neuron/container/soa_container.hpp" #include "node_order_optim/node_order_optim.h" #include "nonvintblock.h" @@ -2213,6 +2214,18 @@ neuron::model_sorted_token nrn_ensure_model_data_are_sorted() { mech_tokens.push_back(mech_data.issue_frozen_token()); already_sorted = already_sorted && mech_data.is_sorted(); }); + // Network SoA containers (Phase 1–3 dual-write; sort after nodes+mechs). + std::vector pp_tokens; + std::vector weight_tokens; + std::vector netcon_tokens; + std::vector presyn_tokens; + pp_tokens.push_back(model.point_processes().issue_frozen_token()); + weight_tokens.push_back(model.weights().issue_frozen_token()); + netcon_tokens.push_back(model.netcons().issue_frozen_token()); + presyn_tokens.push_back(model.presyns().issue_frozen_token()); + already_sorted = already_sorted && model.point_processes().is_sorted() && + model.weights().is_sorted() && model.netcons().is_sorted() && + model.presyns().is_sorted(); // Now the whole model is marked frozen/read-only, but it may or may not be // marked sorted (if it is, the cache should be valid, otherwise it should // not be). @@ -2257,11 +2270,26 @@ neuron::model_sorted_token nrn_ensure_model_data_are_sorted() { // Now that all the mechanism data is sorted we can fill in pdata caches model.apply_to_mechanisms( [&cache](auto& mech_data) { nrn_fill_mech_data_caches(cache, mech_data); }); + // Network: PointProcess / Weight repack / NetCon / PreSyn + fanout. + // doc/network-soa-phase0.md §6.3, §8.4 + neuron::container::network::sort_network_data(cache, + pp_tokens[0], + weight_tokens[0], + netcon_tokens[0], + presyn_tokens[0]); + assert(model.point_processes().is_sorted()); + assert(model.weights().is_sorted()); + assert(model.netcons().is_sorted()); + assert(model.presyns().is_sorted()); // Move our working cache into the global storage. neuron::cache::model = std::move(cache); } // Move our tokens into the return value and be done with it. neuron::model_sorted_token ret{*neuron::cache::model, std::move(node_token)}; ret.mech_data_tokens = std::move(mech_tokens); + ret.point_process_tokens = std::move(pp_tokens); + ret.weight_tokens = std::move(weight_tokens); + ret.netcon_tokens = std::move(netcon_tokens); + ret.presyn_tokens = std::move(presyn_tokens); return ret; } diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index d9671a8480..f381aa3314 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -1,15 +1,19 @@ +#include "neuron/cache/model_data.hpp" #include "neuron/container/network/netcon.hpp" #include "neuron/container/network/point_process.hpp" #include "neuron/container/network/point_process_access.hpp" #include "neuron/container/network/presyn.hpp" #include "neuron/container/network/self_event.hpp" +#include "neuron/container/network/sort.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/model_data.hpp" +#include "nrn_ansi.h" #include "section.h" #include +#include #include #include #include @@ -322,3 +326,90 @@ TEST_CASE("SOA-backed PreSyn structure and fanout ranges", } } } + +TEST_CASE("Network SoA sort partitions PointProcess by thread", + "[Neuron][data_structures][network][sort]") { + auto& pp = neuron::model().point_processes(); + auto& w = neuron::model().weights(); + auto& nc = neuron::model().netcons(); + auto& ps = neuron::model().presyns(); + + // Intentionally create in non-thread order: tid 1, 0, 1, 0 + std::vector rows; + rows.emplace_back(pp); + rows.back().thread_id() = 1; + rows.back().instance() = 10; + rows.emplace_back(pp); + rows.back().thread_id() = 0; + rows.back().instance() = 20; + rows.emplace_back(pp); + rows.back().thread_id() = 1; + rows.back().instance() = 11; + rows.emplace_back(pp); + rows.back().thread_id() = 0; + rows.back().instance() = 21; + + auto w0 = Weight::allocate_weight_rows(1, nullptr); + w0[0].value() = 1.0; + Weight::owning_handle orphan{w}; + orphan.value() = 99.0; + auto w1 = Weight::allocate_weight_rows(1, nullptr); + w1[0].value() = 2.0; + + extern int nrn_nthread; + { + auto pp_tok = pp.issue_frozen_token(); + auto w_tok = w.issue_frozen_token(); + auto nc_tok = nc.issue_frozen_token(); + auto ps_tok = ps.issue_frozen_token(); + + neuron::cache::Model cache{}; + cache.thread.resize(static_cast(std::max(nrn_nthread, 1))); + + neuron::container::network::sort_network_data(cache, pp_tok, w_tok, nc_tok, ps_tok); + + REQUIRE(pp.is_sorted()); + REQUIRE(w.is_sorted()); + REQUIRE(nc.is_sorted()); + REQUIRE(ps.is_sorted()); + + // Handles keep logical values after partition. + REQUIRE(rows[0].thread_id() == 1); + REQUIRE(rows[0].instance() == 10); + REQUIRE(rows[1].thread_id() == 0); + REQUIRE(rows[1].instance() == 20); + + if (nrn_nthread >= 2) { + // Storage order: thread 0 rows then thread 1 rows. + REQUIRE(pp.get(0) == 0); + REQUIRE(pp.get(1) == 0); + REQUIRE(pp.get(2) == 1); + REQUIRE(pp.get(3) == 1); + REQUIRE(cache.thread[0].point_process_offset == 0); + REQUIRE(cache.thread[1].point_process_offset == 2); + } + } // freeze tokens released before destroying owning handles +} + +TEST_CASE("nrn_ensure_model_data_are_sorted freezes network containers", + "[Neuron][data_structures][network][sort]") { + // Needs a minimal model (same precondition as other ensure_sorted unit tests). + REQUIRE(hoc_oc("create s\nfinitialize(-65)\n") == 0); + neuron::model().point_processes().mark_as_unsorted(); + neuron::model().weights().mark_as_unsorted(); + neuron::model().netcons().mark_as_unsorted(); + neuron::model().presyns().mark_as_unsorted(); + { + auto token = nrn_ensure_model_data_are_sorted(); + REQUIRE(neuron::model().point_processes().is_sorted()); + REQUIRE(neuron::model().weights().is_sorted()); + REQUIRE(neuron::model().netcons().is_sorted()); + REQUIRE(neuron::model().presyns().is_sorted()); + REQUIRE(neuron::cache::model); + REQUIRE_FALSE(token.point_process_tokens.empty()); + REQUIRE_FALSE(token.weight_tokens.empty()); + REQUIRE_FALSE(token.netcon_tokens.empty()); + REQUIRE_FALSE(token.presyn_tokens.empty()); + } // release freeze before structural cleanup + REQUIRE(hoc_oc("delete_section()\n") == 0); +} From 821b658143a3ed3858b409ceef42cd9fb3921bea Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Tue, 21 Jul 2026 17:15:58 -0400 Subject: [PATCH 12/33] SaveState: dual-write weight restore and SelfEvent NetCon-index identity Save NetCon weights from Weight SoA (HOC-primary) and restore into both heap and SoA so materialize-on-deliver matches SaveState. Resolve SelfEvent ownership via NetCon object index and weight_index2netcon when the heap pointer is absent; index2netcon maps all NetCons by HOC index. --- GROK-NETWORK-SOA.md | 23 ++++++---- doc/network-soa-phase0.md | 7 +-- src/nrncvode/netcon.h | 9 ++++ src/nrncvode/netcvode.cpp | 58 +++++++++++++++++++++--- src/nrniv/savstate.cpp | 32 +++++++++++-- test/pytest/test_network_soa_delivery.py | 39 ++++++++++++++++ 6 files changed, 145 insertions(+), 23 deletions(-) diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 1eccbe52cf..61f61bbb02 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -14,7 +14,8 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s | **3** PreSyn SoA + fanout `NcIndex`/`NcCount` | **Done** | Rebuild at `init_events` + lazy on spike | | **4** SelfEvent `weight_index` + receive-by-index | **Done** | M2: still `pnt_receive(..., double*, flag)` | | **Sort wiring** | **Done** | Network in `nrn_ensure_model_data_are_sorted` (thread slices, weight repack) | -| **Heap-drop policy** | **In progress** | Short-lived materialize on deliver; heap kept for FOR_NETCONS / SaveState / INITIAL | +| **Heap-drop policy** | **In progress** | Short-lived materialize on deliver; heap kept for FOR_NETCONS / INITIAL | +| **SaveState dual-write** | **Done** | Weight save/restore via SoA; SelfEvent identity NetCon index + weight_index | | **5** GPU net buffers | **Out of scope** | `local/gpu-native-qualification` after merge | **Dual-write complete for the CoreNEURON-shaped data plane.** Network SoA participates in the global sort gate. Legacy pointers/`dil_`/`weight_` heap remain for interpreter, queue, and nocmodl ABI where required. @@ -24,7 +25,7 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s - Long-lived `NetCon::weight_` heap still allocated; hot-path deliver uses **short-lived materialize** unless the target type has `FOR_NETCONS` (needs stable heap bases). Full drop needs SaveState index keys + INITIAL/FOR_NETCONS migration. - Default codegen is **nocmodl** (`NRN_ENABLE_NMODL=OFF`); native `weight_index` MOD ABI is a separate project. - `InputPreSyn` / thin remote gid→fanout deferred (see phase0 §5.5.1). -- SaveState still largely keyed by weight pointers; weight SoA permute keeps handles stable but pointer map is independent. +- BBSaveState still uses weight pointer equality for SelfEvent binding (Commit B). - Full `ctest -j N` needs a complete build/install; judge network work with filters below first. ### Default codegen @@ -104,7 +105,8 @@ Implementation: `src/nrncvode/network_soa_sort.cpp`, `neuron/container/network/s | NetCon deliver (no FOR_NETCONS) | SoA → **temp buffer** → `pnt_receive` | Hot path; heap not required for content | | NetCon deliver (FOR_NETCONS) | SoA → long-lived `weight_` heaps | MOD walks other NetCon `weight_` pointers | | `pnt_receive_init` / HOC INITIAL | heap buffer + SoA sync | Keep until INITIAL uses index | -| SaveState / `weight2netcon` | heap pointer map | Migrate to `weight_index` / NetCon id before free | +| SaveState | NetCon obj index + SoA weights | Dual-write restore sync done; `weight2netcon` remains live-queue helper | +| BBSaveState | heap pointer match for SelfEvent | Next: same NetCon-index policy as SaveState | | HOC `weight[i]` | SoA `data_handle` | Already SoA-primary | **Do not free `weight_` until** FOR_NETCONS, SaveState, and INITIAL no longer need stable heap bases. Prefer short-lived materialize everywhere else (current default for simple deliver). @@ -115,9 +117,10 @@ Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep mate 1. ~~Wire network into `nrn_ensure_model_data_are_sorted`~~ (done). 2. ~~Heap-drop policy + short-lived materialize on simple deliver~~ (done; full free deferred). -3. SaveState / `weight2netcon` → id or weight_index keys (unblocks full heap free). -4. FOR_NETCONS / INITIAL without long-lived heap; then drop `weight_` allocation. -5. Merge toward master; GPU track rebases for Phase 5-style buffers. +3. ~~SaveState dual-write restore + SelfEvent NetCon-index identity~~ (done). +4. BBSaveState: SoA weight sync + SelfEvent NetCon-index binding (same policy as SaveState). +5. FOR_NETCONS / INITIAL without long-lived heap; then drop `weight_` allocation. +6. Merge toward master; GPU track rebases for Phase 5-style buffers. --- @@ -164,10 +167,10 @@ Read ~/neuron/cpu_net_soa/GROK-NETWORK-SOA.md, AGENTS.md, and doc/network-soa-ph Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa. Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU network buffers paused). -Phases 0–4 dual-write + network sort wiring are done. Heap-drop policy: -short-lived materialize on simple deliver; keep weight_ for FOR_NETCONS / -SaveState / INITIAL. Next: SaveState weight_index keys, or full heap free -preconditions — not GPU net_buf_receive on this branch. +Phases 0–4 dual-write + network sort wiring + SaveState dual-write are done. +Heap-drop policy: short-lived materialize on simple deliver; keep weight_ +for FOR_NETCONS / INITIAL / BBSaveState. Next: BBSaveState NetCon-index +SelfEvent binding, or full heap free preconditions — not GPU on this branch. ``` --- diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index a1fc4d33e9..b7e953c13c 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -492,10 +492,11 @@ Phase 1 only needs M1 scaffolding (container + optional pointer into SoA). **No | Concern | Impact | |---------|--------| -| `NetConSave` weight pointer tables | Today maps `double* → NetCon*`. After SoA, key by `weight_index` or stable NetCon id. Call `NetConSave::invalid()` on weight repack. | +| `NetConSave` weight pointer tables | Live helper `weight2netcon(double*)` remains; SaveState SelfEvent **file** identity is NetCon obj index + `weight_index2netcon`. Call `NetConSave::invalid()` when heap bases reallocate. | +| SaveState weight values | Save prefers Weight SoA; restore writes heap + SoA (`restorenet`). | | `PreSynSave` / `hi_index_` | Keep index tables; rebuild on unsorted. | -| BBSaveState fanout | Uses `PreSyn::fanout`; switch to `NcIndex`/`NcCount` in Phase 3. | -| Binary layout | Do not freeze on-disk format on intermediate dual-write; document stable index fields when dual-write ends. | +| BBSaveState | Still pointer-equality SelfEvent binding + heap weight IO — migrate like SaveState (Commit B). | +| Binary layout | SelfEvent line already stores `ncindex`; keep compatible. | Minimal Phase 1 impact: none if PointProcess/Weight SoA is not yet referenced by SaveState. diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index de0a51f027..35766aa34d 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -151,8 +151,17 @@ class NetConSave: public DiscreteEvent { NetCon* netcon_; static void invalid(); + /** @brief Map long-lived NetCon weight_ heap base → NetCon*. */ static NetCon* weight2netcon(double*); + /** @brief Map HOC NetCon object index → NetCon*. */ static NetCon* index2netcon(long); + /** + * @brief Map Weight SoA base row (weight_index) → NetCon*. + * + * Used when SelfEvent identity is carried as weight_index_ without a + * reliable weight_ heap pointer (dual-write / heap-drop path). + */ + static NetCon* weight_index2netcon(int weight_index); private: static NetConSaveWeightTable* wtable_; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index a213a02cad..6cb829c66e 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -3414,7 +3414,16 @@ DiscreteEvent* SelfEvent::savestate_save() { void SelfEvent::savestate_restore(double tt, NetCvode* nc) { // pr("savestate_restore", tt, nc); - nrn_net_send(movable_, weight_, target_, tt, flag_); + // Prefer long-lived NetCon heap base when known; if only weight_index_ survived + // (heap-drop / index identity), rebind via NetConSave::weight_index2netcon. + double* w = weight_; + if (!w && weight_index_ >= 0) { + if (NetCon* owner = NetConSave::weight_index2netcon(weight_index_)) { + w = owner->weight_; + weight_ = w; + } + } + nrn_net_send(movable_, w, target_, tt, flag_); } DiscreteEvent* SelfEvent::savestate_read(FILE* f) { @@ -3432,7 +3441,9 @@ DiscreteEvent* SelfEvent::savestate_read(FILE* f) { se->weight_index_ = -1; se->target_row_ = nrn_point_process_soa_row(se->target_); if (ncindex >= 0) { + // File identity is NetCon object index (not weight_ pointer). NetCon* nc = NetConSave::index2netcon(ncindex); + assert(nc); se->weight_ = nc->weight_; if (!nc->weight_soa_.empty()) { se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); @@ -3477,11 +3488,17 @@ void SelfEvent::savestate_write(FILE* f) { fprintf(f, "%d\n", SelfEventType); int const moff = movable_ ? (movable_ - target_->prop->dparam) : -1; int ncindex = -1; - // find the NetCon index for weight_ + // SelfEvent identity for SaveState: NetCon object index (stable), not weight_*. + // Prefer heap map when present; fall back to Weight SoA weight_index. + NetCon* owner = nullptr; if (weight_) { - NetCon* nc = NetConSave::weight2netcon(weight_); - assert(nc); - ncindex = nc->obj_->index; + owner = NetConSave::weight2netcon(weight_); + assert(owner); + } else if (weight_index_ >= 0) { + owner = NetConSave::weight_index2netcon(weight_index_); + } + if (owner && owner->obj_) { + ncindex = owner->obj_->index; } fprintf(f, @@ -5164,7 +5181,9 @@ NetCon* NetConSave::index2netcon(long id) { ITERATE(q, sym->u.ctemplate->olist) { Object* obj = OBJ(q); nc = (NetCon*) obj->u.this_pointer; - if (nc->weight_) { + // Index all NetCons by HOC object index (identity for SaveState), + // not only those with a weight_ heap (heap-drop readiness). + if (nc) { (*idxtable_)[obj->index] = nc; } } @@ -5179,6 +5198,33 @@ NetCon* NetConSave::index2netcon(long id) { } } +NetCon* NetConSave::weight_index2netcon(int weight_index) { + if (weight_index < 0) { + return nullptr; + } + Symbol* sym = hoc_lookup("NetCon"); + if (!sym || !sym->u.ctemplate || !sym->u.ctemplate->olist) { + return nullptr; + } + hoc_Item* q = nullptr; + ITERATE(q, sym->u.ctemplate->olist) { + auto* nc = static_cast(OBJ(q)->u.this_pointer); + if (!nc) { + continue; + } + int base = -1; + if (!nc->weight_soa_.empty()) { + base = static_cast(nc->weight_soa_.front().current_row()); + } else { + base = nc->_soa.weight_index(); + } + if (base == weight_index) { + return nc; + } + } + return nullptr; +} + void nrn_update_ps2nt() { net_cvode_instance->update_ps2nt(); } diff --git a/src/nrniv/savstate.cpp b/src/nrniv/savstate.cpp index bc6e9f2b6a..0967c77448 100644 --- a/src/nrniv/savstate.cpp +++ b/src/nrniv/savstate.cpp @@ -14,6 +14,8 @@ #include "vrecitem.h" #include "utils/enumerate.h" +#include + typedef void (*ReceiveFunc)(Point_process*, double*, double); #include "membfunc.h" @@ -929,8 +931,19 @@ void SaveState::savenet() { const NetCon* d = (NetCon*) ob->u.this_pointer; int n = ncs_[i].nstate; double* w = ncs_[i].state; - for (int j = 0; j < n; ++j) { - w[j] = d->weight_[j]; + // HOC weight[] is SoA-primary under dual-write; prefer SoA over heap. + if (!d->weight_soa_.empty()) { + int const m = std::min(n, static_cast(d->weight_soa_.size())); + for (int j = 0; j < m; ++j) { + w[j] = d->weight_soa_[j].value(); + } + for (int j = m; j < n; ++j) { + w[j] = d->weight_ ? d->weight_[j] : 0.; + } + } else if (d->weight_) { + for (int j = 0; j < n; ++j) { + w[j] = d->weight_[j]; + } } ++i; } @@ -974,8 +987,19 @@ void SaveState::restorenet() { NetCon* d = (NetCon*) ob->u.this_pointer; int n = ncs_[i].nstate; const double* w = ncs_[i].state; - for (int j = 0; j < n; ++j) { - d->weight_[j] = w[j]; + // Restore into heap buffer (nocmodl / FOR_NETCONS) and Weight SoA so HOC + // dual-write and materialize-on-deliver stay consistent. + if (d->weight_) { + for (int j = 0; j < n; ++j) { + d->weight_[j] = w[j]; + } + } + if (!d->weight_soa_.empty()) { + int const m = std::min(n, static_cast(d->weight_soa_.size())); + for (int j = 0; j < m; ++j) { + d->weight_soa_[j].value() = w[j]; + } + d->soa_sync(); // WeightIndex / reverse edges } ++i; } diff --git a/test/pytest/test_network_soa_delivery.py b/test/pytest/test_network_soa_delivery.py index b2d29e44c3..4aba836aac 100644 --- a/test/pytest/test_network_soa_delivery.py +++ b/test/pytest/test_network_soa_delivery.py @@ -68,3 +68,42 @@ def test_netcon_weight_hoc_roundtrip(): g_max = max(g_max, float(syn.g)) # Larger weight → larger g peak than default-zero would give assert g_max > 0.1, f"weight 0.12 should produce larger g, g_max={g_max}" + + +def test_savestate_weight_dualwrite_roundtrip(): + """SaveState restore must update Weight SoA, not only the weight_ heap.""" + s = h.Section(name="soma_ss") + s.insert("pas") + s.L = s.diam = 10 + syn = h.ExpSyn(s(0.5)) + syn.tau = 1 + syn.e = 0 + ns = h.NetStim() + ns.start = 1.0 + ns.number = 1 + ns.noise = 0 + nc = h.NetCon(ns, syn) + nc.weight[0] = 0.05 + nc.delay = 0.025 + + h.dt = 0.025 + h.finitialize(-65) + ss = h.SaveState() + ss.save() + + # Mutate after save (SoA-primary HOC path). + nc.weight[0] = 0.99 + assert abs(nc.weight[0] - 0.99) < 1e-12 + + ss.restore() + assert abs(nc.weight[0] - 0.05) < 1e-12, ( + f"SaveState restore should restore HOC/SoA weight, got {nc.weight[0]}" + ) + + # Delivery after restore must use restored weight (SoA materialize path). + g_max = 0.0 + while h.t < 3.0 - h.dt / 2: + h.fadvance() + g_max = max(g_max, float(syn.g)) + assert g_max > 0.04, f"restored weight 0.05 should produce ExpSyn g peak, g_max={g_max}" + assert g_max < 0.5, f"mutated weight 0.99 must not leak into delivery, g_max={g_max}" From 9b2c789ae0033ecc8a70d50de728606d2dd04688 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Tue, 21 Jul 2026 17:21:13 -0400 Subject: [PATCH 13/33] BBSaveState: SoA weight sync and SelfEvent NetCon-index binding Materialize Weight SoA into the heap before BBSaveState weight IO and mirror heap back to SoA after restore. Match SelfEvents to target NetCons by heap base or weight_index, and rebind both fields from the DEList ncindex on restore (same dual-write policy as SaveState). --- GROK-NETWORK-SOA.md | 15 +++--- doc/network-soa-phase0.md | 2 +- src/nrniv/bbsavestate.cpp | 67 +++++++++++++++++++++--- test/pytest/test_network_soa_delivery.py | 43 +++++++++++++++ 4 files changed, 113 insertions(+), 14 deletions(-) diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 61f61bbb02..a840b61dc1 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -16,6 +16,7 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s | **Sort wiring** | **Done** | Network in `nrn_ensure_model_data_are_sorted` (thread slices, weight repack) | | **Heap-drop policy** | **In progress** | Short-lived materialize on deliver; heap kept for FOR_NETCONS / INITIAL | | **SaveState dual-write** | **Done** | Weight save/restore via SoA; SelfEvent identity NetCon index + weight_index | +| **BBSaveState dual-write** | **Done** | Weight SoA sync; SelfEvent DEList ncindex + weight_index match | | **5** GPU net buffers | **Out of scope** | `local/gpu-native-qualification` after merge | **Dual-write complete for the CoreNEURON-shaped data plane.** Network SoA participates in the global sort gate. Legacy pointers/`dil_`/`weight_` heap remain for interpreter, queue, and nocmodl ABI where required. @@ -25,7 +26,7 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s - Long-lived `NetCon::weight_` heap still allocated; hot-path deliver uses **short-lived materialize** unless the target type has `FOR_NETCONS` (needs stable heap bases). Full drop needs SaveState index keys + INITIAL/FOR_NETCONS migration. - Default codegen is **nocmodl** (`NRN_ENABLE_NMODL=OFF`); native `weight_index` MOD ABI is a separate project. - `InputPreSyn` / thin remote gid→fanout deferred (see phase0 §5.5.1). -- BBSaveState still uses weight pointer equality for SelfEvent binding (Commit B). +- FOR_NETCONS / INITIAL still need long-lived `weight_` heap before full free. - Full `ctest -j N` needs a complete build/install; judge network work with filters below first. ### Default codegen @@ -106,7 +107,7 @@ Implementation: `src/nrncvode/network_soa_sort.cpp`, `neuron/container/network/s | NetCon deliver (FOR_NETCONS) | SoA → long-lived `weight_` heaps | MOD walks other NetCon `weight_` pointers | | `pnt_receive_init` / HOC INITIAL | heap buffer + SoA sync | Keep until INITIAL uses index | | SaveState | NetCon obj index + SoA weights | Dual-write restore sync done; `weight2netcon` remains live-queue helper | -| BBSaveState | heap pointer match for SelfEvent | Next: same NetCon-index policy as SaveState | +| BBSaveState | DEList ncindex + SoA weights | SelfEvent match heap or weight_index; bind both on restore | | HOC `weight[i]` | SoA `data_handle` | Already SoA-primary | **Do not free `weight_` until** FOR_NETCONS, SaveState, and INITIAL no longer need stable heap bases. Prefer short-lived materialize everywhere else (current default for simple deliver). @@ -118,7 +119,7 @@ Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep mate 1. ~~Wire network into `nrn_ensure_model_data_are_sorted`~~ (done). 2. ~~Heap-drop policy + short-lived materialize on simple deliver~~ (done; full free deferred). 3. ~~SaveState dual-write restore + SelfEvent NetCon-index identity~~ (done). -4. BBSaveState: SoA weight sync + SelfEvent NetCon-index binding (same policy as SaveState). +4. ~~BBSaveState SoA weight sync + SelfEvent NetCon-index binding~~ (done). 5. FOR_NETCONS / INITIAL without long-lived heap; then drop `weight_` allocation. 6. Merge toward master; GPU track rebases for Phase 5-style buffers. @@ -167,10 +168,10 @@ Read ~/neuron/cpu_net_soa/GROK-NETWORK-SOA.md, AGENTS.md, and doc/network-soa-ph Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa. Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU network buffers paused). -Phases 0–4 dual-write + network sort wiring + SaveState dual-write are done. -Heap-drop policy: short-lived materialize on simple deliver; keep weight_ -for FOR_NETCONS / INITIAL / BBSaveState. Next: BBSaveState NetCon-index -SelfEvent binding, or full heap free preconditions — not GPU on this branch. +Phases 0–4 dual-write, network sort wiring, SaveState + BBSaveState dual-write +are done. Heap-drop policy: short-lived materialize on simple deliver; keep +weight_ for FOR_NETCONS / INITIAL. Next: FOR_NETCONS/INITIAL without long-lived +heap, then drop weight_ — not GPU net_buf_receive on this branch. ``` --- diff --git a/doc/network-soa-phase0.md b/doc/network-soa-phase0.md index b7e953c13c..46062947c6 100644 --- a/doc/network-soa-phase0.md +++ b/doc/network-soa-phase0.md @@ -495,7 +495,7 @@ Phase 1 only needs M1 scaffolding (container + optional pointer into SoA). **No | `NetConSave` weight pointer tables | Live helper `weight2netcon(double*)` remains; SaveState SelfEvent **file** identity is NetCon obj index + `weight_index2netcon`. Call `NetConSave::invalid()` when heap bases reallocate. | | SaveState weight values | Save prefers Weight SoA; restore writes heap + SoA (`restorenet`). | | `PreSynSave` / `hi_index_` | Keep index tables; rebuild on unsorted. | -| BBSaveState | Still pointer-equality SelfEvent binding + heap weight IO — migrate like SaveState (Commit B). | +| BBSaveState | Weight SoA sync on IO; SelfEvent DEList `ncindex` + heap/`weight_index` match (Commit B done). | | Binary layout | SelfEvent line already stores `ncindex`; keep compatible. | Minimal Phase 1 impact: none if PointProcess/Weight SoA is not yet referenced by SaveState. diff --git a/src/nrniv/bbsavestate.cpp b/src/nrniv/bbsavestate.cpp index 81ded04c7e..f2dd5f7d2d 100644 --- a/src/nrniv/bbsavestate.cpp +++ b/src/nrniv/bbsavestate.cpp @@ -1107,13 +1107,57 @@ class SEWrap: public DiscreteEvent { double tt; int ncindex; // in the DEList or -1 if no NetCon for self event. }; + +/** True if SelfEvent is bound to this NetCon (heap base and/or Weight SoA index). */ +static bool selfevent_matches_netcon(SelfEvent const* se, NetCon const* nc) { + if (!se || !nc) { + return false; + } + // Legacy: long-lived heap base pointer identity. + if (se->weight_ && nc->weight_ && se->weight_ == nc->weight_) { + return true; + } + // Dual-write / heap-drop: Weight SoA base row (weight_index). + int se_widx = se->weight_index_; + if (se_widx < 0) { + return false; + } + int nc_widx = -1; + if (!nc->weight_soa_.empty()) { + nc_widx = static_cast(nc->weight_soa_.front().current_row()); + } else { + nc_widx = nc->_soa.weight_index(); + } + return nc_widx == se_widx; +} + +/** Bind SelfEvent weight_ / weight_index_ from a NetCon (after BBSaveState restore). */ +static void selfevent_bind_netcon(SelfEvent* se, NetCon* nc) { + if (!se) { + return; + } + if (!nc) { + se->weight_ = nullptr; + se->weight_index_ = -1; + return; + } + se->weight_ = nc->weight_; + if (!nc->weight_soa_.empty()) { + se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); + } else { + se->weight_index_ = nc->_soa.weight_index(); + } +} + SEWrap::SEWrap(const TQItem* tq, DEList* dl) { tt = tq->t_; se = (SelfEvent*) tq->data_; - if (se->weight_) { + // SelfEvent identity for BBSaveState: index into target's NetCon DEList + // (same policy as SaveState NetCon object index), not weight_* alone. + if (se->weight_ || se->weight_index_ >= 0) { ncindex = 0; for (; dl && dl->de && dl->de->type() == NetConType; dl = dl->next, ++ncindex) { - if (se->weight_ == ((NetCon*) dl->de)->weight_) { + if (selfevent_matches_netcon(se, static_cast(dl->de))) { return; } } @@ -2201,7 +2245,18 @@ void BBSaveState::netrecv_pp(Point_process* pp) { f->s(buf, 1); for (; dl && dl->de->type() == NetConType; dl = dl->next) { NetCon* nc = (NetCon*) dl->de; + // Dual-write: HOC weight[] is SoA-primary. Materialize SoA → heap before + // OUT/CNT so f->d sees current values; after IN, mirror heap → SoA. + if (f->type() != BBSS_IO::IN) { + nc->weights_soa_to_heap(); + } f->d(nc->cnt_, nc->weight_); + if (f->type() == BBSS_IO::IN) { + nc->weights_heap_to_soa(); + if (!nc->weight_soa_.empty()) { + nc->soa_sync(); + } + } if (f->type() != BBSS_IO::IN) { // writing, counting DblList* db = 0; int j = 0; @@ -2277,7 +2332,7 @@ void BBSaveState::netrecv_pp(Point_process* pp) { // since the queue has been cleared. for (int i = 0; i < cnt; ++i) { int ncindex, moff; - double flag, tt, *w; + double flag, tt; f->s(buf); f->d(1, flag); f->d(1, tt); @@ -2302,16 +2357,16 @@ void BBSaveState::netrecv_pp(Point_process* pp) { se->movable_ = movable; } if (ncindex == -1) { - w = NULL; + selfevent_bind_netcon(se, nullptr); } else { int j; for (j = 0, dl1 = dliter->second; j < ncindex; ++j, dl1 = dl1->next) { ; } assert(dl1 && dl1->de->type() == NetConType); - w = ((NetCon*) dl1->de)->weight_; + // ncindex is position in target DEList (file identity), not weight_*. + selfevent_bind_netcon(se, static_cast(dl1->de)); } - se->weight_ = w; } } if (debug) { diff --git a/test/pytest/test_network_soa_delivery.py b/test/pytest/test_network_soa_delivery.py index 4aba836aac..4f97b88003 100644 --- a/test/pytest/test_network_soa_delivery.py +++ b/test/pytest/test_network_soa_delivery.py @@ -107,3 +107,46 @@ def test_savestate_weight_dualwrite_roundtrip(): g_max = max(g_max, float(syn.g)) assert g_max > 0.04, f"restored weight 0.05 should produce ExpSyn g peak, g_max={g_max}" assert g_max < 0.5, f"mutated weight 0.99 must not leak into delivery, g_max={g_max}" + + +def test_bbsavestate_weight_dualwrite_roundtrip(tmp_path): + """BBSaveState restore must update Weight SoA (NetCon on a gid cell).""" + # BBSaveState requires a real cell with a gid (section parented to a cell object). + class Cell: + def __init__(self): + self.s = h.Section("soma", self) + self.s.insert("pas") + self.s.L = self.s.diam = 10 + self.syn = h.ExpSyn(self.s(0.5)) + self.syn.tau = 1 + self.syn.e = 0 + + pc = h.ParallelContext() + cell = Cell() + ns = h.NetStim() + ns.start = 1.0 + ns.number = 1 + ns.noise = 0 + nc = h.NetCon(ns, cell.syn) + nc.weight[0] = 0.05 + nc.delay = 0.025 + + gid = 7 + pc.set_gid2node(gid, pc.id()) + pc.cell(gid, h.NetCon(cell.s(0.5)._ref_v, None, sec=cell.s)) + + h.dt = 0.025 + h.finitialize(-65) + path = str(tmp_path / "bbss_weights.bbss") + bbss = h.BBSaveState() + bbss.save(path) + + nc.weight[0] = 0.99 + assert abs(nc.weight[0] - 0.99) < 1e-12 + + bbss.restore(path) + assert abs(nc.weight[0] - 0.05) < 1e-12, ( + f"BBSaveState restore should restore HOC/SoA weight, got {nc.weight[0]}" + ) + + pc.gid_clear() From 5de433cc7db9341e02106b616c228ae4d9d85fec Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Tue, 21 Jul 2026 18:23:22 -0400 Subject: [PATCH 14/33] Fix network sort PP sync and keep long-lived weight_ for net_send sort_network_data must resolve HOC point-process objects via ob2pntproc_0 (dataspace), not this_pointer. NetCon deliver must pass the long-lived weight_ heap into pnt_receive so MOD net_send(..., _w) preserves SelfEvent identity for SaveState; resolve owner by weight_index when needed. Softens SelfEvent savestate_write when no NetCon is found. --- GROK-NETWORK-SOA.md | 4 +- src/nrncvode/netcvode.cpp | 66 +++++++++++++++++++------------ src/nrncvode/network_soa_sort.cpp | 9 ++++- 3 files changed, 49 insertions(+), 30 deletions(-) diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index a840b61dc1..c95c7393f7 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -103,8 +103,8 @@ Implementation: `src/nrncvode/network_soa_sort.cpp`, `neuron/container/network/s | Path | Weight source | Notes | |------|---------------|--------| -| NetCon deliver (no FOR_NETCONS) | SoA → **temp buffer** → `pnt_receive` | Hot path; heap not required for content | -| NetCon deliver (FOR_NETCONS) | SoA → long-lived `weight_` heaps | MOD walks other NetCon `weight_` pointers | +| NetCon deliver | SoA → long-lived `weight_` → `pnt_receive` → SoA | Heap is MOD scratch; **required** so `net_send(..., _w)` keeps SelfEvent identity | +| NetCon deliver (FOR_NETCONS) | Same + sync all NetCons on target | MOD walks other NetCon `weight_` pointers | | `pnt_receive_init` / HOC INITIAL | heap buffer + SoA sync | Keep until INITIAL uses index | | SaveState | NetCon obj index + SoA weights | Dual-write restore sync done; `weight2netcon` remains live-queue helper | | BBSaveState | DEList ncindex + SoA weights | SelfEvent match heap or weight_index; bind both on restore | diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 6cb829c66e..8439915b94 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2338,25 +2338,36 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, int const n = pnt_receive_size[type]; bool const fornet = type_has_fornetcon(type); - // Prefer NetCon dual-write owners when heap base is a known NetCon weight_. + // Prefer a long-lived NetCon weight_ heap buffer for pnt_receive. Temporary + // materialize buffers are unsafe: nocmodl NET_RECEIVE often does + // net_send(..., _w, ...) and SelfEvent must keep a stable weight_ identity + // for SaveState/BBSaveState (weight2netcon). + NetCon* owner = nullptr; if (weight_heap) { - if (NetCon* nc = NetConSave::weight2netcon(weight_heap)) { - if (fornet) { - // FOR_NETCONS walks weight_ of all NetCons with this target. - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); - } else { - nc->weights_soa_to_heap(); - } - POINT_RECEIVE(type, pnt, weight_heap, flag); - if (fornet) { - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); - } else { - nc->weights_heap_to_soa(); - } - return; + owner = NetConSave::weight2netcon(weight_heap); + } + if (!owner && weight_index >= 0) { + owner = NetConSave::weight_index2netcon(weight_index); + if (owner && owner->weight_) { + weight_heap = owner->weight_; } } - // Index-only path: materialize a temporary (or fill provided heap) from Weight SoA. + if (owner) { + if (fornet) { + // FOR_NETCONS walks weight_ of all NetCons with this target. + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); + } else { + owner->weights_soa_to_heap(); + } + POINT_RECEIVE(type, pnt, weight_heap ? weight_heap : owner->weight_, flag); + if (fornet) { + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); + } else { + owner->weights_heap_to_soa(); + } + return; + } + // No NetCon owner (null-weight / flag-only path): short-lived buffer OK. if (fornet) { sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); } @@ -3114,16 +3125,15 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); - // Phase 4 + heap-drop policy: prefer weight_index → short-lived materialize. - // Pass long-lived weight_ heap only when FOR_NETCONS needs stable bases for all - // NetCons on this target; otherwise a temporary buffer is enough for nocmodl ABI. + // Phase 4: deliver via weight_index; use long-lived weight_ heap as MOD + // buffer so net_send(..., _w, ...) keeps a stable SelfEvent identity. + // SoA remains HOC-primary via materialize around pnt_receive. int widx = _soa.weight_index(); if (widx < 0 && !weight_soa_.empty()) { widx = static_cast(weight_soa_.front().current_row()); } if (widx >= 0) { - double* heap_for_abi = type_has_fornetcon(type) ? weight_ : nullptr; - nrn_pnt_receive_by_weight_index(target_, widx, 0., heap_for_abi); + nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); } else { weights_soa_to_heap(); POINT_RECEIVE(type, target_, weight_, 0); @@ -3149,8 +3159,7 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { widx = static_cast(weight_soa_.front().current_row()); } if (widx >= 0) { - double* heap_for_abi = type_has_fornetcon(type) ? weight_ : nullptr; - nrn_pnt_receive_by_weight_index(target_, widx, 0., heap_for_abi); + nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); } else { weights_soa_to_heap(); POINT_RECEIVE(type, target_, weight_, 0); @@ -3489,16 +3498,21 @@ void SelfEvent::savestate_write(FILE* f) { int const moff = movable_ ? (movable_ - target_->prop->dparam) : -1; int ncindex = -1; // SelfEvent identity for SaveState: NetCon object index (stable), not weight_*. - // Prefer heap map when present; fall back to Weight SoA weight_index. + // Prefer heap map; fall back to weight_index. Do not assert if unresolved — + // flag-only / ignored events use ncindex = -1 (BBSaveState uses -2 for ignore). NetCon* owner = nullptr; if (weight_) { owner = NetConSave::weight2netcon(weight_); - assert(owner); - } else if (weight_index_ >= 0) { + } + if (!owner && weight_index_ >= 0) { owner = NetConSave::weight_index2netcon(weight_index_); } if (owner && owner->obj_) { ncindex = owner->obj_->index; + // Rebind to long-lived heap if we resolved via index only. + if (!weight_ && owner->weight_) { + weight_ = owner->weight_; + } } fprintf(f, diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index 209f651e80..293cda2ded 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -6,6 +6,7 @@ #include "neuron/container/network/sort.hpp" #include "netcon.h" #include "netcvode.h" +#include "nrniv_mf.h" #include "section.h" #include @@ -34,8 +35,12 @@ void sync_all_point_processes() { } hoc_Item* q = nullptr; ITERATE(q, tmp->olist) { - auto* pnt = static_cast(OBJ(q)->u.this_pointer); - nrn_point_process_soa_sync(pnt); + // HOC point-process templates store Point_process* in dataspace + // (not this_pointer); use ob2pntproc_0 for both C and HOC shells. + Point_process* pnt = ob2pntproc_0(OBJ(q)); + if (pnt) { + nrn_point_process_soa_sync(pnt); + } } } } From 415a51ff088f8aabb9e466a3d95b9119b22ec2fa Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Wed, 22 Jul 2026 07:07:41 -0400 Subject: [PATCH 15/33] Document network SoA topology, heap policy, and sort (layered docs) Add doc/network-soa/ with L0 README, topology (fanout authority, CoreNEURON mapping), dual-write/heap roadmap, and sort packing. Point GROK handoff at the layered map and correct stale short-lived-materialize status text. --- GROK-NETWORK-SOA.md | 49 ++++---- doc/network-soa/README.md | 62 ++++++++++ doc/network-soa/dual-write-and-heap.md | 59 +++++++++ doc/network-soa/sort-and-packing.md | 33 +++++ doc/network-soa/topology.md | 159 +++++++++++++++++++++++++ 5 files changed, 336 insertions(+), 26 deletions(-) create mode 100644 doc/network-soa/README.md create mode 100644 doc/network-soa/dual-write-and-heap.md create mode 100644 doc/network-soa/sort-and-packing.md create mode 100644 doc/network-soa/topology.md diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index c95c7393f7..8df713bed6 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -8,26 +8,24 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s | Phase | Status | Notes | |-------|--------|--------| -| **0** Spec | **Done** | `doc/network-soa-phase0.md` authoritative | +| **0** Spec | **Done** | `doc/network-soa-phase0.md` + layered `doc/network-soa/` | | **1** PointProcess + Weight SoA dual-write | **Done** | Handles + create/destroy | -| **2** NetCon SoA; HOC weight → Weight SoA | **Done** | Delivery materializes `double*` for MOD | -| **3** PreSyn SoA + fanout `NcIndex`/`NcCount` | **Done** | Rebuild at `init_events` + lazy on spike | -| **4** SelfEvent `weight_index` + receive-by-index | **Done** | M2: still `pnt_receive(..., double*, flag)` | -| **Sort wiring** | **Done** | Network in `nrn_ensure_model_data_are_sorted` (thread slices, weight repack) | -| **Heap-drop policy** | **In progress** | Short-lived materialize on deliver; heap kept for FOR_NETCONS / INITIAL | -| **SaveState dual-write** | **Done** | Weight save/restore via SoA; SelfEvent identity NetCon index + weight_index | -| **BBSaveState dual-write** | **Done** | Weight SoA sync; SelfEvent DEList ncindex + weight_index match | -| **5** GPU net buffers | **Out of scope** | `local/gpu-native-qualification` after merge | +| **2** NetCon SoA; HOC weight → Weight SoA | **Done** | SoA HOC-primary; heap MOD scratch | +| **3** PreSyn SoA + fanout `NcIndex`/`NcCount` | **Done** | Rebuild at sort / `init_events` + lazy on spike | +| **4** SelfEvent `weight_index` + receive-by-index | **Done** | Still `pnt_receive(..., double*, flag)` (nocmodl) | +| **Sort wiring** | **Done** | Network in `nrn_ensure_model_data_are_sorted` | +| **Heap policy** | **Settled (no full free yet)** | Long-lived `weight_` required for `net_send(_w)` / FOR_NETCONS / INITIAL | +| **SaveState / BBSaveState dual-write** | **Done** | SoA values + NetCon-index SelfEvent identity | +| **5** GPU net buffers | **Out of scope** | After SoA is reviewable / on master | -**Dual-write complete for the CoreNEURON-shaped data plane.** Network SoA participates in the global sort gate. Legacy pointers/`dil_`/`weight_` heap remain for interpreter, queue, and nocmodl ABI where required. +**Developer docs (preferred reading order):** `doc/network-soa/README.md` → topology → dual-write → sort → phase0. -### Known gaps (do not “just drop heap” yet) +### Known gaps -- Long-lived `NetCon::weight_` heap still allocated; hot-path deliver uses **short-lived materialize** unless the target type has `FOR_NETCONS` (needs stable heap bases). Full drop needs SaveState index keys + INITIAL/FOR_NETCONS migration. -- Default codegen is **nocmodl** (`NRN_ENABLE_NMODL=OFF`); native `weight_index` MOD ABI is a separate project. -- `InputPreSyn` / thin remote gid→fanout deferred (see phase0 §5.5.1). -- FOR_NETCONS / INITIAL still need long-lived `weight_` heap before full free. -- Full `ctest -j N` needs a complete build/install; judge network work with filters below first. +- Full free of `NetCon::weight_`: needs FOR_NETCONS + INITIAL + SelfEvent never keying on temp `double*` (see `doc/network-soa/dual-write-and-heap.md`). +- Default codegen **nocmodl**; CoreNEURON-style `weight_index` receive is a separate ABI project. +- Fanout: SoA ranges preferred when sorted; `dil_` still filled and used as rebuild source / fallback (**fanout authority** not fully flipped — see topology doc). +- `InputPreSyn` thin gid→fanout deferred (phase0 §5.5.1). ### Default codegen @@ -116,12 +114,11 @@ Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep mate ## Recommended next work -1. ~~Wire network into `nrn_ensure_model_data_are_sorted`~~ (done). -2. ~~Heap-drop policy + short-lived materialize on simple deliver~~ (done; full free deferred). -3. ~~SaveState dual-write restore + SelfEvent NetCon-index identity~~ (done). -4. ~~BBSaveState SoA weight sync + SelfEvent NetCon-index binding~~ (done). -5. FOR_NETCONS / INITIAL without long-lived heap; then drop `weight_` allocation. -6. Merge toward master; GPU track rebases for Phase 5-style buffers. +1. ~~Sort + SaveState + BBSaveState dual-write~~ (done). +2. **Decide PR posture:** draft/review PR of dual-write+sort (no claim “land for perf yet”) vs private until heap-free / GPU (see `doc/network-soa/dual-write-and-heap.md` §5). +3. If pursuing heap-free: FOR_NETCONS index view + INITIAL by index + SelfEvent identity without heap pointer (doc L2 roadmap). +4. If pursuing authority cleanup: make fanout SoA sole hot-path truth; stop using `dil_` on spike (topology L1). +5. GPU Phase 5 only after SoA shape is stable enough to upload the same columns. --- @@ -168,10 +165,10 @@ Read ~/neuron/cpu_net_soa/GROK-NETWORK-SOA.md, AGENTS.md, and doc/network-soa-ph Repo: ~/neuron/cpu_net_soa, branch local/cpu-network-soa. Sibling: ~/neuron/nrngpu @ local/gpu-native-qualification (GPU network buffers paused). -Phases 0–4 dual-write, network sort wiring, SaveState + BBSaveState dual-write -are done. Heap-drop policy: short-lived materialize on simple deliver; keep -weight_ for FOR_NETCONS / INITIAL. Next: FOR_NETCONS/INITIAL without long-lived -heap, then drop weight_ — not GPU net_buf_receive on this branch. +Phases 0–4 dual-write, sort wiring, SaveState/BBSaveState dual-write done. +Read doc/network-soa/README.md (topology, heap policy, sort). weight_ remains +MOD scratch under nocmodl. Next: PR posture, or FOR_NETCONS/fanout-authority +work toward heap-free — not GPU on this branch unless rebasing after SoA. ``` --- diff --git a/doc/network-soa/README.md b/doc/network-soa/README.md new file mode 100644 index 0000000000..b25a3f34f9 --- /dev/null +++ b/doc/network-soa/README.md @@ -0,0 +1,62 @@ +# Network SoA — developer docs (map) + +**Audience:** people working on `local/cpu-network-soa` +**Status tracker / session handoff:** `GROK-NETWORK-SOA.md` (repo root) +**Phase-0 field tags & checklist:** `doc/network-soa-phase0.md` (detailed, longer) + +Docs here are **segregated by height**. Prefer the shortest document that answers your question. + +| Level | Doc | Length / purpose | +|-------|-----|------------------| +| **L0 — North star** | this file §North star | One screen: goals, non-goals, current posture | +| **L1 — Topology** | [topology.md](topology.md) | PreSyn → NetCon → Point_process; CoreNEURON mapping; fanout authority; rank/thread placement | +| **L2 — Dual-write & heap** | [dual-write-and-heap.md](dual-write-and-heap.md) | What is primary where; why `weight_` still exists; FOR_NETCONS / INITIAL / SaveState | +| **L3 — Sort & packing** | [sort-and-packing.md](sort-and-packing.md) | What `nrn_ensure_model_data_are_sorted` does to network containers | +| **L4 — Spec detail** | `../network-soa-phase0.md` | Field tags, sidecars, SaveState notes, open questions resolved | +| **Ops** | `../../GROK-NETWORK-SOA.md` | Build/test gates, commit status, “what next” | + +--- + +## North star (L0) + +**Goal:** One CPU backing store for *integration-hot* network data (columns + indices), with HOC wrappers as permutation-stable handles — same *spirit* as node/mechanism SoA and CoreNEURON’s per-thread flat arrays. + +**Integration reference:** CoreNEURON (`NrnThread::{pntprocs,netcons,weights,presyns}`, `PreSyn::{nc_index_,nc_cnt_}`, `NetCon::u.weight_index_`). + +**Not the first PR’s job (unless we change strategy):** + +- Full free of `NetCon::weight_` under default **nocmodl** ABI +- GPU net buffers (sibling track after SoA is usable) +- Thin remote `InputPreSyn` / gid→fanout table + +**Current posture (2026-07):** Dual-write complete for CoreNEURON-shaped columns; network participates in global sort; SaveState/BBSaveState dual-write; **`weight_` remains long-lived MOD scratch** because generated `pnt_receive` / `net_send(..., _w)` still use `double*`. + +**PR vs master:** A PR can document progress and gather review without claiming “land now.” Landing criteria (performance/space *and* preferably a path to GPU) are product decisions separate from “is the dual-write design sound?” + +--- + +## Two graphs (memorize this) + +```text +HOT PATH (spike → delivery) — CoreNEURON shape, indices: + PreSyn ──(NcIndex,NcCount)──► ordered NetCon fanout + NetCon ──Target──► PointProcess row + NetCon ──WeightIndex──► contiguous weight block + +MANAGEMENT / HOC (interactive NEURON) — pointers/handles still exist in dual-write: + PreSyn.dil_ / NetCon.src_ / NetCon.target_ / Object* +``` + +**Fanout authority** = which of the two is allowed to decide “who receives this spike” on the hot path. See [topology.md](topology.md). + +--- + +## Related code + +| Concern | Location | +|---------|----------| +| Containers | `src/neuron/container/network/*.hpp` | +| Sort / repack | `src/nrncvode/network_soa_sort.cpp` | +| Fanout dual-write | `PreSyn::ensure_fanout_order` in `netcvode.cpp` | +| Receive by index | `nrn_pnt_receive_by_weight_index` | +| CoreNEURON layouts | `src/coreneuron/sim/multicore.hpp`, `network/netcon.hpp` | diff --git a/doc/network-soa/dual-write-and-heap.md b/doc/network-soa/dual-write-and-heap.md new file mode 100644 index 0000000000..379fbd71a3 --- /dev/null +++ b/doc/network-soa/dual-write-and-heap.md @@ -0,0 +1,59 @@ +# L2 — Dual-write and the weight heap + +Brevity: medium. Topology: [topology.md](topology.md). + +--- + +## 1. Dual-write rule of thumb + +| Plane | Primary for… | Examples | +|-------|----------------|----------| +| **Weight SoA** | HOC `weight[]`, SaveState values, future GPU upload | `weight_soa_`, `Model::weights()` | +| **`weight_` heap** | Generated MOD `double*` ABI, FOR_NETCONS walks, `net_send` capturing `_w` | `NetCon::weight_` | +| **Pointers (`target_`, `src_`, `dil_`)** | Queue / HOC / disconnect until dual-read done | Legacy shells | +| **Indices (`Target`, `WeightIndex`, `NcIndex`)** | CoreNEURON-shaped hot path once dual-read | SoA columns | + +**Invariant:** after any path that mutates weights for HOC visibility, SoA and heap agree *or* the next deliver materializes SoA→heap before MOD runs and heap→SoA after. + +--- + +## 2. Why “short-lived materialize only” failed + +Passing a **temporary** buffer into `pnt_receive` broke SelfEvent identity: nocmodl does `net_send(..., _w, ...)` and SaveState keyed SelfEvents by that pointer (`weight2netcon`). + +**Current policy:** long-lived `weight_` is **MOD scratch** (stable identity for `net_send`). SoA remains HOC-primary; deliver does SoA→heap→MOD→heap→SoA (or equivalent owner path in `nrn_pnt_receive_by_weight_index`). + +--- + +## 3. SaveState / BBSaveState (done on this branch) + +- **Values:** save prefer SoA; restore write heap **and** SoA. +- **SelfEvent identity:** NetCon object index / DEList `ncindex` + `weight_index`, not sole reliance on heap pointer (pointer still works as fallback while heap lives). + +--- + +## 4. Roadmap to full heap free (if chosen) + +Rough commit series (each gateable): + +1. **SelfEvent never stores identity-only-as-temp-pointer** — always set `weight_index_` (done in dual-write spirit); never pass non-owner buffers into `pnt_receive` if MOD can net_send. +2. **FOR_NETCONS** — either CoreNEURON-style weight index perm, or generate index-based FOR_NETCONS; stop requiring foreign NetCon `weight_` bases. +3. **INITIAL** — `pnt_receive_init` by index or materialize into owner heap only. +4. **Stop allocating `new double[cnt_]`** — optional thread-local scratch of `max(pnt_receive_size)` for MOD only if ABI still needs `double*`. +5. **Delete `weight_` field** — after (1–4) and tests (stdp/FOR_NETCONS, SaveState ring, netrec init). + +Until (2)–(3), “full drop” is premature. + +--- + +## 5. PR strategy (honest) + +| Option | Pros | Cons | +|--------|------|------| +| **PR series without heap free** | Reviewable design; CI; early feedback; doesn’t pretend GPU is done | Master doesn’t get space win yet | +| **Hold until heap free** | Stronger “finished dual-write” story | Large blocked-on-codegen risk; long private branch | +| **Hold until GPU** | End-to-end performance story | Couples orthogonal tracks; slowest | + +Recommendation: **document dual-write+sort as a reviewable PR (or draft PR), keep landing bar explicit** (“not for merge until X”). Heap-free and GPU as **follow-on PR chains**, not prerequisites for *having* a PR. + +If the only merge criterion is measured space/time, then next engineering is **benchmarks** (ringtest CPU memory + spike path) *and* heap-free or GPU — not more silent dual-write surface area. diff --git a/doc/network-soa/sort-and-packing.md b/doc/network-soa/sort-and-packing.md new file mode 100644 index 0000000000..38ff3feed4 --- /dev/null +++ b/doc/network-soa/sort-and-packing.md @@ -0,0 +1,33 @@ +# L3 — Sort and packing + +Brevity: short–medium. Spec detail: phase0 §6, §8. + +--- + +## When + +`nrn_ensure_model_data_are_sorted()` after nodes + mechanisms. Network containers issue frozen tokens; if any unsorted → `sort_network_data`. + +## Order of work + +1. Sync dual-write shells → SoA (`ob2pntproc_0` for HOC PP; `soa_sync` NetCon/PreSyn). +2. **PointProcess** — reverse-permute by `ThreadId`; set `cache.thread[i].point_process_offset`. +3. **Weight** — repack so each NetCon’s `weight_soa_` rows are contiguous; NetCons ordered by (target thread, src PreSyn); set `weight_offset`. +4. **NetCon** SoA — same order as weight packing; set `netcon_offset`; refresh indices. +5. **PreSyn** — by thread; set `presyn_offset`; rebuild fanout (`NcIndex`/`NcCount` + order table). + +## Why this order + +- Target-thread packing matches CoreNEURON **per-thread weight/netcon pools**. +- Fanout rebuild **after** PreSyn/NetCon permutes so ranges match current rows. +- Weight contiguity is required for `weight_index` base + `count` semantics (CoreNEURON). + +## What sort does *not* claim + +- Fanout physical layout ≡ NetCon SoA order (may use separate order array). +- FOR_NETCONS adjacency (separate perm, CoreNEURON-style, still TODO for index path). +- Freeing `weight_` heap. + +## Implementation + +`src/nrncvode/network_soa_sort.cpp`, header `neuron/container/network/sort.hpp`. diff --git a/doc/network-soa/topology.md b/doc/network-soa/topology.md new file mode 100644 index 0000000000..b751e6a373 --- /dev/null +++ b/doc/network-soa/topology.md @@ -0,0 +1,159 @@ +# L1 — Topology: PreSyn → NetCon → Point_process + +Brevity: medium. For field tags see `doc/network-soa-phase0.md` §5. + +--- + +## 1. What “fanout authority” means + +On a spike, NEURON must enumerate the NetCons that fire from a given source. + +| Representation | Role historically | Authority? | +|----------------|-------------------|------------| +| `PreSyn::dil_` (`vector`) | Interactive NEURON; append on connect; disconnect walks this | **Was** authority | +| `PreSyn` SoA `NcIndex`/`NcCount` + global order (`g_network_fanout_order` or, later, NetCon SoA range) | CoreNEURON-shaped fanout | **Should be** authority on hot path after dual-read | + +**Fanout authority** = the structure the spike path *must* trust. +Management code (connect, disconnect, HOC inspect) may still *write* `dil_` during dual-write, but must not leave delivery depending on a second, divergent truth. + +Today (dual-write): + +- Connect still fills `dil_`. +- `ensure_fanout_order()` rebuilds order + `NcIndex`/`NcCount` from `dil_` (and at sort / `init_events`). +- Spike path prefers SoA range when sorted, else falls back to `dil_`. + +**Target end state:** rebuild ranges from connectivity at sort (or from a single edge table); `dil_` becomes optional sidecar or dies. Delivery never walks `dil_`. + +That is the same *idea* as CoreNEURON: `nc_index_` / `nc_cnt_` into `netcon_in_presyn_order_[]`, not a per-PreSyn pointer vector. + +--- + +## 2. CoreNEURON layout (what to keep in mind) + +Per **thread** (after transfer), CoreNEURON holds flat arrays: + +```text +nt.pntprocs[n_pntproc] // Point_process { _type, _i_instance, _tid, ... } +nt.netcons[n_netcon] // NetCon { active, delay, target_*, weight_index } +nt.weights[n_weight] // flat double pool +nt.presyns[n_presyn] // local/output PreSyn { threshold, gid, nc_index, nc_cnt, ... } +// plus InputPreSyn for remote gid→fanout only +``` + +### Spike fanout (source-centric) + +```text +PreSyn.nc_index_ / nc_cnt_ + → netcon_in_presyn_order_[nc_index + i] // NetCon* (setup), or index into nt.netcons +``` + +Source is **implicit**: NetCons in that range belong to this PreSyn. There is **no** hot-path `src_` on NetCon. + +### Delivery (target-centric data, not fanout) + +```text +NetCon.target_ → Point_process* into nt.pntprocs +NetCon.u.weight_index_ → base into nt.weights[] +pnt_receive(target, weight_index, flag) // CoreNEURON: index, not double* +``` + +Weight **block length** is fixed by the target mechanism’s `NET_RECEIVE` arity (`pnt_receive_size[type]`). CoreNEURON does **not** store weight count on NetCon in the hot struct the same way; the size is known from `target_->_type`. + +### FOR_NETCONS (target-centric *permutation*) + +NetCons that share a target are **not** adjacent in construction order. CoreNEURON builds: + +- `_fornetcon_weight_perm` — indices into `weights` so one target’s NetCon weight groups become adjacent in *index space* +- `_fornetcon_perm_indices` — displacement into that perm for each target instance + +So: **fanout order is by source PreSyn; FOR_NETCONS order is a separate target-grouped view of weight indices.** Two orderings, one weight pool. + +### Rank / thread placement (your intuition) + +| Constraint | CoreNEURON / multicore NEURON | +|------------|-------------------------------| +| NetCon target PP must be local to the rank that owns the synapse | Yes — remote connectivity is **gid → InputPreSyn fanout → local NetCons**, not a NetCon with a remote target pointer | +| NetCon delivery thread = target PP’s thread | Yes — `PreSyn::send` enqueues on `PP2NT(target)`; cross-thread only as interthread event | +| Weights live with **target** thread’s pool | Yes — `nt.weights` is per-thread (CoreNEURON) | + +**NetCon is “about” the target PP** for: + +- weight arity (`NET_RECEIVE` args) +- delivery locality (same rank; same thread as target) +- `pnt_receive` / FOR_NETCONS + +**NetCon is “about” the source PreSyn** only for: + +- *who spikes it into existence* (fanout membership) +- delay / active on that edge + +So both of these are true: + +1. **Data plane packing:** prefer **target-thread** slices (weights, NetCon rows that deliver on that thread, PointProcess rows). +2. **Spike enumeration:** **source-centric** ranges (`NcIndex`/`NcCount`). + +NEURON’s global SoA + **thread partition by permutation** (like nodes/mechs) is the elegant way to get (1) without per-thread heap graphs. Fanout ranges (2) are a *logical* order that may be a separate index array (CoreNEURON’s `netcon_in_presyn_order_`) even if NetCon SoA rows are packed by target thread. + +--- + +## 3. How dual-write NEURON maps today + +| CoreNEURON | NEURON dual-write | +|------------|-------------------| +| `nt.pntprocs[]` | `Model::point_processes()` + legacy `Point_process*` shell | +| `nt.netcons[]` | `Model::netcons()` + `NetCon` DiscreteEvent shell | +| `nt.weights[]` | `Model::weights()` + long-lived `NetCon::weight_` MOD scratch | +| `nt.presyns[]` | `Model::presyns()` + `PreSyn` shell + `dil_` | +| `nc_index_/nc_cnt_` | SoA fields + `g_network_fanout_order` (NetCon* table) | +| `weight_index_` | SoA `WeightIndex` (+ `weight_soa_.front().current_row()`) | +| `pnt_receive(..., weight_index, flag)` | `nrn_pnt_receive_by_weight_index` → still calls nocmodl `double*` form | + +**Not yet CoreNEURON-true:** + +- NetCon SoA row order is not required to equal fanout order (we have a **separate** fanout pointer table). +- `pnt_receive` still takes `double*` (nocmodl). +- `weight_` heap still allocated per NetCon. +- Remote sources still fat PreSyn in `gid2in_` (InputPreSyn role deferred). + +--- + +## 4. Ordering arrangements (summary table) + +| Order | Key | Used for | +|-------|-----|----------| +| **Thread pack** | `thread_id` of target PP (weights/NetCons/PPs); PreSyn by its thread | Cache locality, future GPU upload slices, same as node/mech sort | +| **Spike fanout** | Source PreSyn (contiguous range) | `PreSyn::send` / deliver fanout | +| **FOR_NETCONS** | Target PP instance (perm of weight bases) | CoreNEURON `_fornetcon_*`; NEURON still walks heaps/pointers today | +| **HOC construction** | Object create order | SaveState lists, some BBSaveState assumptions — **not** hot-path locality | + +Sort wiring today: partition PP/PreSyn by thread; pack weight blocks by NetCon ordered (target thread, src PreSyn); rebuild fanout ranges. That is deliberately **compatible** with CoreNEURON’s split between target-thread pools and source-centric fanout indices. + +--- + +## 5. Implications for “full heap drop” + +Heap `weight_` is **not** required by topology. It is required by: + +1. **nocmodl ABI** — `pnt_receive(Point_process*, double*, flag)` and `net_send` capturing `_w` as SelfEvent identity. +2. **FOR_NETCONS** — generated code walks other NetCons’ `weight_` bases (or needs a CoreNEURON-like weight perm). +3. **INITIAL** — `pnt_receive_init(..., double*, ...)`. + +CoreNEURON already dropped (1) by generating `weight_index` receive. NEURON default builds use nocmodl, so (1) is a **codegen/ABI** project unless we keep a permanent materialize shim (SoA → scratch buffer that is *not* per-NetCon heap — e.g. thread-local scratch of max arity, *if* SelfEvent identity is always `weight_index` / NetCon id, never the scratch pointer). + +That last sentence is the real fork for “full drop without full GPU”: + +- **Path H (heap-free, nocmodl kept):** SelfEvent and FOR_NETCONS never store/compare `double* weight_`; only indices; scratch is ephemeral and never identity. +- **Path C (CoreNEURON ABI):** change generated receive to `weight_index` (nocmodl or NMODL project). + +Either can be a PR series; neither is required to *open* a design/review PR of dual-write+sort. + +--- + +## 6. Discussion anchors (for back-and-forth) + +1. **Should fanout order array stay `NetCon*` or become integer NetCon SoA rows?** + Integers match GPU/CoreNEURON export; pointers ease dual-write with DiscreteEvent queue. +2. **Should NetCon SoA physical order follow target-thread, source, or construction?** + Recommendation: **target-thread primary** (weights co-located); fanout stays a separate index range (CoreNEURON pattern). +3. **Is rank-local target enough, or must we also assert same-thread at connect?** + NEURON already delivers to `PP2NT(target)`; cross-thread NetCon is allowed as interthread send (same as CoreNEURON). From d8911a3087d661a18484ec45592f1d267b67c554 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Wed, 22 Jul 2026 21:14:11 -0400 Subject: [PATCH 16/33] Fix CoreNEURON weight dual-write transfer and CI formatting Sync Weight SoA when exporting NetCon weights to CoreNEURON and when returning weights after a CoreNEURON run (HOC weight[] is SoA-primary). Apply black/clang-format line-wrap fixes for network SoA tests. Merge with master already on branch for formatting toolchain updates. --- .../callbacks/nrncore_callbacks.cpp | 4 +++ test/pytest/test_network_soa_delivery.py | 26 ++++++++++++------- test/unit_tests/container/network.cpp | 5 ++-- 3 files changed, 24 insertions(+), 11 deletions(-) diff --git a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp index 169ef8ba1f..64e10d02ef 100644 --- a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp +++ b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp @@ -143,6 +143,8 @@ void nrnthreads_all_weights_return(std::vector& weights) { for (int i = 0; i < nc->cnt_; ++i) { nc->weight_[i] = weights[ith][iw[ith]++]; } + // Dual-write: HOC weight[] is SoA-primary; keep Weight SoA in sync. + nc->weights_heap_to_soa(); } } @@ -508,6 +510,8 @@ int nrnthread_dat2_3(int tid, int iw = 0; for (int i = 0; i < n; ++i) { NetCon* nc = cg.netcons[i]; + // Dual-write: HOC may have written Weight SoA only; materialize heap. + nc->weights_soa_to_heap(); for (int j = 0; j < nc->cnt_; ++j) { weights[iw++] = nc->weight_[j]; } diff --git a/test/pytest/test_network_soa_delivery.py b/test/pytest/test_network_soa_delivery.py index 4f97b88003..28c8473425 100644 --- a/test/pytest/test_network_soa_delivery.py +++ b/test/pytest/test_network_soa_delivery.py @@ -6,6 +6,7 @@ - nrn_pnt_receive_by_weight_index around pnt_receive - PreSyn fanout order rebuild at init_events """ + from neuron import h @@ -42,7 +43,9 @@ def test_netstim_expsyn_delivery_near_1_025_ms(): t_gmax = float(h.t) assert g_max > 0.04, f"expected ExpSyn conductance peak, got g_max={g_max}" - assert 1.0 <= t_gmax <= 1.2, f"expected peak near 1.025 ms window, got t_gmax={t_gmax}" + assert ( + 1.0 <= t_gmax <= 1.2 + ), f"expected peak near 1.025 ms window, got t_gmax={t_gmax}" def test_netcon_weight_hoc_roundtrip(): @@ -96,21 +99,26 @@ def test_savestate_weight_dualwrite_roundtrip(): assert abs(nc.weight[0] - 0.99) < 1e-12 ss.restore() - assert abs(nc.weight[0] - 0.05) < 1e-12, ( - f"SaveState restore should restore HOC/SoA weight, got {nc.weight[0]}" - ) + assert ( + abs(nc.weight[0] - 0.05) < 1e-12 + ), f"SaveState restore should restore HOC/SoA weight, got {nc.weight[0]}" # Delivery after restore must use restored weight (SoA materialize path). g_max = 0.0 while h.t < 3.0 - h.dt / 2: h.fadvance() g_max = max(g_max, float(syn.g)) - assert g_max > 0.04, f"restored weight 0.05 should produce ExpSyn g peak, g_max={g_max}" - assert g_max < 0.5, f"mutated weight 0.99 must not leak into delivery, g_max={g_max}" + assert ( + g_max > 0.04 + ), f"restored weight 0.05 should produce ExpSyn g peak, g_max={g_max}" + assert ( + g_max < 0.5 + ), f"mutated weight 0.99 must not leak into delivery, g_max={g_max}" def test_bbsavestate_weight_dualwrite_roundtrip(tmp_path): """BBSaveState restore must update Weight SoA (NetCon on a gid cell).""" + # BBSaveState requires a real cell with a gid (section parented to a cell object). class Cell: def __init__(self): @@ -145,8 +153,8 @@ def __init__(self): assert abs(nc.weight[0] - 0.99) < 1e-12 bbss.restore(path) - assert abs(nc.weight[0] - 0.05) < 1e-12, ( - f"BBSaveState restore should restore HOC/SoA weight, got {nc.weight[0]}" - ) + assert ( + abs(nc.weight[0] - 0.05) < 1e-12 + ), f"BBSaveState restore should restore HOC/SoA weight, got {nc.weight[0]}" pc.gid_clear() diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index f381aa3314..e68cbf6f80 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -21,7 +21,8 @@ using namespace neuron::container::network; -TEST_CASE("SOA-backed PointProcess structure", "[Neuron][data_structures][network][point_process]") { +TEST_CASE("SOA-backed PointProcess structure", + "[Neuron][data_structures][network][point_process]") { auto& storage = neuron::model().point_processes(); REQUIRE(storage.size() == 0); @@ -115,7 +116,7 @@ TEST_CASE("SOA-backed PointProcess structure", "[Neuron][data_structures][networ } TEST_CASE("Point_process dual-write into network SoA", - "[Neuron][data_structures][network][point_process][dualwrite]") { + "[Neuron][data_structures][network][point_process][dualwrite]") { auto& storage = neuron::model().point_processes(); auto const before = storage.size(); GIVEN("A default-constructed Point_process shell") { From 15b4db3fedc5fc26d31730c938ab51468cf371db Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Thu, 23 Jul 2026 07:04:16 -0400 Subject: [PATCH 17/33] Fix FOR_NETCONS SoA sync on null-weight SelfEvent; apply format-pr SelfEvent from mech INITIAL often has nullptr weight; still route through nrn_pnt_receive_by_weight_index so FOR_NETCONS heap mutations sync to SoA. Apply ninja format-pr clang-format fixes for network SoA sources. --- src/neuron/container/network/presyn.hpp | 4 ++-- src/neuron/container/network/self_event.hpp | 5 +++-- src/neuron/model_data.hpp | 3 ++- src/nrncvode/netcvode.cpp | 21 +++++++++------------ src/nrncvode/network_soa_sort.cpp | 14 +++++++------- src/nrnoc/point.cpp | 5 ++--- src/nrnoc/treeset.cpp | 7 ++----- 7 files changed, 27 insertions(+), 32 deletions(-) diff --git a/src/neuron/container/network/presyn.hpp b/src/neuron/container/network/presyn.hpp index 7fc9c68212..20493e16c7 100644 --- a/src/neuron/container/network/presyn.hpp +++ b/src/neuron/container/network/presyn.hpp @@ -139,8 +139,8 @@ struct handle_interface: handle_base { friend std::ostream& operator<<(std::ostream& os, handle_interface const& handle) { if (handle.id()) { return os << "PreSyn{" << handle.id() << '/' << handle.underlying_storage().size() - << " thr=" << handle.threshold() << " gid=" << handle.gid() - << " nc=[" << handle.nc_index() << "," << handle.nc_count() << ")" + << " thr=" << handle.threshold() << " gid=" << handle.gid() << " nc=[" + << handle.nc_index() << "," << handle.nc_count() << ")" << " out=" << handle.output_index() << " tid=" << handle.thread_id() << '}'; } return os << "PreSyn{null}"; diff --git a/src/neuron/container/network/self_event.hpp b/src/neuron/container/network/self_event.hpp index 6bfd3e9eab..3bc9444d31 100644 --- a/src/neuron/container/network/self_event.hpp +++ b/src/neuron/container/network/self_event.hpp @@ -56,8 +56,9 @@ inline void materialize_weight_block(int weight_index, int count, double* out) { auto const n = static_cast(store.size()); for (int i = 0; i < count; ++i) { int const row = weight_index + i; - out[i] = (row >= 0 && row < n) ? store.get(static_cast(row)) - : 0.; + out[i] = (row >= 0 && row < n) + ? store.get(static_cast(row)) + : 0.; } } diff --git a/src/neuron/model_data.hpp b/src/neuron/model_data.hpp index cae73b78ac..e074b7f175 100644 --- a/src/neuron/model_data.hpp +++ b/src/neuron/model_data.hpp @@ -251,7 +251,8 @@ struct model_sorted_token { * Held as vectors (0 or 1 element) because frozen_token_type is not default- * constructible; same pattern as mech_data_tokens. */ - std::vector point_process_tokens{}; + std::vector + point_process_tokens{}; std::vector weight_tokens{}; std::vector netcon_tokens{}; std::vector presyn_tokens{}; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 8439915b94..274b954870 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -655,8 +655,8 @@ static double nc_setpost(void* v) { for (int i = 0; i < d->cnt_; ++i) { d->weight_[i] = 0.0; } - d->weight_soa_ = - neuron::container::network::Weight::allocate_weight_rows(d->cnt_, d->weight_); + d->weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(d->cnt_, + d->weight_); } d->soa_sync(); return 0.; @@ -2325,9 +2325,9 @@ void sync_netcon_weights_for_target(Point_process* pnt, bool soa_to_heap) { } // namespace void nrn_pnt_receive_by_weight_index(Point_process* pnt, - int weight_index, - double flag, - double* weight_heap) { + int weight_index, + double flag, + double* weight_heap) { if (!pnt || !pnt->prop) { return; } @@ -3563,13 +3563,10 @@ void SelfEvent::pgvts_deliver(double tt, NetCvode* ns) { } void SelfEvent::call_net_receive(NetCvode* ns) { STATISTICS(selfevent_deliver_); - // Phase 4 M2: prefer weight_index path; still pass double* to generated MOD. - if (weight_index_ >= 0) { - nrn_pnt_receive_by_weight_index(target_, weight_index_, flag_, weight_); - } else { - // Null weights (flag-only self events) or unresolved index: legacy path. - POINT_RECEIVE(target_->prop->_type, target_, weight_, flag_); - } + // Always use dual-write receive path: even when weight_ is nullptr (e.g. + // mech INITIAL net_send without a NetCon weight), FOR_NETCONS types still + // mutate peer NetCon heaps and must sync back to Weight SoA. + nrn_pnt_receive_by_weight_index(target_, weight_index_, flag_, weight_); if (errno) { if (nrn_errno_check(target_->prop->_type)) { hoc_warning("errno set during SelfEvent deliver to NET_RECEIVE", (char*) 0); diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index 293cda2ded..5d3610fc4e 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -179,14 +179,13 @@ void sort_network_data(neuron::cache::Model& cache, auto perm = partition_by_key( pp_store.size(), nrn_nthread, - [&](std::size_t row) { - return pp_store.get(row); - }, + [&](std::size_t row) { return pp_store.get(row); }, pp_offsets); for (int tid = 0; tid < nrn_nthread; ++tid) { cache.thread[static_cast(tid)].point_process_offset = - tid < static_cast(pp_offsets.size()) ? pp_offsets[static_cast(tid)] - : 0; + tid < static_cast(pp_offsets.size()) + ? pp_offsets[static_cast(tid)] + : 0; } apply_if_needed(pp_store, pp_token, std::move(perm)); } @@ -336,8 +335,9 @@ void sort_network_data(neuron::cache::Model& cache, ps_offsets); for (int tid = 0; tid < nrn_nthread; ++tid) { cache.thread[static_cast(tid)].presyn_offset = - tid < static_cast(ps_offsets.size()) ? ps_offsets[static_cast(tid)] - : 0; + tid < static_cast(ps_offsets.size()) + ? ps_offsets[static_cast(tid)] + : 0; } apply_if_needed(ps_store, ps_token, std::move(perm)); } diff --git a/src/nrnoc/point.cpp b/src/nrnoc/point.cpp index 04fbdd6965..2011bd916b 100644 --- a/src/nrnoc/point.cpp +++ b/src/nrnoc/point.cpp @@ -39,9 +39,8 @@ std::unordered_mapsecond; + auto& owner = + g_point_process_soa_owners.emplace(this, neuron::model().point_processes()).first->second; _soa_id = owner.id(); } diff --git a/src/nrnoc/treeset.cpp b/src/nrnoc/treeset.cpp index bf00829cc2..80a2f67308 100644 --- a/src/nrnoc/treeset.cpp +++ b/src/nrnoc/treeset.cpp @@ -2272,11 +2272,8 @@ neuron::model_sorted_token nrn_ensure_model_data_are_sorted() { [&cache](auto& mech_data) { nrn_fill_mech_data_caches(cache, mech_data); }); // Network: PointProcess / Weight repack / NetCon / PreSyn + fanout. // doc/network-soa-phase0.md §6.3, §8.4 - neuron::container::network::sort_network_data(cache, - pp_tokens[0], - weight_tokens[0], - netcon_tokens[0], - presyn_tokens[0]); + neuron::container::network::sort_network_data( + cache, pp_tokens[0], weight_tokens[0], netcon_tokens[0], presyn_tokens[0]); assert(model.point_processes().is_sorted()); assert(model.weights().is_sorted()); assert(model.netcons().is_sorted()); From d55851a70da00cc430c52f3cd286c5acb7b91da1 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Thu, 23 Jul 2026 09:08:14 -0400 Subject: [PATCH 18/33] Fix Point_process SoA UAF on free and NetCon weight ctor dual-write ASan CI: nrn_point_process_soa_sync during free_one_point/relocate read freed Prop or dangling state. Stop syncing on free (SoA row released in ~Point_process); harden sync to verify prop still owns this pnt and that _vnt is a live NrnThread. NetCon factory wrote magnitude only to weight_ heap; HOC weight[] reads SoA and init_events soa_to_heap wiped the value (0.0 != 0.1). Mirror magnitude into Weight SoA after construction. --- src/nrncvode/netcvode.cpp | 7 +++++++ src/nrnoc/point.cpp | 30 +++++++++++++++++++++--------- 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 274b954870..2c1fc8c558 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -4903,6 +4903,13 @@ NetCon* NetCvode::install_deliver(neuron::container::data_handle dsrc, NetCon* d = new NetCon(ps, target); d->delay_ = delay; d->weight_[0] = magnitude; + // Dual-write: constructor arg is written to heap; HOC weight[] reads SoA. + // INITIAL (init_events) does soa_to_heap then heap_to_soa — SoA must already + // hold magnitude or weight[0] is wiped to 0. + if (!d->weight_soa_.empty()) { + d->weight_soa_[0].value() = magnitude; + } + d->soa_sync(); structure_change_cnt_ = 0; return d; } diff --git a/src/nrnoc/point.cpp b/src/nrnoc/point.cpp index 2011bd916b..3eb29c4502 100644 --- a/src/nrnoc/point.cpp +++ b/src/nrnoc/point.cpp @@ -60,17 +60,27 @@ void nrn_point_process_soa_sync(Point_process* pnt) { if (!pnt || !pnt->_soa_id) { return; } + // Owner must still be alive (owning_handle in g_point_process_soa_owners). + if (!g_point_process_soa_owners.count(pnt)) { + return; + } auto h = neuron::container::network::point_process_soa(pnt); - if (pnt->prop) { - h.mech_type() = pnt->prop->_type; - // Prop for a point process always owns a mechanism SoA row. - h.instance() = static_cast(pnt->prop->current_row()); + // Only read Prop* when non-null and still linked to this Point_process. + // During free/relocate, prop may already be deleted or half-torn-down. + Prop* p = pnt->prop; + if (p && p->dparam && p->dparam[1].get() == pnt) { + h.mech_type() = p->_type; + // Prop for a point process owns a mechanism SoA row while live. + h.instance() = static_cast(p->current_row()); } else { h.mech_type() = -1; h.instance() = -1; } - if (pnt->_vnt) { - h.thread_id() = static_cast(pnt->_vnt)->id; + // _vnt is NrnThread* when set; only use if it looks like a live thread. + auto* nt = static_cast(pnt->_vnt); + if (nt && nrn_threads && nrn_nthread > 0 && nt >= nrn_threads && + nt < nrn_threads + nrn_nthread) { + h.thread_id() = nt->id; } else { h.thread_id() = -1; } @@ -359,10 +369,11 @@ void connect_point_process_pointer(void) { static void free_one_point(Point_process* pnt) { auto* p = pnt->prop; if (!p) { - nrn_point_process_soa_sync(pnt); + // SoA row is released in ~Point_process; do not touch Prop/_vnt here. + pnt->_vnt = nullptr; return; } - if (!nrn_is_artificial_[p->_type]) { + if (!nrn_is_artificial_[p->_type] && pnt->node) { auto* p1 = pnt->node->prop; if (p1 == p) { pnt->node->prop = p1->next; @@ -387,11 +398,12 @@ static void free_one_point(Point_process* pnt) { delete p; pnt->prop = (Prop*) 0; pnt->node = (Node*) 0; + pnt->_vnt = nullptr; if (pnt->sec) { section_unref(pnt->sec); } pnt->sec = (Section*) 0; - nrn_point_process_soa_sync(pnt); + // No soa_sync: reading prop/_vnt after teardown is UAF (ASan). Row freed in ~Point_process. } // called from prop_free From 77297b9e0b5059d5020705e24d1b736a4844dab4 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Thu, 23 Jul 2026 10:10:54 -0400 Subject: [PATCH 19/33] Fix null-weight SelfEvent temp buffer and init_events null prop Do not invent a short-lived weight buffer when weight_index < 0 (flag-only / INITIAL net_send); MOD may net_send that pointer and corrupt later queue / BBSaveState handling. Skip NET_RECEIVE INITIAL for NetCons whose target Prop is already freed. --- src/nrncvode/netcvode.cpp | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 2c1fc8c558..0f82f32fd5 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2367,17 +2367,18 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, } return; } - // No NetCon owner (null-weight / flag-only path): short-lived buffer OK. + // No NetCon owner: flag-only SelfEvent (weight_ often nullptr from mech INITIAL + // net_send) or unresolved index. Preserve historical nullptr for zero-weight + // cases — do **not** invent a temp buffer when weight_index < 0, or MOD may + // net_send that stack pointer and corrupt later queue handling (TQueue UAF). if (fornet) { sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); } double* buf = weight_heap; std::vector tmp; - if (!buf && n > 0) { + if (!buf && weight_index >= 0 && n > 0) { tmp.resize(static_cast(n), 0.); buf = tmp.data(); - } - if (weight_index >= 0 && n > 0 && buf) { neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); } POINT_RECEIVE(type, pnt, buf, flag); @@ -2934,8 +2935,9 @@ void NetCvode::init_events() { ITERATE(q, nclist) { Object* obj = OBJ(q); auto* d = static_cast(obj->u.this_pointer); - if (d->target_) { - int type = d->target_->prop->_type; // somehow prop is non-deterministically-null here + // target_ may outlive Prop (unlocated / free_one_point); skip dead targets. + if (d->target_ && d->target_->prop) { + int type = d->target_->prop->_type; // Dual-write: INITIAL and HOC weight[] must share one value stream. // SoA is HOC-primary; heap is the buffer for generated pnt_receive_init. d->weights_soa_to_heap(); From 1abe045645ac6f9cdafc0e786c70071655431c7a Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Thu, 23 Jul 2026 21:06:58 -0400 Subject: [PATCH 20/33] Start heap-free branch: CoreNEURON charter and index types Branch-only follow-on to dual-write PR tip. Record settled heap-free policy (O(1) weight block, packing A, queue clear on structure change, indices in bulk tables). Add weight_index_t/netcon_index_t and wire NetCon SoA WeightIndex/WeightCount to them. --- AGENTS.md | 8 ++- GROK-NETWORK-SOA.md | 11 ++-- doc/network-soa/README.md | 7 +- doc/network-soa/dual-write-and-heap.md | 15 +++-- doc/network-soa/heap-free.md | 84 ++++++++++++++++++++++++ src/neuron/container/network/indices.hpp | 29 ++++++++ src/neuron/container/network/netcon.hpp | 7 +- 7 files changed, 142 insertions(+), 19 deletions(-) create mode 100644 doc/network-soa/heap-free.md create mode 100644 src/neuron/container/network/indices.hpp diff --git a/AGENTS.md b/AGENTS.md index b542a101d7..907679bfd8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,9 @@ Full handoff and starting prompt: **`GROK-NETWORK-SOA.md`** (read on new session ## Workspace -- Repo worktree: `~/neuron/cpu_net_soa` (branch `local/cpu-network-soa`, tracks `origin/master`). +- Repo worktree: `~/neuron/cpu_net_soa`. + - `local/cpu-network-soa` — dual-write PR #3822 (keep green with master). + - `local/cpu-net-soa-heap-free` — heap-free follow-on (**branch only**, no PR); rebase onto PR tip. - Primary git object store: `~/neuron/nrngpu` — commit from **this** worktree cwd. - Sibling GPU track (paused network buffers): `~/neuron/nrngpu` @ `local/gpu-native-qualification`. @@ -17,10 +19,11 @@ source ~/neuron/bin/nrnenv nrngpu build-cpu-net-soa # create on first session Prefer CPU-only or default GPU-off builds until integration explicitly needs GPU mirrors. -## Scope (this branch) +## Scope - Network SoA: `Point_process`, `NetCon`, `PreSyn`, `weights`, `SelfEvent` in `neuron::container` style. - HOC wrappers as permutation-stable handles over backing store — **not** a second pointer graph. +- **Heap-free branch:** follow CoreNEURON as much as feasible; charter in `doc/network-soa/heap-free.md`. - **Out of scope:** Stage 2/3 GPU `net_buf_receive`, ringtest GPU network buffers (resume after SoA merges to master). ## Execute, don’t delegate @@ -32,6 +35,7 @@ Run builds and tests yourself (`ctest`, ringtest CPU spike parity). Do not tell | Topic | Path | |-------|------| | Handoff | `GROK-NETWORK-SOA.md` | +| Heap-free charter | `doc/network-soa/heap-free.md` | | Phase 0 scaffold | `doc/network-soa-phase0.md` | | Node/mechanism SoA pattern | `src/neuron/container/soa_container.hpp`, `data_handle.hpp` | | PreSyn `thvar_` handle (prototype) | `src/nrncvode/netcon.h` | diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 8df713bed6..c4f4a9c4da 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -14,7 +14,7 @@ Use this file when starting a **new** Grok session rooted in `~/neuron/cpu_net_s | **3** PreSyn SoA + fanout `NcIndex`/`NcCount` | **Done** | Rebuild at sort / `init_events` + lazy on spike | | **4** SelfEvent `weight_index` + receive-by-index | **Done** | Still `pnt_receive(..., double*, flag)` (nocmodl) | | **Sort wiring** | **Done** | Network in `nrn_ensure_model_data_are_sorted` | -| **Heap policy** | **Settled (no full free yet)** | Long-lived `weight_` required for `net_send(_w)` / FOR_NETCONS / INITIAL | +| **Heap policy** | **PR: keep `weight_`; heap-free branch: charter settled** | See `doc/network-soa/heap-free.md` | | **SaveState / BBSaveState dual-write** | **Done** | SoA values + NetCon-index SelfEvent identity | | **5** GPU net buffers | **Out of scope** | After SoA is reviewable / on master | @@ -39,7 +39,8 @@ Network SoA is a focused CPU/infrastructure PR. It should **not** carry the GPU- | Worktree | Branch | Purpose | |----------|--------|---------| -| `~/neuron/cpu_net_soa` | `local/cpu-network-soa` | Network SoA → PR to **master** | +| `~/neuron/cpu_net_soa` | `local/cpu-network-soa` | Network SoA dual-write → PR #3822 (keep green w/ master) | +| `~/neuron/cpu_net_soa` | `local/cpu-net-soa-heap-free` | Heap-free / CoreNEURON-shaped path (**branch only**, no PR); rebase onto PR tip | | `~/neuron/nrngpu` | `local/gpu-native-qualification` | Mechanism GPU, Stage 1 buffer plumbing; Stages 2–3 **paused** until SoA lands | --- @@ -115,9 +116,9 @@ Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep mate ## Recommended next work 1. ~~Sort + SaveState + BBSaveState dual-write~~ (done). -2. **Decide PR posture:** draft/review PR of dual-write+sort (no claim “land for perf yet”) vs private until heap-free / GPU (see `doc/network-soa/dual-write-and-heap.md` §5). -3. If pursuing heap-free: FOR_NETCONS index view + INITIAL by index + SelfEvent identity without heap pointer (doc L2 roadmap). -4. If pursuing authority cleanup: make fanout SoA sole hot-path truth; stop using `dil_` on spike (topology L1). +2. ~~PR dual-write green; heap-free charter~~ (`doc/network-soa/heap-free.md`, branch `local/cpu-net-soa-heap-free`). +3. **On heap-free branch:** O(1) NetCon weight-block ownership (SoA base+count); index fanout tables; FOR_NETCONS codegen; drop `weight_` when green. +4. Keep PR tip rebased on master and green; rebase heap-free onto PR tip after refreshes. 5. GPU Phase 5 only after SoA shape is stable enough to upload the same columns. --- diff --git a/doc/network-soa/README.md b/doc/network-soa/README.md index b25a3f34f9..53cdbdc7cc 100644 --- a/doc/network-soa/README.md +++ b/doc/network-soa/README.md @@ -1,6 +1,6 @@ # Network SoA — developer docs (map) -**Audience:** people working on `local/cpu-network-soa` +**Audience:** `local/cpu-network-soa` (PR dual-write) and `local/cpu-net-soa-heap-free` (branch-only) **Status tracker / session handoff:** `GROK-NETWORK-SOA.md` (repo root) **Phase-0 field tags & checklist:** `doc/network-soa-phase0.md` (detailed, longer) @@ -11,6 +11,7 @@ Docs here are **segregated by height**. Prefer the shortest document that answer | **L0 — North star** | this file §North star | One screen: goals, non-goals, current posture | | **L1 — Topology** | [topology.md](topology.md) | PreSyn → NetCon → Point_process; CoreNEURON mapping; fanout authority; rank/thread placement | | **L2 — Dual-write & heap** | [dual-write-and-heap.md](dual-write-and-heap.md) | What is primary where; why `weight_` still exists; FOR_NETCONS / INITIAL / SaveState | +| **L2b — Heap-free charter** | [heap-free.md](heap-free.md) | Settled decisions; CoreNEURON-as-feasible; queue/structure policy; implementation order | | **L3 — Sort & packing** | [sort-and-packing.md](sort-and-packing.md) | What `nrn_ensure_model_data_are_sorted` does to network containers | | **L4 — Spec detail** | `../network-soa-phase0.md` | Field tags, sidecars, SaveState notes, open questions resolved | | **Ops** | `../../GROK-NETWORK-SOA.md` | Build/test gates, commit status, “what next” | @@ -29,9 +30,9 @@ Docs here are **segregated by height**. Prefer the shortest document that answer - GPU net buffers (sibling track after SoA is usable) - Thin remote `InputPreSyn` / gid→fanout table -**Current posture (2026-07):** Dual-write complete for CoreNEURON-shaped columns; network participates in global sort; SaveState/BBSaveState dual-write; **`weight_` remains long-lived MOD scratch** because generated `pnt_receive` / `net_send(..., _w)` still use `double*`. +**Current posture (2026-07):** Dual-write complete on `local/cpu-network-soa` (PR #3822); network sort + SaveState/BBSaveState dual-write green. **`weight_` remains long-lived MOD scratch** on the PR branch. Heap-free work lives on **`local/cpu-net-soa-heap-free`** (no PR): follow CoreNEURON as much as feasible — see [heap-free.md](heap-free.md). -**PR vs master:** A PR can document progress and gather review without claiming “land now.” Landing criteria (performance/space *and* preferably a path to GPU) are product decisions separate from “is the dual-write design sound?” +**PR vs master:** Dual-write PR can gather review without claiming “land for perf.” Heap-free stays private until a green, reviewable slice exists. --- diff --git a/doc/network-soa/dual-write-and-heap.md b/doc/network-soa/dual-write-and-heap.md index 379fbd71a3..c0fab224c1 100644 --- a/doc/network-soa/dual-write-and-heap.md +++ b/doc/network-soa/dual-write-and-heap.md @@ -32,17 +32,20 @@ Passing a **temporary** buffer into `pnt_receive` broke SelfEvent identity: nocm --- -## 4. Roadmap to full heap free (if chosen) +## 4. Roadmap to full heap free + +**Active branch:** `local/cpu-net-soa-heap-free` (no PR). Settled charter: [heap-free.md](heap-free.md). Rough commit series (each gateable): 1. **SelfEvent never stores identity-only-as-temp-pointer** — always set `weight_index_` (done in dual-write spirit); never pass non-owner buffers into `pnt_receive` if MOD can net_send. -2. **FOR_NETCONS** — either CoreNEURON-style weight index perm, or generate index-based FOR_NETCONS; stop requiring foreign NetCon `weight_` bases. -3. **INITIAL** — `pnt_receive_init` by index or materialize into owner heap only. -4. **Stop allocating `new double[cnt_]`** — optional thread-local scratch of `max(pnt_receive_size)` for MOD only if ABI still needs `double*`. -5. **Delete `weight_` field** — after (1–4) and tests (stdp/FOR_NETCONS, SaveState ring, netrec init). +2. **O(1) NetCon** — one weight block (base+arity on SoA); no per-arg owning handle vector on the shell. +3. **FOR_NETCONS** — packing **A** (target-instance adjacency) + nocmodl index walk; stop requiring foreign NetCon `weight_` bases. +4. **INITIAL** — `pnt_receive_init` by index or materialize into ephemeral/owner scratch only. +5. **Stop allocating `new double[cnt_]`** — optional thread-local scratch of `max(pnt_receive_size)` for MOD only if ABI still needs `double*`. +6. **Delete `weight_` field** — after tests (stdp/FOR_NETCONS, SaveState ring, netrec init, ctest + asan). -Until (2)–(3), “full drop” is premature. +Structure change / NetCon free with a live network queue: **clear queue** (freeze after `finitialize`). See heap-free charter. --- diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md new file mode 100644 index 0000000000..872cf3ac38 --- /dev/null +++ b/doc/network-soa/heap-free.md @@ -0,0 +1,84 @@ +# L2b — Heap-free charter (`local/cpu-net-soa-heap-free`) + +**Branch only — no PR** until a reviewable slice is green. +**Base:** keep rebasing onto green `local/cpu-network-soa` (PR #3822) after master merges. + +**Charter:** Follow CoreNEURON as much as feasible for the data plane and hot path. +Interpreter, structure change, and queue polymorphism stay NEURON-specific (policies below). + +--- + +## Settled decisions + +### Data plane (CoreNEURON-shaped) + +| Item | Choice | +|------|--------| +| Weight block | One logical block of length `arity` per NetCon edge | +| NetCon shell / SoA | **O(1)** — no `vector` of per-arg owning handles | +| Record of base | NetCon SoA `(WeightIndex, WeightCount)`; arity also known from target type | +| Weight storage | Flat / Weight SoA pool; deliver uses base index | +| Indices in bulk tables | Prefer indices (`int` / `uint32_t` typedefs), not `NetCon*` / `size_t`, for weights and fanout | +| Fanout | Source-centric range (`NcIndex`/`NcCount`) into ordered NetCons | +| Packing | **A** — physical layout; thread packing required; target-instance adjacency in the same sort key (FOR_NETCONS-friendly) | +| SelfEvent identity | `weight_index` (`-1` if no NetCon); never temp `double*` as identity | +| FOR_NETCONS | Index walk over contiguous bases under A (nocmodl codegen change as needed) | +| `NET_RECEIVE` INITIAL | Index-shaped API preferred; finitialize-only | + +### NEURON-only policy + +| Item | Choice | +|------|--------| +| HOC `NetCon.weight[i]` | SoA `data_handle` from base+i; `Object*` shell not rewritten on permute | +| Ownership | NetCon owns the weight **block**; destroy invalidates the block (reclaim now or at compact/sort) | +| Target PP destroyed | Null `target_`, deactivate; **weights stay** until NetCon destroyed | +| Teardown order | `pc.gid_clear` → destroy NetCons → destroy cells (recommended) | +| Structure change / NetCon free with live network queue | **Clear queue** (or later generation-noop); no mid-run surgery requirement | +| Freeze | After `finitialize`, connectivity/layout fixed until next `finitialize` (or explicit clear + re-init) | +| TQueue | Keep `(tdeliver, DiscreteEvent*)` for now; compress later if needed | +| GPU | Out of scope on this branch | + +### Explicitly not required for v1 + +- Arbitrary model edits between `pc.psolve()` with a live queue +- Queue scrub or refcount-pin on every NetCon destroy +- Full nocmodl ABI change to `pnt_receive(…, weight_index, flag)` before materialize shim is removable +- Permanent FOR_NETCONS perm table **if** sort key already groups by target instance (**A**) + +--- + +## Contract (identity) + +> A NetCon weight block is identified by a base row in the Weight SoA (arity fixed by target mechanism type). Queued SelfEvents and generated `net_send` carry that base as `int` (or −1). FOR_NETCONS never walks foreign `double*`; it walks a target-local list of bases, contiguous after sort under packing **A**. Per-NetCon `weight_` heap is MOD scratch only until codegen/tests allow deletion. + +--- + +## Implementation order (gateable) + +1. **Docs + index typedefs** — this file; `weight_index_t` / `netcon_index_t` (this commit). +2. **O(1) ownership path** — stop growing shell with *n* owning handles; NetCon SoA base+count authority; HOC steer base+i. +3. **Fanout tables as indices** where we store many refs (rebuild at sort; CoreNEURON ranges). +4. **nocmodl FOR_NETCONS** by base list / contiguous walk. +5. **Ephemeral MOD scratch only** (thread-local max arity) where `double*` ABI remains; never identity. +6. **Delete `NetCon::weight_`** when tests (stdp/FOR_NETCONS, SaveState, netrec init, ctest + asan) stay green. +7. Rebase onto green PR tip after each PR↔master refresh. + +--- + +## Workflow + +```text +master ──► local/cpu-network-soa (PR #3822, keep green) + └── local/cpu-net-soa-heap-free (this branch, private) +``` + +Prefer rebase of heap-free onto PR tip after small green PR fixes; merge if history is too messy to rewrite. + +--- + +## Related + +- Topology / CoreNEURON map: [topology.md](topology.md) +- Dual-write history: [dual-write-and-heap.md](dual-write-and-heap.md) +- Sort: [sort-and-packing.md](sort-and-packing.md) +- CoreNEURON: `src/coreneuron/sim/multicore.hpp`, `network/netcon.hpp`, `io/setup_fornetcon.cpp` diff --git a/src/neuron/container/network/indices.hpp b/src/neuron/container/network/indices.hpp new file mode 100644 index 0000000000..84ef48d721 --- /dev/null +++ b/src/neuron/container/network/indices.hpp @@ -0,0 +1,29 @@ +#pragma once +/** + * @file network/indices.hpp + * @brief Integer index types for network bulk tables (CoreNEURON-shaped). + * + * Prefer these over NetCon* / size_t in fanout ranges, weight bases, and other + * dense tables. Shell pointers remain appropriate for HOC and TQueue + * DiscreteEvent* polymorphism. + * + * See doc/network-soa/heap-free.md. + */ +#include + +namespace neuron::container::network { + +/** @brief Base row into Weight SoA / flat weight pool (-1 = no weights). */ +using weight_index_t = std::int32_t; + +/** @brief Index into NetCon SoA or fanout order (-1 = none). */ +using netcon_index_t = std::int32_t; + +/** @brief Count of NetCons in a fanout or weight arity (non-negative). */ +using netcon_count_t = std::int32_t; + +/** @brief Sentinel: no weight block / no NetCon. */ +inline constexpr weight_index_t invalid_weight_index = -1; +inline constexpr netcon_index_t invalid_netcon_index = -1; + +} // namespace neuron::container::network diff --git a/src/neuron/container/network/netcon.hpp b/src/neuron/container/network/netcon.hpp index d626ff457e..685aaa46d7 100644 --- a/src/neuron/container/network/netcon.hpp +++ b/src/neuron/container/network/netcon.hpp @@ -11,6 +11,7 @@ * weight_soa_ is populated (see netcvode.cpp). */ #include "neuron/container/data_handle.hpp" +#include "neuron/container/network/indices.hpp" #include "neuron/container/soa_container.hpp" #include "neuron/container/view_utils.hpp" @@ -30,15 +31,15 @@ struct Target { /** @brief Base row in Weight storage for this NetCon's weight block. */ struct WeightIndex { - using type = int; + using type = weight_index_t; constexpr type default_value() const { - return -1; + return invalid_weight_index; } }; /** @brief Number of weights (pnt_receive_size). */ struct WeightCount { - using type = int; + using type = netcon_count_t; constexpr type default_value() const { return 0; } From 929f4af8fda9ff7c2823d515b8d9a50338ac3bdd Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Thu, 23 Jul 2026 22:10:29 -0400 Subject: [PATCH 21/33] Heap-free step 1: O(1) NetCon weight block ownership Replace in-shell vector of Weight owning handles with an off-shell WeightBlock (unique_ptr). NetCon SoA WeightIndex/WeightCount are the authority for the base; HOC weight[i] uses a stable data_handle from the block. Dual-write weight_ heap remains for MOD until a later step. sizeof(NetCon) 104 -> 88 on this platform. --- doc/network-soa/heap-free.md | 4 +- src/neuron/container/network/netcon.hpp | 6 +- src/neuron/container/network/weight_block.hpp | 107 +++++++++++++++--- src/nrncvode/netcon.h | 37 +++++- src/nrncvode/netcvode.cpp | 77 ++++++------- src/nrncvode/network_soa_sort.cpp | 19 ++-- src/nrniv/bbsavestate.cpp | 15 +-- src/nrniv/savstate.cpp | 12 +- test/unit_tests/container/network.cpp | 32 +++--- 9 files changed, 208 insertions(+), 101 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index 872cf3ac38..ef77936109 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -55,8 +55,8 @@ Interpreter, structure change, and queue polymorphism stay NEURON-specific (poli ## Implementation order (gateable) -1. **Docs + index typedefs** — this file; `weight_index_t` / `netcon_index_t` (this commit). -2. **O(1) ownership path** — stop growing shell with *n* owning handles; NetCon SoA base+count authority; HOC steer base+i. +1. **Docs + index typedefs** — this file; `weight_index_t` / `netcon_index_t`. +2. **O(1) ownership path** — `WeightBlock` off-shell (`unique_ptr`); NetCon SoA base+count authority; HOC `weight_soa_handle(i)`. 3. **Fanout tables as indices** where we store many refs (rebuild at sort; CoreNEURON ranges). 4. **nocmodl FOR_NETCONS** by base list / contiguous walk. 5. **Ephemeral MOD scratch only** (thread-local max arity) where `double*` ABI remains; never identity. diff --git a/src/neuron/container/network/netcon.hpp b/src/neuron/container/network/netcon.hpp index 685aaa46d7..a96651c628 100644 --- a/src/neuron/container/network/netcon.hpp +++ b/src/neuron/container/network/netcon.hpp @@ -6,9 +6,9 @@ * CoreNEURON reference: target_, u.weight_index_, delay_, active_. * Design: doc/network-soa-phase0.md §5.4, §5.4.1. * - * Phase 2 dual-write: C++ NetCon keeps DiscreteEvent + pointers; this SoA holds - * the CoreNEURON-shaped columns. HOC weight() steers into Weight SoA when - * weight_soa_ is populated (see netcvode.cpp). + * Phase 2 / heap-free step 1: C++ NetCon keeps DiscreteEvent + pointers; this + * SoA holds CoreNEURON-shaped columns. WeightIndex/WeightCount are authority for + * the weight block base; HOC weight() steers via the off-shell WeightBlock. */ #include "neuron/container/data_handle.hpp" #include "neuron/container/network/indices.hpp" diff --git a/src/neuron/container/network/weight_block.hpp b/src/neuron/container/network/weight_block.hpp index 15c70a9315..5cb57f48ea 100644 --- a/src/neuron/container/network/weight_block.hpp +++ b/src/neuron/container/network/weight_block.hpp @@ -1,40 +1,121 @@ #pragma once /** * @file network/weight_block.hpp - * @brief Helpers to dual-write NetCon weight blocks into Weight SoA. + * @brief One logical weight block (arity scalars) owned off the NetCon shell. * - * Kept separate from weights.hpp so model_data.hpp can include Weight storage - * without a circular include. + * Heap-free step 1: NetCon is O(1); authority for the base is NetCon SoA + * WeightIndex/WeightCount. This type holds the container owning_identifiers + * (required by soa<> lifetime/permute) without embedding a vector in NetCon. * - * Phase 1: heap `double* weight_` remains the delivery primary; SoA rows are - * owned in parallel for layout readiness (Phase 2 WeightIndex). + * HOC weight[i] uses value_handle(i) (stable across permute). Contiguous + * base+i access is valid after sort packs the block. + * + * See doc/network-soa/heap-free.md. */ +#include "neuron/container/network/indices.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/model_data.hpp" #include +#include +#include #include namespace neuron::container::network::Weight { /** - * @brief Allocate @p n weight SoA rows, optionally mirroring heap values. + * @brief Owning storage for one NetCon's contiguous-logical weight block. + * + * Rows remain individually tracked for permute; sort repacks them contiguous. */ -inline std::vector allocate_weight_rows(int n, double const* mirror = nullptr) { +struct WeightBlock { std::vector rows; + + [[nodiscard]] int size() const { + return static_cast(rows.size()); + } + + [[nodiscard]] bool empty() const { + return rows.empty(); + } + + /** @brief Current base row (-1 if empty). Updates after permute/erase. */ + [[nodiscard]] weight_index_t base_row() const { + if (rows.empty() || !rows.front().id()) { + return invalid_weight_index; + } + return static_cast(rows.front().current_row()); + } + + [[nodiscard]] field::Value::type& value(int i) { + assert(i >= 0 && i < size()); + return rows[static_cast(i)].value(); + } + + [[nodiscard]] field::Value::type value(int i) const { + assert(i >= 0 && i < size()); + return rows[static_cast(i)].value(); + } + + /** @brief Permutation-stable handle for HOC weight[i] / _ref_weight[i]. */ + [[nodiscard]] data_handle value_handle(int i) { + assert(i >= 0 && i < size()); + return rows[static_cast(i)].value_handle(); + } +}; + +/** + * @brief Allocate @p n weight SoA rows, optionally mirroring heap values. + * @return Owning block (nullptr if n <= 0). + */ +inline std::unique_ptr allocate_weight_block(int n, double const* mirror = nullptr) { if (n <= 0) { - return rows; + return nullptr; } + auto block = std::make_unique(); auto& store = neuron::model().weights(); - rows.reserve(static_cast(n)); + block->rows.reserve(static_cast(n)); for (int i = 0; i < n; ++i) { - rows.emplace_back(store); - rows.back().value() = mirror ? mirror[i] : 0.; + block->rows.emplace_back(store); + block->rows.back().value() = mirror ? mirror[i] : 0.; + } + return block; +} + +/** @brief Copy heap weight values into an existing block. */ +inline void mirror_heap_to_block(WeightBlock& block, double const* heap, int n) { + if (!heap) { + return; + } + auto const m = std::min(block.size(), n); + for (int i = 0; i < m; ++i) { + block.value(i) = heap[i]; + } +} + +/** @brief Copy block values into heap. */ +inline void mirror_block_to_heap(WeightBlock const& block, double* heap, int n) { + if (!heap) { + return; + } + auto const m = std::min(block.size(), n); + for (int i = 0; i < m; ++i) { + heap[i] = block.value(i); + } +} + +// --- Backward-compatible aliases used by unit tests / gradual migration --- + +/** @deprecated Prefer allocate_weight_block. */ +inline std::vector allocate_weight_rows(int n, double const* mirror = nullptr) { + auto block = allocate_weight_block(n, mirror); + if (!block) { + return {}; } - return rows; + return std::move(block->rows); } -/** @brief Copy heap weight values into already-allocated SoA rows. */ +/** @deprecated Prefer mirror_heap_to_block. */ inline void mirror_weights_to_soa(std::vector& rows, double const* heap, int n) { if (!heap) { return; diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index 35766aa34d..5e4a54b8df 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -20,6 +20,9 @@ #include #include +// Weight::WeightBlock (O(1) NetCon shell ownership of weight SoA rows) +#include "neuron/container/network/weight_block.hpp" + #if 0 #define STATISTICS(arg) ++arg #else @@ -122,6 +125,31 @@ class NetCon: public DiscreteEvent { /** @brief Copy heap weight_ → Weight SoA (after pnt_receive / MOD writes). */ void weights_heap_to_soa(); + /** @brief Allocate/replace the Weight SoA block (arity = cnt_); updates SoA base. */ + void allocate_weight_soa(double const* mirror = nullptr); + /** @brief True if a Weight SoA block is owned. */ + [[nodiscard]] bool has_weight_soa() const { + return weight_block_ && !weight_block_->empty(); + } + /** @brief Current Weight SoA base row (-1 if none). Authority: refreshed in soa_sync. */ + [[nodiscard]] neuron::container::network::weight_index_t weight_base() const { + if (has_weight_soa()) { + return weight_block_->base_row(); + } + return _soa.weight_index(); + } + /** @brief SoA value for weight[i] (requires has_weight_soa()). */ + [[nodiscard]] double& weight_soa_value(int i) { + return weight_block_->value(i); + } + [[nodiscard]] double weight_soa_value(int i) const { + return weight_block_->value(i); + } + /** @brief HOC-stable data_handle for weight[i]. */ + [[nodiscard]] neuron::container::data_handle weight_soa_handle(int i) { + return weight_block_->value_handle(i); + } + double delay_; PreSyn* src_; Point_process* target_; @@ -129,8 +157,13 @@ class NetCon: public DiscreteEvent { Object* obj_; int cnt_; bool active_; - /** @brief Phase 1 dual-write owners for Weight SoA rows (see weight_block.hpp). */ - std::vector weight_soa_{}; + /** + * @brief Owns Weight SoA rows for this edge (heap; shell stays O(1)). + * + * Authority for base/count is NetCon SoA WeightIndex/WeightCount (soa_sync). + * See doc/network-soa/heap-free.md. + */ + std::unique_ptr weight_block_{}; /** @brief Phase 2 dual-write: NetCon integration row. */ neuron::container::network::NetCon::owning_handle _soa{neuron::model().netcons()}; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 0f82f32fd5..c6165a9ac8 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -655,8 +655,7 @@ static double nc_setpost(void* v) { for (int i = 0; i < d->cnt_; ++i) { d->weight_[i] = 0.0; } - d->weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(d->cnt_, - d->weight_); + d->allocate_weight_soa(d->weight_); } d->soa_sync(); return 0.; @@ -798,9 +797,9 @@ static void steer_val(void* v) { s->arayinfo->sub[0] = d->cnt_; index = hoc_araypt(s, SYMBOL); } - // Phase 2: HOC weight() steers into flat Weight SoA when dual-write rows exist. - if (index >= 0 && index < static_cast(d->weight_soa_.size())) { - hoc_push(d->weight_soa_[index].value_handle()); + // SoA-primary: stable data_handle into the weight block (base+i logical). + if (d->has_weight_soa() && index >= 0 && index < d->weight_block_->size()) { + hoc_push(d->weight_soa_handle(index)); } else { hoc_pushpx(d->weight_ + index); } @@ -2282,11 +2281,7 @@ void selfevent_set_indices(SelfEvent* se, Point_process* pnt, double* weight) { } // weight* is almost always a NetCon heap base; resolve Weight SoA index. if (NetCon* nc = NetConSave::weight2netcon(weight)) { - if (!nc->weight_soa_.empty()) { - se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); - } else { - se->weight_index_ = nc->_soa.weight_index(); - } + se->weight_index_ = static_cast(nc->weight_base()); } } } // namespace @@ -3131,8 +3126,8 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // buffer so net_send(..., _w, ...) keeps a stable SelfEvent identity. // SoA remains HOC-primary via materialize around pnt_receive. int widx = _soa.weight_index(); - if (widx < 0 && !weight_soa_.empty()) { - widx = static_cast(weight_soa_.front().current_row()); + if (widx < 0) { + widx = static_cast(weight_base()); } if (widx >= 0) { nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); @@ -3157,8 +3152,8 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { int type = target_->prop->_type; STATISTICS(netcon_deliver_); int widx = _soa.weight_index(); - if (widx < 0 && !weight_soa_.empty()) { - widx = static_cast(weight_soa_.front().current_row()); + if (widx < 0) { + widx = static_cast(weight_base()); } if (widx >= 0) { nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); @@ -3456,11 +3451,7 @@ DiscreteEvent* SelfEvent::savestate_read(FILE* f) { NetCon* nc = NetConSave::index2netcon(ncindex); assert(nc); se->weight_ = nc->weight_; - if (!nc->weight_soa_.empty()) { - se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); - } else { - se->weight_index_ = nc->_soa.weight_index(); - } + se->weight_index_ = static_cast(nc->weight_base()); } se->flag_ = flag; se->movable_ = (moff >= 0) ? (se->target_->prop->dparam + moff) : nullptr; @@ -4908,8 +4899,8 @@ NetCon* NetCvode::install_deliver(neuron::container::data_handle dsrc, // Dual-write: constructor arg is written to heap; HOC weight[] reads SoA. // INITIAL (init_events) does soa_to_heap then heap_to_soa — SoA must already // hold magnitude or weight[0] is wiped to 0. - if (!d->weight_soa_.empty()) { - d->weight_soa_[0].value() = magnitude; + if (d->has_weight_soa()) { + d->weight_soa_value(0) = magnitude; } d->soa_sync(); structure_change_cnt_ = 0; @@ -4994,21 +4985,30 @@ void DiscreteEvent::savestate_write(FILE* f) { fprintf(f, "%d\n", DiscreteEventType); } +void NetCon::allocate_weight_soa(double const* mirror) { + weight_block_ = neuron::container::network::Weight::allocate_weight_block(cnt_, mirror); + // Authority: SoA base + count (CoreNEURON-shaped). + if (weight_block_) { + _soa.weight_index() = weight_block_->base_row(); + _soa.weight_count() = weight_block_->size(); + } else { + _soa.weight_index() = neuron::container::network::invalid_weight_index; + _soa.weight_count() = 0; + } +} + void NetCon::weights_soa_to_heap() { - if (!weight_ || weight_soa_.empty()) { + if (!weight_ || !has_weight_soa()) { return; } - auto const n = std::min(cnt_, static_cast(weight_soa_.size())); - for (int i = 0; i < n; ++i) { - weight_[i] = weight_soa_[i].value(); - } + neuron::container::network::Weight::mirror_block_to_heap(*weight_block_, weight_, cnt_); } void NetCon::weights_heap_to_soa() { - if (!weight_ || weight_soa_.empty()) { + if (!weight_ || !has_weight_soa()) { return; } - neuron::container::network::Weight::mirror_weights_to_soa(weight_soa_, weight_, cnt_); + neuron::container::network::Weight::mirror_heap_to_block(*weight_block_, weight_, cnt_); } void NetCon::soa_sync() { @@ -5027,12 +5027,14 @@ void NetCon::soa_sync() { } else { _soa.src_presyn() = -1; } - if (!weight_soa_.empty()) { - _soa.weight_index() = static_cast(weight_soa_.front().current_row()); + // Weight block base is SoA authority (refresh after permute/erase). + if (has_weight_soa()) { + _soa.weight_index() = weight_block_->base_row(); + _soa.weight_count() = weight_block_->size(); // Do not mirror heap → SoA here: HOC weight() writes SoA first; heap is only // a pnt_receive buffer. heap→SoA after MOD runs (weights_heap_to_soa). } else { - _soa.weight_index() = -1; + _soa.weight_index() = neuron::container::network::invalid_weight_index; } } @@ -5052,7 +5054,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { cnt_ = 1; weight_ = new double[cnt_]; weight_[0] = 0.0; - weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(cnt_, weight_); + allocate_weight_soa(weight_); soa_sync(); return; } @@ -5071,7 +5073,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { for (int i = 0; i < cnt_; ++i) { weight_[i] = 0.0; } - weight_soa_ = neuron::container::network::Weight::allocate_weight_rows(cnt_, weight_); + allocate_weight_soa(weight_); } soa_sync(); } @@ -5083,7 +5085,7 @@ NetCon::~NetCon() { if (cnt_) { delete[] weight_; } - weight_soa_.clear(); + weight_block_.reset(); // frees Weight SoA block rows // _soa owning_handle frees the NetCon SoA row. #if DISCRETE_EVENT_OBSERVER if (target_) { @@ -5232,12 +5234,7 @@ NetCon* NetConSave::weight_index2netcon(int weight_index) { if (!nc) { continue; } - int base = -1; - if (!nc->weight_soa_.empty()) { - base = static_cast(nc->weight_soa_.front().current_row()); - } else { - base = nc->_soa.weight_index(); - } + int const base = static_cast(nc->weight_base()); if (base == weight_index) { return nc; } diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index 5d3610fc4e..7927322b33 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -240,14 +240,17 @@ void sort_network_data(neuron::cache::Model& cache, fp = desired.size(); } } - for (auto& wh: k.nc->weight_soa_) { - if (!wh.id()) { - continue; - } - auto const row = wh.current_row(); - if (row < wsize && !used[row]) { - used[row] = 1; - desired.push_back(row); + // Contiguous logical block owned off-shell; sort packs by NetCon order. + if (k.nc->has_weight_soa()) { + for (auto& wh: k.nc->weight_block_->rows) { + if (!wh.id()) { + continue; + } + auto const row = wh.current_row(); + if (row < wsize && !used[row]) { + used[row] = 1; + desired.push_back(row); + } } } } diff --git a/src/nrniv/bbsavestate.cpp b/src/nrniv/bbsavestate.cpp index f2dd5f7d2d..27668781db 100644 --- a/src/nrniv/bbsavestate.cpp +++ b/src/nrniv/bbsavestate.cpp @@ -1122,12 +1122,7 @@ static bool selfevent_matches_netcon(SelfEvent const* se, NetCon const* nc) { if (se_widx < 0) { return false; } - int nc_widx = -1; - if (!nc->weight_soa_.empty()) { - nc_widx = static_cast(nc->weight_soa_.front().current_row()); - } else { - nc_widx = nc->_soa.weight_index(); - } + int const nc_widx = static_cast(nc->weight_base()); return nc_widx == se_widx; } @@ -1142,11 +1137,7 @@ static void selfevent_bind_netcon(SelfEvent* se, NetCon* nc) { return; } se->weight_ = nc->weight_; - if (!nc->weight_soa_.empty()) { - se->weight_index_ = static_cast(nc->weight_soa_.front().current_row()); - } else { - se->weight_index_ = nc->_soa.weight_index(); - } + se->weight_index_ = static_cast(nc->weight_base()); } SEWrap::SEWrap(const TQItem* tq, DEList* dl) { @@ -2253,7 +2244,7 @@ void BBSaveState::netrecv_pp(Point_process* pp) { f->d(nc->cnt_, nc->weight_); if (f->type() == BBSS_IO::IN) { nc->weights_heap_to_soa(); - if (!nc->weight_soa_.empty()) { + if (nc->has_weight_soa()) { nc->soa_sync(); } } diff --git a/src/nrniv/savstate.cpp b/src/nrniv/savstate.cpp index 0967c77448..597305d995 100644 --- a/src/nrniv/savstate.cpp +++ b/src/nrniv/savstate.cpp @@ -932,10 +932,10 @@ void SaveState::savenet() { int n = ncs_[i].nstate; double* w = ncs_[i].state; // HOC weight[] is SoA-primary under dual-write; prefer SoA over heap. - if (!d->weight_soa_.empty()) { - int const m = std::min(n, static_cast(d->weight_soa_.size())); + if (d->has_weight_soa()) { + int const m = std::min(n, d->weight_block_->size()); for (int j = 0; j < m; ++j) { - w[j] = d->weight_soa_[j].value(); + w[j] = d->weight_soa_value(j); } for (int j = m; j < n; ++j) { w[j] = d->weight_ ? d->weight_[j] : 0.; @@ -994,10 +994,10 @@ void SaveState::restorenet() { d->weight_[j] = w[j]; } } - if (!d->weight_soa_.empty()) { - int const m = std::min(n, static_cast(d->weight_soa_.size())); + if (d->has_weight_soa()) { + int const m = std::min(n, d->weight_block_->size()); for (int j = 0; j < m; ++j) { - d->weight_soa_[j].value() = w[j]; + d->weight_soa_value(j) = w[j]; } d->soa_sync(); // WeightIndex / reverse edges } diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index e68cbf6f80..f6e58317c4 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -190,20 +190,22 @@ TEST_CASE("SOA-backed Weight structure", "[Neuron][data_structures][network][wei } } - GIVEN("allocate_weight_rows dual-write helper") { + GIVEN("allocate_weight_block (O(1) NetCon ownership helper)") { double heap[3] = {1.5, 2.5, 3.5}; - auto rows = Weight::allocate_weight_rows(3, heap); - THEN("SoA mirrors the heap values") { - REQUIRE(rows.size() == 3); + auto block = Weight::allocate_weight_block(3, heap); + THEN("SoA mirrors the heap values and exposes a base row") { + REQUIRE(block); + REQUIRE(block->size() == 3); REQUIRE(storage.size() == 3); - REQUIRE(rows[0].value() == 1.5); - REQUIRE(rows[1].value() == 2.5); - REQUIRE(rows[2].value() == 3.5); + REQUIRE(block->value(0) == 1.5); + REQUIRE(block->value(1) == 2.5); + REQUIRE(block->value(2) == 3.5); + REQUIRE(block->base_row() >= 0); } WHEN("heap is updated and remirrored") { heap[1] = 9.0; - Weight::mirror_weights_to_soa(rows, heap, 3); - REQUIRE(rows[1].value() == 9.0); + Weight::mirror_heap_to_block(*block, heap, 3); + REQUIRE(block->value(1) == 9.0); } } } @@ -250,13 +252,13 @@ TEST_CASE("SOA-backed NetCon structure", "[Neuron][data_structures][network][net } } - GIVEN("Weight block linked like Phase 2 dual-write") { - auto wrows = Weight::allocate_weight_rows(2, nullptr); - wrows[0].value() = 0.1; - wrows[1].value() = 0.2; + GIVEN("Weight block linked via SoA base+count (heap-free step 1)") { + auto wblock = Weight::allocate_weight_block(2, nullptr); + wblock->value(0) = 0.1; + wblock->value(1) = 0.2; NetCon::owning_handle nc{storage}; - nc.weight_index() = static_cast(wrows[0].current_row()); - nc.weight_count() = 2; + nc.weight_index() = wblock->base_row(); + nc.weight_count() = wblock->size(); THEN("WeightIndex points at first SoA weight row") { REQUIRE(wstore.get(nc.weight_index()) == 0.1); REQUIRE(wstore.get(nc.weight_index() + 1) == 0.2); From 8f0e0a8c3bd923afe5705c18cfaf24793e20501b Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 06:50:50 -0400 Subject: [PATCH 22/33] Heap-free step 3: fanout order stores NetCon SoA indices Replace g_network_fanout_order vector with vector (SoA rows). Rebuild g_netcon_by_soa_row for O(1) shell resolve on deliver. PreSyn NcIndex/NcCount use the compact index types. dil_ remains the rebuild source and mid-update fallback. Unit tests cover index width and resolve model. --- doc/network-soa/heap-free.md | 3 +- doc/network-soa/topology.md | 2 +- src/neuron/container/network/presyn.hpp | 16 ++++--- src/nrncvode/netcvode.cpp | 57 ++++++++++++++++++++++--- src/nrncvode/network_soa_sort.cpp | 2 +- test/unit_tests/container/network.cpp | 29 +++++++++++++ 6 files changed, 93 insertions(+), 16 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index ef77936109..e04a409595 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -57,7 +57,8 @@ Interpreter, structure change, and queue polymorphism stay NEURON-specific (poli 1. **Docs + index typedefs** — this file; `weight_index_t` / `netcon_index_t`. 2. **O(1) ownership path** — `WeightBlock` off-shell (`unique_ptr`); NetCon SoA base+count authority; HOC `weight_soa_handle(i)`. -3. **Fanout tables as indices** where we store many refs (rebuild at sort; CoreNEURON ranges). +3. **Fanout tables as indices** — `g_network_fanout_order` holds `netcon_index_t` (SoA rows); resolve via `g_netcon_by_soa_row`. + 4. **nocmodl FOR_NETCONS** by base list / contiguous walk. 5. **Ephemeral MOD scratch only** (thread-local max arity) where `double*` ABI remains; never identity. 6. **Delete `NetCon::weight_`** when tests (stdp/FOR_NETCONS, SaveState, netrec init, ctest + asan) stay green. diff --git a/doc/network-soa/topology.md b/doc/network-soa/topology.md index b751e6a373..9792b984fa 100644 --- a/doc/network-soa/topology.md +++ b/doc/network-soa/topology.md @@ -104,7 +104,7 @@ NEURON’s global SoA + **thread partition by permutation** (like nodes/mechs) i | `nt.netcons[]` | `Model::netcons()` + `NetCon` DiscreteEvent shell | | `nt.weights[]` | `Model::weights()` + long-lived `NetCon::weight_` MOD scratch | | `nt.presyns[]` | `Model::presyns()` + `PreSyn` shell + `dil_` | -| `nc_index_/nc_cnt_` | SoA fields + `g_network_fanout_order` (NetCon* table) | +| `nc_index_/nc_cnt_` | SoA fields + `g_network_fanout_order` (`netcon_index_t` SoA rows) + row→shell resolve | | `weight_index_` | SoA `WeightIndex` (+ `weight_soa_.front().current_row()`) | | `pnt_receive(..., weight_index, flag)` | `nrn_pnt_receive_by_weight_index` → still calls nocmodl `double*` form | diff --git a/src/neuron/container/network/presyn.hpp b/src/neuron/container/network/presyn.hpp index 20493e16c7..a94fd6fc34 100644 --- a/src/neuron/container/network/presyn.hpp +++ b/src/neuron/container/network/presyn.hpp @@ -6,11 +6,13 @@ * CoreNEURON reference: nc_index_, nc_cnt_, thvar_index_, threshold_, gid_. * Design: doc/network-soa-phase0.md §5.5. * - * Phase 3 dual-write: legacy PreSyn keeps dil_ and thvar_; this SoA holds - * CoreNEURON-shaped columns. Fanout order is rebuilt into a global NetCon* - * table (see netcvode.cpp); NcIndex/NcCount describe ranges in that table. + * Phase 3 dual-write / heap-free step 3: legacy PreSyn keeps dil_ (rebuild + * source) and thvar_; this SoA holds CoreNEURON-shaped columns. Fanout order is + * a global table of NetCon SoA row indices (see netcvode.cpp); NcIndex/NcCount + * describe ranges in that table. */ #include "neuron/container/data_handle.hpp" +#include "neuron/container/network/indices.hpp" #include "neuron/container/soa_container.hpp" #include "neuron/container/view_utils.hpp" @@ -36,17 +38,17 @@ struct Gid { } }; -/** @brief Start index into global fanout order (NetCon*). */ +/** @brief Start index into global fanout order (NetCon SoA rows). */ struct NcIndex { - using type = int; + using type = netcon_index_t; constexpr type default_value() const { - return -1; + return invalid_netcon_index; } }; /** @brief Fanout count (replaces dil_.size() on hot path when sorted). */ struct NcCount { - using type = int; + using type = netcon_count_t; constexpr type default_value() const { return 0; } diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index c6165a9ac8..c8005477a4 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -3180,10 +3180,45 @@ void NetCon::pr(const char* s, double tt, NetCvode* ns) { } namespace { -// Global CoreNEURON-shaped fanout: NetCon* ranges described by PreSyn NcIndex/NcCount. -std::vector g_network_fanout_order; +// Global CoreNEURON-shaped fanout (heap-free step 3): +// g_network_fanout_order — NetCon SoA row indices (not NetCon*). +// g_netcon_by_soa_row — O(1) resolve row → shell for TQueue DiscreteEvent*. +// PreSyn NcIndex/NcCount describe ranges in g_network_fanout_order. +// dil_ remains the rebuild source (and mid-update fallback). +using neuron::container::network::netcon_index_t; + +std::vector g_network_fanout_order; +std::vector g_netcon_by_soa_row; bool g_network_fanout_sorted = true; +/** @brief Rebuild SoA-row → NetCon* table (valid only until next NetCon SoA permute). */ +void rebuild_netcon_by_soa_row() { + auto const n = neuron::model().netcons().size(); + g_netcon_by_soa_row.assign(n, nullptr); + Symbol* sym = hoc_lookup("NetCon"); + if (!sym || !sym->u.ctemplate || !sym->u.ctemplate->olist) { + return; + } + hoc_Item* q = nullptr; + ITERATE(q, sym->u.ctemplate->olist) { + auto* nc = static_cast(OBJ(q)->u.this_pointer); + if (!nc) { + continue; + } + auto const row = nc->_soa.current_row(); + if (row < n) { + g_netcon_by_soa_row[row] = nc; + } + } +} + +[[nodiscard]] NetCon* netcon_from_soa_row(netcon_index_t row) { + if (row < 0 || static_cast(row) >= g_netcon_by_soa_row.size()) { + return nullptr; + } + return g_netcon_by_soa_row[static_cast(row)]; +} + template void for_each_fanout_netcon(PreSyn* ps, F&& fn) { PreSyn::ensure_fanout_order(); @@ -3192,7 +3227,10 @@ void for_each_fanout_netcon(PreSyn* ps, F&& fn) { if (base >= 0 && cnt == static_cast(ps->dil_.size()) && base + cnt <= static_cast(g_network_fanout_order.size())) { for (int i = 0; i < cnt; ++i) { - fn(g_network_fanout_order[static_cast(base + i)]); + NetCon* d = netcon_from_soa_row(g_network_fanout_order[static_cast(base + i)]); + if (d) { + fn(d); + } } } else { // Fallback while topology is mid-update or PreSyn not in psl_. @@ -3212,6 +3250,7 @@ void PreSyn::ensure_fanout_order() { return; } g_network_fanout_order.clear(); + rebuild_netcon_by_soa_row(); if (net_cvode_instance && net_cvode_instance->psl_) { for (PreSyn* ps: *net_cvode_instance->psl_) { // Refresh dual-write scalars (threshold may have been HOC-steered via double*). @@ -3230,9 +3269,15 @@ void PreSyn::ensure_fanout_order() { } ps->_soa.nc_index() = static_cast(g_network_fanout_order.size()); for (NetCon* nc: ps->dil_) { - g_network_fanout_order.push_back(nc); - // Keep NetCon reverse edge / delay dual-write current before sim. + // Store SoA row (index), not NetCon* — half the fanout table width on LP64. + auto const row = static_cast(nc->_soa.current_row()); + g_network_fanout_order.push_back(row); + // Keep reverse edge / delay dual-write current before sim. nc->soa_sync(); + // Ensure resolve table has this shell (soa_sync may not change row). + if (row >= 0 && static_cast(row) < g_netcon_by_soa_row.size()) { + g_netcon_by_soa_row[static_cast(row)] = nc; + } } ps->_soa.nc_count() = static_cast(ps->dil_.size()); } @@ -3276,7 +3321,7 @@ void PreSyn::send(double tt, NetCvode* ns, NrnThread* nt) { } } else { STATISTICS(presyn_send_direct_); - // Phase 3: fanout via NcIndex/NcCount into global order (dual-write with dil_). + // Fanout via NcIndex/NcCount into global SoA-row order (dil_ is rebuild source). for_each_fanout_netcon(this, [&](NetCon* d) { if (d->active_ && d->target_) { NrnThread* n = PP2NT(d->target_); diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index 7927322b33..fac9860cd4 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -345,7 +345,7 @@ void sort_network_data(neuron::cache::Model& cache, apply_if_needed(ps_store, ps_token, std::move(perm)); } - // Fanout order uses NetCon* ranges; rebuild after topology/sort. + // Fanout order is NetCon SoA row indices; rebuild after topology/sort. ::PreSyn::mark_fanout_unsorted(); ::PreSyn::ensure_fanout_order(); diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index f6e58317c4..2cf8bd5890 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -5,6 +5,7 @@ #include "neuron/container/network/presyn.hpp" #include "neuron/container/network/self_event.hpp" #include "neuron/container/network/sort.hpp" +#include "neuron/container/network/indices.hpp" #include "neuron/container/network/weight_block.hpp" #include "neuron/container/network/weights.hpp" #include "neuron/model_data.hpp" @@ -14,9 +15,11 @@ #include #include +#include #include #include #include +#include #include using namespace neuron::container::network; @@ -327,9 +330,35 @@ TEST_CASE("SOA-backed PreSyn structure and fanout ranges", REQUIRE(b.nc_count() == 2); REQUIRE(b.gid() == 11); } + THEN("NcIndex/NcCount use compact index types (heap-free step 3)") { + STATIC_REQUIRE(sizeof(neuron::container::network::netcon_index_t) == 4); + STATIC_REQUIRE(sizeof(neuron::container::network::netcon_count_t) == 4); + STATIC_REQUIRE(std::is_same_v); + STATIC_REQUIRE(std::is_same_v); + } } } +TEST_CASE("Fanout order stores SoA indices not pointers", + "[Neuron][data_structures][network][fanout]") { + // Logical model: bulk fanout table is netcon_index_t (SoA row), half of NetCon* + // on LP64. Resolve table NetCon*[] is separate and rebuilt with fanout. + using neuron::container::network::netcon_index_t; + std::vector fanout_order{0, 2, 5, 1}; + std::vector shells(6, nullptr); + void* a = reinterpret_cast(static_cast(0x1000)); + void* b = reinterpret_cast(static_cast(0x2000)); + shells[0] = a; + shells[2] = b; + REQUIRE(sizeof(netcon_index_t) * fanout_order.size() < + sizeof(void*) * fanout_order.size()); + REQUIRE(shells[fanout_order[0]] == a); + REQUIRE(shells[fanout_order[1]] == b); + REQUIRE(shells[fanout_order[2]] == nullptr); +} + TEST_CASE("Network SoA sort partitions PointProcess by thread", "[Neuron][data_structures][network][sort]") { auto& pp = neuron::model().point_processes(); From 011ba62ec487f373da62711589148f6d1f585098 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 07:09:59 -0400 Subject: [PATCH 23/33] Heap-free step 4: FOR_NETCONS by Weight SoA bases nocmodl emits walks over weight bases via _nrn_netcon_weight_bases and _nrn_fornetcon_weight (resolve base to dual-write weight_ heap). ForNetConsInfo stores both bases and legacy double** argslist. Sort packs NetCon/weights by target PP SoA row after thread (packing A) for peer locality. --- doc/network-soa/heap-free.md | 4 ++- doc/network-soa/sort-and-packing.md | 5 +-- src/nocmodl/nocpout.cpp | 13 ++++++-- src/nrncvode/netcvode.cpp | 52 +++++++++++++++++++++++------ src/nrncvode/network_soa_sort.cpp | 15 +++++++-- src/nrnoc/nrn_ansi.h | 6 ++++ 6 files changed, 76 insertions(+), 19 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index e04a409595..f260578a05 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -59,7 +59,9 @@ Interpreter, structure change, and queue polymorphism stay NEURON-specific (poli 2. **O(1) ownership path** — `WeightBlock` off-shell (`unique_ptr`); NetCon SoA base+count authority; HOC `weight_soa_handle(i)`. 3. **Fanout tables as indices** — `g_network_fanout_order` holds `netcon_index_t` (SoA rows); resolve via `g_netcon_by_soa_row`. -4. **nocmodl FOR_NETCONS** by base list / contiguous walk. +4. **nocmodl FOR_NETCONS** — walk Weight SoA bases (`_nrn_netcon_weight_bases` / + `_nrn_fornetcon_weight`); sort packs by target instance (packing A). + 5. **Ephemeral MOD scratch only** (thread-local max arity) where `double*` ABI remains; never identity. 6. **Delete `NetCon::weight_`** when tests (stdp/FOR_NETCONS, SaveState, netrec init, ctest + asan) stay green. 7. Rebase onto green PR tip after each PR↔master refresh. diff --git a/doc/network-soa/sort-and-packing.md b/doc/network-soa/sort-and-packing.md index 38ff3feed4..3c3c970843 100644 --- a/doc/network-soa/sort-and-packing.md +++ b/doc/network-soa/sort-and-packing.md @@ -25,8 +25,9 @@ Brevity: short–medium. Spec detail: phase0 §6, §8. ## What sort does *not* claim - Fanout physical layout ≡ NetCon SoA order (may use separate order array). -- FOR_NETCONS adjacency (separate perm, CoreNEURON-style, still TODO for index path). -- Freeing `weight_` heap. +- FOR_NETCONS: NetCon/weight sort key includes **target PP SoA row** so peers are adjacent + (packing A); runtime walks a per-target list of weight bases (not a second perm table). +- Freeing `weight_` heap (later heap-free step). ## Implementation diff --git a/src/nocmodl/nocpout.cpp b/src/nocmodl/nocpout.cpp index 90e176eb8a..f91509902f 100644 --- a/src/nocmodl/nocpout.cpp +++ b/src/nocmodl/nocpout.cpp @@ -1176,6 +1176,10 @@ static const char *_mechanism[] = {\n\ if (net_receive_) { Lappendstr(defs_list, "static void _net_receive(Point_process*, double*, double);\n"); if (for_netcons_) { + // Heap-free step 4: walk by Weight SoA bases; resolve double* for MOD. + Lappendstr(defs_list, "extern int _nrn_netcon_weight_bases(void*, int**);\n"); + Lappendstr(defs_list, "extern double* _nrn_fornetcon_weight(int);\n"); + // Legacy dual-write ABI (still available for hand-written code). Lappendstr(defs_list, "extern int _nrn_netcon_args(void*, double***);\n"); } if (net_init_q1_) { @@ -3189,9 +3193,12 @@ void fornetcon(Item* keyword, Item* par1, Item* args, Item* par2, Item* stmt, It ++for_netcons_; deltokens(par1, par2); i = for_netcons_; + /* Heap-free step 4: iterate Weight SoA bases, resolve double* per edge. + * (Packing A groups same-target blocks for locality; list remains correct + * even when not contiguous.) */ Sprintf(buf, - "{int _ifn%d, _nfn%d; double* _fnargs%d, **_fnargslist%d;\n\ -\t_nfn%d = _nrn_netcon_args(_ppvar[_fnc_index].get(), &_fnargslist%d);\n\ + "{int _ifn%d, _nfn%d; int* _fnbases%d; double* _fnargs%d;\n\ +\t_nfn%d = _nrn_netcon_weight_bases(_ppvar[_fnc_index].get(), &_fnbases%d);\n\ \tfor (_ifn%d = 0; _ifn%d < _nfn%d; ++_ifn%d) {\n", i, i, @@ -3204,7 +3211,7 @@ void fornetcon(Item* keyword, Item* par1, Item* args, Item* par2, Item* stmt, It i, i); replacstr(keyword, buf); - Sprintf(buf, "\t _fnargs%d = _fnargslist%d[_ifn%d];\n", i, i, i); + Sprintf(buf, "\t _fnargs%d = _nrn_fornetcon_weight(_fnbases%d[_ifn%d]);\n", i, i, i); insertstr(keyword->next, buf); insertstr(qend->next, "\t}}\n"); i = 0; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index c8005477a4..454338fd9f 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -236,9 +236,19 @@ extern bool nrn_use_localgid_; extern void nrn_outputevent(unsigned char, double); #endif +/** + * @brief Per-target FOR_NETCONS peer list (heap-free step 4). + * + * Primary identity is Weight SoA bases (CoreNEURON-shaped). argslist remains + * the long-lived NetCon::weight_ pointers for the dual-write MOD ABI until + * weight_ is dropped. Generated code uses weight_bases + _nrn_fornetcon_weight. + */ struct ForNetConsInfo { - double** argslist; - int size; + int size{}; + /** @brief Weight SoA base row for each NetCon targeting this PP. */ + int* weight_bases{}; + /** @brief Legacy double* list (same order as weight_bases); dual-write heap. */ + double** argslist{}; }; static unsigned long deliver_cnt_, net_event_cnt_; @@ -4343,8 +4353,6 @@ void NetCvode::fornetcon_prepare() { _nrn_free_fornetcon(v); ForNetConsInfo* fnc = new ForNetConsInfo; *v = fnc; - fnc->argslist = 0; - fnc->size = 0; } } else { for (NrnThread* nt: for_threads(nrn_threads, nrn_nthread)) @@ -4356,13 +4364,11 @@ void NetCvode::fornetcon_prepare() { _nrn_free_fornetcon(v); ForNetConsInfo* fnc = new ForNetConsInfo; *v = fnc; - fnc->argslist = 0; - fnc->size = 0; } } } } - // two loops over all netcons. one to count, one to fill in argslist + // two loops over all netcons. one to count, one to fill bases + argslist // count if (psl_) for (const PreSyn* ps: *psl_) { @@ -4378,7 +4384,7 @@ void NetCvode::fornetcon_prepare() { } } - // allocate argslist space and initialize for another count + // allocate weight_bases + argslist; reset size for fill pass for (i = 0; i < nrn_fornetcon_cnt_; ++i) { int index = nrn_fornetcon_index_[i]; int type = nrn_fornetcon_type_[i]; @@ -4387,6 +4393,7 @@ void NetCvode::fornetcon_prepare() { for (j = 0; j < m->nodecount; ++j) { auto* fnc = static_cast(m->pdata[j][index].get()); if (fnc->size > 0) { + fnc->weight_bases = new int[fnc->size]; fnc->argslist = new double*[fnc->size]; fnc->size = 0; } @@ -4400,6 +4407,7 @@ void NetCvode::fornetcon_prepare() { auto* fnc = static_cast( m->pdata[j][index].get()); if (fnc->size > 0) { + fnc->weight_bases = new int[fnc->size]; fnc->argslist = new double*[fnc->size]; fnc->size = 0; } @@ -4407,7 +4415,7 @@ void NetCvode::fornetcon_prepare() { } } } - // fill in argslist and count again + // fill weight bases (SoA identity) and dual-write heap pointers if (psl_) { for (const PreSyn* ps: *psl_) { const NetConPList& dil = ps->dil_; @@ -4416,8 +4424,10 @@ void NetCvode::fornetcon_prepare() { if (pnt && t2i[pnt->prop->_type] > -1) { auto* fnc = static_cast( pnt->prop->dparam[t2i[pnt->prop->_type]].get()); - fnc->argslist[fnc->size] = d1->weight_; - fnc->size += 1; + int const slot = fnc->size; + fnc->weight_bases[slot] = static_cast(d1->weight_base()); + fnc->argslist[slot] = d1->weight_; + fnc->size = slot + 1; } } } @@ -4425,15 +4435,35 @@ void NetCvode::fornetcon_prepare() { } int _nrn_netcon_args(void* v, double*** argslist) { + // Legacy ABI: still used if old generated code is linked; prefer bases API. auto* fnc = static_cast(v); assert(fnc); *argslist = fnc->argslist; return fnc->size; } +int _nrn_netcon_weight_bases(void* v, int** bases) { + auto* fnc = static_cast(v); + assert(fnc); + *bases = fnc->weight_bases; + return fnc->size; +} + +double* _nrn_fornetcon_weight(int weight_base) { + // Resolve SoA base → long-lived NetCon weight_ heap (MOD scratch / dual-write). + if (weight_base < 0) { + return nullptr; + } + if (NetCon* nc = NetConSave::weight_index2netcon(weight_base)) { + return nc->weight_; + } + return nullptr; +} + void _nrn_free_fornetcon(void** v) { if (auto* fnc = static_cast(*v); fnc) { delete[] std::exchange(fnc->argslist, nullptr); + delete[] std::exchange(fnc->weight_bases, nullptr); delete fnc; *v = nullptr; } diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index fac9860cd4..f2b39544d8 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -190,10 +190,12 @@ void sort_network_data(neuron::cache::Model& cache, apply_if_needed(pp_store, pp_token, std::move(perm)); } - // Order NetCons by (target thread, src PreSyn SoA row, NetCon SoA row). + // Order NetCons by (target thread, target PP SoA row, src PreSyn, NetCon row). + // Target-instance adjacency (packing A) makes FOR_NETCONS weight bases local. struct NcSortKey { ::NetCon* nc{}; int tid{-1}; + int tgt_row{-1}; int src_row{-1}; std::size_t soa_row{0}; }; @@ -203,6 +205,7 @@ void sort_network_data(neuron::cache::Model& cache, NcSortKey k; k.nc = nc; k.tid = target_thread_id(nc); + k.tgt_row = nc && nc->target_ ? nrn_point_process_soa_row(nc->target_) : -1; k.src_row = src_presyn_row(nc); k.soa_row = nc->_soa.current_row(); ordered.push_back(k); @@ -216,6 +219,14 @@ void sort_network_data(neuron::cache::Model& cache, return at < bt; } } + if (a.tgt_row != b.tgt_row) { + // Unassigned targets after assigned. + int const at = a.tgt_row < 0 ? std::numeric_limits::max() : a.tgt_row; + int const bt = b.tgt_row < 0 ? std::numeric_limits::max() : b.tgt_row; + if (at != bt) { + return at < bt; + } + } if (a.src_row != b.src_row) { return a.src_row < b.src_row; } @@ -223,7 +234,7 @@ void sort_network_data(neuron::cache::Model& cache, }); // ------------------------------------------------------------------ - // 2. Weight repack: contiguous blocks per NetCon, packed by target thread. + // 2. Weight repack: contiguous blocks per NetCon (same order as above). // ------------------------------------------------------------------ { std::size_t const wsize = w_store.size(); diff --git a/src/nrnoc/nrn_ansi.h b/src/nrnoc/nrn_ansi.h index 0ac243ec6d..dec4376c3e 100644 --- a/src/nrnoc/nrn_ansi.h +++ b/src/nrnoc/nrn_ansi.h @@ -30,6 +30,12 @@ extern "C" void modl_reg(void); // nrnmech stuff extern void _nrn_free_fornetcon(void**); +/** @brief Legacy FOR_NETCONS: return double** list of NetCon weight_ heaps. */ +extern int _nrn_netcon_args(void*, double***); +/** @brief FOR_NETCONS by Weight SoA base list (heap-free step 4). */ +extern int _nrn_netcon_weight_bases(void*, int**); +/** @brief Resolve weight SoA base → long-lived weight_ for MOD body. */ +extern double* _nrn_fornetcon_weight(int weight_base); extern double nrn_call_mech_func(Symbol*, int narg, Prop*, int type); // mod stuff From 47164a29e5a8c4e0df926474ada6e8de236cef36 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 07:27:50 -0400 Subject: [PATCH 24/33] Heap-free step 5: ephemeral MOD weight scratch, not identity Non-FOR_NETCONS pnt_receive materializes Weight SoA into a thread-local buffer and writes back after MOD. TLS active weight_index binds net_send SelfEvents so scratch pointers are never queued. FOR_NETCONS still uses long-lived weight_ heaps until step 6. nrn_net_send only retains weight_ when it is a known NetCon heap base. --- doc/network-soa/heap-free.md | 4 +- src/nrncvode/netcvode.cpp | 169 +++++++++++++++++++++-------------- 2 files changed, 107 insertions(+), 66 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index f260578a05..d7ef69358b 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -62,7 +62,9 @@ Interpreter, structure change, and queue polymorphism stay NEURON-specific (poli 4. **nocmodl FOR_NETCONS** — walk Weight SoA bases (`_nrn_netcon_weight_bases` / `_nrn_fornetcon_weight`); sort packs by target instance (packing A). -5. **Ephemeral MOD scratch only** (thread-local max arity) where `double*` ABI remains; never identity. +5. **Ephemeral MOD scratch only** — thread-local buffer for non-FOR_NETCONS + `pnt_receive`; TLS active `weight_index` so `net_send(..., _w)` never queues + scratch as identity. FOR_NETCONS still uses long-lived `weight_` until step 6. 6. **Delete `NetCon::weight_`** when tests (stdp/FOR_NETCONS, SaveState, netrec init, ctest + asan) stay green. 7. Rebase onto green PR tip after each PR↔master refresh. diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 454338fd9f..12613601ec 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2282,16 +2282,48 @@ void NetCvode::remove_event(TQItem* q, int tid) { } namespace { -/** Phase 4: fill SelfEvent dual-write index fields from weight* / Point_process. */ +/** + * Heap-free step 5: ephemeral MOD scratch for pnt_receive double* ABI. + * Never use this buffer as SelfEvent / net_send identity — only weight_index. + */ +thread_local std::vector g_mod_weight_scratch; +/** @brief Weight SoA base of the NET_RECEIVE currently on the stack (−1 if none). */ +thread_local int g_tls_receive_weight_index = -1; + +double* mod_weight_scratch(int n) { + if (n <= 0) { + return nullptr; + } + if (static_cast(g_mod_weight_scratch.size()) < n) { + g_mod_weight_scratch.resize(static_cast(n)); + } + return g_mod_weight_scratch.data(); +} + +struct ReceiveWeightIndexGuard { + int prev; + explicit ReceiveWeightIndexGuard(int idx) + : prev(g_tls_receive_weight_index) { + g_tls_receive_weight_index = idx; + } + ~ReceiveWeightIndexGuard() { + g_tls_receive_weight_index = prev; + } +}; + +/** Phase 4/5: SelfEvent identity is Weight SoA base (and optional long-lived heap). */ void selfevent_set_indices(SelfEvent* se, Point_process* pnt, double* weight) { se->target_row_ = nrn_point_process_soa_row(pnt); se->weight_index_ = -1; - if (!weight) { - return; + if (weight) { + if (NetCon* nc = NetConSave::weight2netcon(weight)) { + se->weight_index_ = static_cast(nc->weight_base()); + return; + } } - // weight* is almost always a NetCon heap base; resolve Weight SoA index. - if (NetCon* nc = NetConSave::weight2netcon(weight)) { - se->weight_index_ = static_cast(nc->weight_base()); + // net_send(..., _w) during NET_RECEIVE with ephemeral scratch: inherit active receive base. + if (g_tls_receive_weight_index >= 0) { + se->weight_index_ = g_tls_receive_weight_index; } } } // namespace @@ -2342,56 +2374,59 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, } int const n = pnt_receive_size[type]; bool const fornet = type_has_fornetcon(type); + // Publish active base so net_send(..., _w) during this receive binds SelfEvent + // by weight_index even when _w is ephemeral scratch (heap-free step 5). + ReceiveWeightIndexGuard const tls_guard(weight_index); - // Prefer a long-lived NetCon weight_ heap buffer for pnt_receive. Temporary - // materialize buffers are unsafe: nocmodl NET_RECEIVE often does - // net_send(..., _w, ...) and SelfEvent must keep a stable weight_ identity - // for SaveState/BBSaveState (weight2netcon). NetCon* owner = nullptr; if (weight_heap) { owner = NetConSave::weight2netcon(weight_heap); } if (!owner && weight_index >= 0) { owner = NetConSave::weight_index2netcon(weight_index); - if (owner && owner->weight_) { - weight_heap = owner->weight_; - } - } - if (owner) { - if (fornet) { - // FOR_NETCONS walks weight_ of all NetCons with this target. - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); - } else { - owner->weights_soa_to_heap(); - } - POINT_RECEIVE(type, pnt, weight_heap ? weight_heap : owner->weight_, flag); - if (fornet) { - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); - } else { - owner->weights_heap_to_soa(); - } - return; } - // No NetCon owner: flag-only SelfEvent (weight_ often nullptr from mech INITIAL - // net_send) or unresolved index. Preserve historical nullptr for zero-weight - // cases — do **not** invent a temp buffer when weight_index < 0, or MOD may - // net_send that stack pointer and corrupt later queue handling (TQueue UAF). + + // FOR_NETCONS mutates peer NetCon weight_ heaps in place — keep long-lived + // dual-write heaps for those types until step 6 drops weight_. if (fornet) { sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); + double* buf = nullptr; + if (owner && owner->weight_) { + buf = owner->weight_; + } else if (weight_heap) { + buf = weight_heap; + } else if (weight_index >= 0 && n > 0) { + buf = mod_weight_scratch(n); + neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, + n, + buf); + } + // flag-only / no weights: historical nullptr (do not invent scratch when n==0 + // or weight_index < 0 without owner). + POINT_RECEIVE(type, pnt, buf, flag); + if (weight_index >= 0 && n > 0 && buf && buf == mod_weight_scratch(n)) { + neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); + } + sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); + return; } - double* buf = weight_heap; - std::vector tmp; - if (!buf && weight_index >= 0 && n > 0) { - tmp.resize(static_cast(n), 0.); - buf = tmp.data(); - neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); + + // Non-FOR_NETCONS: SoA is source of truth; MOD sees thread-local scratch only. + // Scratch is never SelfEvent identity (see nrn_net_send + tls_guard). + if (weight_index < 0 || n <= 0) { + // Flag-only SelfEvent / zero-arity: pass nullptr (historical). + POINT_RECEIVE(type, pnt, nullptr, flag); + return; } + double* buf = mod_weight_scratch(n); + neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); POINT_RECEIVE(type, pnt, buf, flag); - if (weight_index >= 0 && n > 0 && buf) { - neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); - } - if (fornet) { - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); + neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); + // Keep dual-write heap mirror for BBSaveState / FOR_NETCONS peers until step 6. + if (owner && owner->weight_) { + for (int i = 0; i < n; ++i) { + owner->weight_[i] = buf[i]; + } } } @@ -2404,8 +2439,18 @@ void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, doubl SelfEvent* se = p.sepool_->alloc(); se->flag_ = flag; se->target_ = pnt; - se->weight_ = weight; + // Identity: weight_index (and optional long-lived NetCon weight_ only). + // Never queue an ephemeral MOD scratch pointer (heap-free step 5). selfevent_set_indices(se, pnt, weight); + se->weight_ = nullptr; + if (weight) { + if (NetCon* nc = NetConSave::weight2netcon(weight)) { + se->weight_ = nc->weight_; + if (se->weight_index_ < 0) { + se->weight_index_ = static_cast(nc->weight_base()); + } + } + } se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -2432,8 +2477,16 @@ void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, d SelfEvent* se = p.sepool_->alloc(); se->flag_ = flag; se->target_ = pnt; - se->weight_ = weight; selfevent_set_indices(se, pnt, weight); + se->weight_ = nullptr; + if (weight) { + if (NetCon* nc = NetConSave::weight2netcon(weight)) { + se->weight_ = nc->weight_; + if (se->weight_index_ < 0) { + se->weight_index_ = static_cast(nc->weight_base()); + } + } + } se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -3132,20 +3185,13 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); - // Phase 4: deliver via weight_index; use long-lived weight_ heap as MOD - // buffer so net_send(..., _w, ...) keeps a stable SelfEvent identity. - // SoA remains HOC-primary via materialize around pnt_receive. + // Heap-free step 5: deliver by weight_index; MOD sees ephemeral scratch + // (or FOR_NETCONS long-lived heaps). SelfEvent identity is weight_index. int widx = _soa.weight_index(); if (widx < 0) { widx = static_cast(weight_base()); } - if (widx >= 0) { - nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); - } else { - weights_soa_to_heap(); - POINT_RECEIVE(type, target_, weight_, 0); - weights_heap_to_soa(); - } + nrn_pnt_receive_by_weight_index(target_, widx, 0., /*weight_heap*/ nullptr); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3165,13 +3211,7 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { if (widx < 0) { widx = static_cast(weight_base()); } - if (widx >= 0) { - nrn_pnt_receive_by_weight_index(target_, widx, 0., weight_); - } else { - weights_soa_to_heap(); - POINT_RECEIVE(type, target_, weight_, 0); - weights_heap_to_soa(); - } + nrn_pnt_receive_by_weight_index(target_, widx, 0., /*weight_heap*/ nullptr); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3611,9 +3651,8 @@ void SelfEvent::pgvts_deliver(double tt, NetCvode* ns) { } void SelfEvent::call_net_receive(NetCvode* ns) { STATISTICS(selfevent_deliver_); - // Always use dual-write receive path: even when weight_ is nullptr (e.g. - // mech INITIAL net_send without a NetCon weight), FOR_NETCONS types still - // mutate peer NetCon heaps and must sync back to Weight SoA. + // Deliver by weight_index; weight_ is optional long-lived NetCon heap only. + // Ephemeral scratch is never stored on SelfEvent (heap-free step 5). nrn_pnt_receive_by_weight_index(target_, weight_index_, flag_, weight_); if (errno) { if (nrn_errno_check(target_->prop->_type)) { From ac543296f48f99c7f1c8b29ca0f5073d4174eee0 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 07:45:11 -0400 Subject: [PATCH 25/33] Heap-free step 6: remove NetCon::weight_ heap Weight SoA is sole storage for edge weights. Non-FOR_NETCONS deliver uses thread-local scratch; FOR_NETCONS owns per-target scratch buffers keyed by weight bases. weight_soa_data() exposes contiguous SoA doubles for legacy APIs (BBSS, CoreNEURON export, nrn_netcon_weight). SelfEvent identity is weight_index only. sizeof(NetCon) 80 on this platform. --- doc/network-soa/heap-free.md | 5 +- src/nrncvode/netcon.h | 31 +- src/nrncvode/netcvode.cpp | 351 +++++++++--------- src/nrncvode/network_soa_sort.cpp | 2 + src/nrniv/bbsavestate.cpp | 38 +- .../callbacks/nrncore_callbacks.cpp | 32 +- src/nrniv/prcellstate.cpp | 7 +- src/nrniv/savstate.cpp | 18 +- src/nrnoc/nrn_ansi.h | 6 +- 9 files changed, 254 insertions(+), 236 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index d7ef69358b..c6f37a3ab5 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -64,8 +64,9 @@ Interpreter, structure change, and queue polymorphism stay NEURON-specific (poli 5. **Ephemeral MOD scratch only** — thread-local buffer for non-FOR_NETCONS `pnt_receive`; TLS active `weight_index` so `net_send(..., _w)` never queues - scratch as identity. FOR_NETCONS still uses long-lived `weight_` until step 6. -6. **Delete `NetCon::weight_`** when tests (stdp/FOR_NETCONS, SaveState, netrec init, ctest + asan) stay green. + scratch as identity. +6. **Delete `NetCon::weight_`** — SoA only; FOR_NETCONS owns per-target scratch + buffers keyed by weight bases; `weight_soa_data()` for contiguous double* APIs. 7. Rebase onto green PR tip after each PR↔master refresh. --- diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index 5e4a54b8df..11882c813a 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -118,12 +118,8 @@ class NetCon: public DiscreteEvent { void replace_src(PreSyn*); virtual void disconnect(Observable*); - /** @brief Sync legacy NetCon fields into NetCon + Weight SoA (Phase 2). */ + /** @brief Sync legacy NetCon fields into NetCon + Weight SoA. */ void soa_sync(); - /** @brief Copy Weight SoA values → heap weight_ (before pnt_receive). */ - void weights_soa_to_heap(); - /** @brief Copy heap weight_ → Weight SoA (after pnt_receive / MOD writes). */ - void weights_heap_to_soa(); /** @brief Allocate/replace the Weight SoA block (arity = cnt_); updates SoA base. */ void allocate_weight_soa(double const* mirror = nullptr); @@ -131,7 +127,7 @@ class NetCon: public DiscreteEvent { [[nodiscard]] bool has_weight_soa() const { return weight_block_ && !weight_block_->empty(); } - /** @brief Current Weight SoA base row (-1 if none). Authority: refreshed in soa_sync. */ + /** @brief Current Weight SoA base row (-1 if none). */ [[nodiscard]] neuron::container::network::weight_index_t weight_base() const { if (has_weight_soa()) { return weight_block_->base_row(); @@ -149,22 +145,28 @@ class NetCon: public DiscreteEvent { [[nodiscard]] neuron::container::data_handle weight_soa_handle(int i) { return weight_block_->value_handle(i); } + /** + * @brief Pointer to first Weight SoA value of this edge (cnt_ consecutive rows). + * + * Valid while the model is not mid-permute and the block remains contiguous + * (true after allocate and after network sort). Used for CoreNEURON export + * and legacy double* APIs — not for SelfEvent identity. + */ + [[nodiscard]] double* weight_soa_data(); + [[nodiscard]] double const* weight_soa_data() const; double delay_; PreSyn* src_; Point_process* target_; - double* weight_; Object* obj_; int cnt_; bool active_; /** - * @brief Owns Weight SoA rows for this edge (heap; shell stays O(1)). - * - * Authority for base/count is NetCon SoA WeightIndex/WeightCount (soa_sync). - * See doc/network-soa/heap-free.md. + * @brief Owns Weight SoA rows for this edge (shell O(1); no per-NetCon weight_ heap). + * See doc/network-soa/heap-free.md step 6. */ std::unique_ptr weight_block_{}; - /** @brief Phase 2 dual-write: NetCon integration row. */ + /** @brief NetCon integration SoA row. */ neuron::container::network::NetCon::owning_handle _soa{neuron::model().netcons()}; static unsigned long netcon_send_active_; @@ -184,7 +186,10 @@ class NetConSave: public DiscreteEvent { NetCon* netcon_; static void invalid(); - /** @brief Map long-lived NetCon weight_ heap base → NetCon*. */ + /** + * @brief Map double* → NetCon* if it is a FOR_NETCONS scratch buffer or + * Weight SoA data pointer known for a live edge. Prefer weight_index2netcon. + */ static NetCon* weight2netcon(double*); /** @brief Map HOC NetCon object index → NetCon*. */ static NetCon* index2netcon(long); diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 12613601ec..1cfd655f3a 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -44,6 +44,7 @@ #include #include #include +#include #include #include @@ -121,10 +122,8 @@ void nrn_netcon_set_delay(NetCon* nc, double d) { nc->soa_sync(); } int nrn_netcon_weight(NetCon* nc, double** pw) { - // SoA is HOC source of truth when weight_soa_ is populated; keep heap current - // for MOD code that writes through the returned pointer. - nc->weights_soa_to_heap(); - *pw = nc->weight_; + // Heap-free: contiguous Weight SoA block (not per-NetCon weight_ heap). + *pw = nc->weight_soa_data(); return nc->cnt_; } double nrn_event_queue_stats(double* stats) { @@ -163,8 +162,7 @@ int nrn_netcon_info(NetCon* nc, double** pw, Point_process** target, double** th *target = (nc->target_) ? nc->target_ : (Point_process*) 0; *th = (nc->src_) ? &(nc->src_->threshold_) : (double*) 0; *del = &nc->delay_; - nc->weights_soa_to_heap(); - *pw = nc->weight_; + *pw = nc->weight_soa_data(); return nc->cnt_; } @@ -237,20 +235,28 @@ extern void nrn_outputevent(unsigned char, double); #endif /** - * @brief Per-target FOR_NETCONS peer list (heap-free step 4). + * @brief Per-target FOR_NETCONS peer list (heap-free steps 4–6). * - * Primary identity is Weight SoA bases (CoreNEURON-shaped). argslist remains - * the long-lived NetCon::weight_ pointers for the dual-write MOD ABI until - * weight_ is dropped. Generated code uses weight_bases + _nrn_fornetcon_weight. + * Identity is Weight SoA bases. argslist[i] points into owned scratch of length + * arity (not NetCon::weight_ — that field is gone). Generated code uses + * weight_bases + _nrn_fornetcon_weight. */ struct ForNetConsInfo { int size{}; + int arity{}; /** @brief Weight SoA base row for each NetCon targeting this PP. */ int* weight_bases{}; - /** @brief Legacy double* list (same order as weight_bases); dual-write heap. */ + /** @brief MOD-facing double* list (owned scratch, same order as bases). */ double** argslist{}; + /** @brief Flat storage for argslist[i][0..arity). */ + double* weight_storage{}; }; +/** @brief base → FOR_NETCONS scratch (rebuilt in fornetcon_prepare). */ +std::unordered_map g_fornet_base_to_buf; +/** @brief base → NetCon* for weight2netcon when pointer is SoA data or fornet buf. */ +std::unordered_map g_weight_ptr_to_netcon; + static unsigned long deliver_cnt_, net_event_cnt_; unsigned long DiscreteEvent::discretevent_send_; unsigned long DiscreteEvent::discretevent_deliver_; @@ -660,12 +666,7 @@ static double nc_setpost(void* v) { } if (d->cnt_ != cnt) { d->cnt_ = cnt; - delete[] d->weight_; - d->weight_ = d->cnt_ ? new double[d->cnt_] : nullptr; - for (int i = 0; i < d->cnt_; ++i) { - d->weight_[i] = 0.0; - } - d->allocate_weight_soa(d->weight_); + d->allocate_weight_soa(nullptr); } d->soa_sync(); return 0.; @@ -713,7 +714,8 @@ static double nc_event(void* v) { hoc_execerror("Can only send fake self-events to ARTIFICIAL_CELLs", 0); } auto* pq = pnt->prop->dparam + nrn_artcell_qindex_[type]; - nrn_net_send(pq, d->weight_, pnt, td, flag); + // Pass Weight SoA base pointer when present so weight2netcon can bind. + nrn_net_send(pq, d->weight_soa_data(), pnt, td, flag); } else { net_cvode_instance->event(td, d, PP2NT(d->target_)); } @@ -811,7 +813,9 @@ static void steer_val(void* v) { if (d->has_weight_soa() && index >= 0 && index < d->weight_block_->size()) { hoc_push(d->weight_soa_handle(index)); } else { - hoc_pushpx(d->weight_ + index); + static double dummy_weight = 0.; + dummy_weight = 0.; + hoc_pushpx(&dummy_weight); } } else if (strcmp(s->name, "x") == 0) { static double dummy = 0.; @@ -2338,27 +2342,45 @@ bool type_has_fornetcon(int type) { return false; } -/** Sync all NetCon weight heaps ↔ SoA that share a target (FOR_NETCONS mutates them). */ -void sync_netcon_weights_for_target(Point_process* pnt, bool soa_to_heap) { - if (!pnt) { +/** Sync FOR_NETCONS owned scratch buffers ↔ Weight SoA for one target PP. */ +void sync_fornetcon_buffers_for_target(Point_process* pnt, bool soa_to_buf) { + if (!pnt || !pnt->prop) { return; } - Symbol* sym = hoc_lookup("NetCon"); - if (!sym || !sym->u.ctemplate || !sym->u.ctemplate->olist) { + int const type = pnt->prop->_type; + int fn_slot = -1; + for (int i = 0; i < nrn_fornetcon_cnt_; ++i) { + if (nrn_fornetcon_type_[i] == type) { + fn_slot = nrn_fornetcon_index_[i]; + break; + } + } + if (fn_slot < 0) { return; } - hoc_Item* q = nullptr; - ITERATE(q, sym->u.ctemplate->olist) { - auto* nc = static_cast(OBJ(q)->u.this_pointer); - if (nc && nc->target_ == pnt) { - if (soa_to_heap) { - nc->weights_soa_to_heap(); - } else { - nc->weights_heap_to_soa(); - } + auto* fnc = static_cast(pnt->prop->dparam[fn_slot].get()); + if (!fnc || fnc->size <= 0 || !fnc->weight_bases || !fnc->argslist) { + return; + } + int const arity = fnc->arity > 0 ? fnc->arity : pnt_receive_size[type]; + for (int i = 0; i < fnc->size; ++i) { + int const base = fnc->weight_bases[i]; + double* buf = fnc->argslist[i]; + if (base < 0 || !buf || arity <= 0) { + continue; + } + if (soa_to_buf) { + neuron::container::network::SelfEventFields::materialize_weight_block(base, arity, buf); + } else { + neuron::container::network::SelfEventFields::store_weight_block(base, arity, buf); } } } + +// WatchCondition / legacy name: SoA ↔ FOR_NETCONS owned scratch. +void sync_netcon_weights_for_target(Point_process* pnt, bool soa_to_heap) { + sync_fornetcon_buffers_for_target(pnt, soa_to_heap); +} } // namespace void nrn_pnt_receive_by_weight_index(Point_process* pnt, @@ -2374,47 +2396,38 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, } int const n = pnt_receive_size[type]; bool const fornet = type_has_fornetcon(type); - // Publish active base so net_send(..., _w) during this receive binds SelfEvent - // by weight_index even when _w is ephemeral scratch (heap-free step 5). + // Publish active base so net_send(..., _w) binds SelfEvent by weight_index. ReceiveWeightIndexGuard const tls_guard(weight_index); - NetCon* owner = nullptr; - if (weight_heap) { - owner = NetConSave::weight2netcon(weight_heap); - } - if (!owner && weight_index >= 0) { - owner = NetConSave::weight_index2netcon(weight_index); - } - - // FOR_NETCONS mutates peer NetCon weight_ heaps in place — keep long-lived - // dual-write heaps for those types until step 6 drops weight_. + // FOR_NETCONS: materialize all peer edges into owned scratch, run MOD, write back. if (fornet) { - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ true); + sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ true); double* buf = nullptr; - if (owner && owner->weight_) { - buf = owner->weight_; - } else if (weight_heap) { + if (weight_index >= 0) { + auto it = g_fornet_base_to_buf.find(weight_index); + if (it != g_fornet_base_to_buf.end()) { + buf = it->second; + } + } + if (!buf && weight_heap) { buf = weight_heap; - } else if (weight_index >= 0 && n > 0) { + } + if (!buf && weight_index >= 0 && n > 0) { buf = mod_weight_scratch(n); neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); } - // flag-only / no weights: historical nullptr (do not invent scratch when n==0 - // or weight_index < 0 without owner). POINT_RECEIVE(type, pnt, buf, flag); - if (weight_index >= 0 && n > 0 && buf && buf == mod_weight_scratch(n)) { + if (weight_index >= 0 && n > 0 && buf == mod_weight_scratch(n)) { neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); } - sync_netcon_weights_for_target(pnt, /*soa_to_heap*/ false); + sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ false); return; } - // Non-FOR_NETCONS: SoA is source of truth; MOD sees thread-local scratch only. - // Scratch is never SelfEvent identity (see nrn_net_send + tls_guard). + // Non-FOR_NETCONS: thread-local scratch only (never SelfEvent identity). if (weight_index < 0 || n <= 0) { - // Flag-only SelfEvent / zero-arity: pass nullptr (historical). POINT_RECEIVE(type, pnt, nullptr, flag); return; } @@ -2422,12 +2435,6 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); POINT_RECEIVE(type, pnt, buf, flag); neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); - // Keep dual-write heap mirror for BBSaveState / FOR_NETCONS peers until step 6. - if (owner && owner->weight_) { - for (int i = 0; i < n; ++i) { - owner->weight_[i] = buf[i]; - } - } } // for threads, revised net_send to use absolute time (in the @@ -2439,16 +2446,12 @@ void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, doubl SelfEvent* se = p.sepool_->alloc(); se->flag_ = flag; se->target_ = pnt; - // Identity: weight_index (and optional long-lived NetCon weight_ only). - // Never queue an ephemeral MOD scratch pointer (heap-free step 5). + // Identity is weight_index only (heap-free step 6: no NetCon::weight_). selfevent_set_indices(se, pnt, weight); se->weight_ = nullptr; - if (weight) { + if (weight && se->weight_index_ < 0) { if (NetCon* nc = NetConSave::weight2netcon(weight)) { - se->weight_ = nc->weight_; - if (se->weight_index_ < 0) { - se->weight_index_ = static_cast(nc->weight_base()); - } + se->weight_index_ = static_cast(nc->weight_base()); } } se->movable_ = v; // needed for SaveState @@ -2479,12 +2482,9 @@ void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, d se->target_ = pnt; selfevent_set_indices(se, pnt, weight); se->weight_ = nullptr; - if (weight) { + if (weight && se->weight_index_ < 0) { if (NetCon* nc = NetConSave::weight2netcon(weight)) { - se->weight_ = nc->weight_; - if (se->weight_index_ < 0) { - se->weight_index_ = static_cast(nc->weight_base()); - } + se->weight_index_ = static_cast(nc->weight_base()); } } se->movable_ = v; // needed for SaveState @@ -2998,16 +2998,25 @@ void NetCvode::init_events() { int type = d->target_->prop->_type; // Dual-write: INITIAL and HOC weight[] must share one value stream. // SoA is HOC-primary; heap is the buffer for generated pnt_receive_init. - d->weights_soa_to_heap(); + int const widx = static_cast(d->weight_base()); + int const n = d->cnt_; if (pnt_receive_init[type]) { - (*pnt_receive_init[type])(d->target_, d->weight_, 0); - } else { - // not the first - for (j = d->cnt_ - 1; j > 0; --j) { - d->weight_[j] = 0.; + double* buf = nullptr; + if (widx >= 0 && n > 0) { + buf = mod_weight_scratch(n); + neuron::container::network::SelfEventFields::materialize_weight_block( + widx, n, buf); + } + (*pnt_receive_init[type])(d->target_, buf, 0); + if (widx >= 0 && n > 0 && buf) { + neuron::container::network::SelfEventFields::store_weight_block(widx, n, buf); + } + } else if (d->has_weight_soa()) { + // not the first weight: zero weight[1..] in SoA + for (j = n - 1; j > 0; --j) { + d->weight_soa_value(j) = 0.; } } - d->weights_heap_to_soa(); } } if (gcv_) { @@ -3515,13 +3524,11 @@ DiscreteEvent* SelfEvent::savestate_save() { void SelfEvent::savestate_restore(double tt, NetCvode* nc) { // pr("savestate_restore", tt, nc); - // Prefer long-lived NetCon heap base when known; if only weight_index_ survived - // (heap-drop / index identity), rebind via NetConSave::weight_index2netcon. - double* w = weight_; - if (!w && weight_index_ >= 0) { + // Rebind via weight_index only (no NetCon::weight_ heap). + double* w = nullptr; + if (weight_index_ >= 0) { if (NetCon* owner = NetConSave::weight_index2netcon(weight_index_)) { - w = owner->weight_; - weight_ = w; + w = owner->weight_soa_data(); } } nrn_net_send(movable_, w, target_, tt, flag_); @@ -3545,7 +3552,7 @@ DiscreteEvent* SelfEvent::savestate_read(FILE* f) { // File identity is NetCon object index (not weight_ pointer). NetCon* nc = NetConSave::index2netcon(ncindex); assert(nc); - se->weight_ = nc->weight_; + se->weight_ = nullptr; se->weight_index_ = static_cast(nc->weight_base()); } se->flag_ = flag; @@ -3585,22 +3592,15 @@ void SelfEvent::savestate_write(FILE* f) { fprintf(f, "%d\n", SelfEventType); int const moff = movable_ ? (movable_ - target_->prop->dparam) : -1; int ncindex = -1; - // SelfEvent identity for SaveState: NetCon object index (stable), not weight_*. - // Prefer heap map; fall back to weight_index. Do not assert if unresolved — - // flag-only / ignored events use ncindex = -1 (BBSaveState uses -2 for ignore). + // SelfEvent identity for SaveState: NetCon object index via weight_index. NetCon* owner = nullptr; - if (weight_) { - owner = NetConSave::weight2netcon(weight_); - } - if (!owner && weight_index_ >= 0) { + if (weight_index_ >= 0) { owner = NetConSave::weight_index2netcon(weight_index_); + } else if (weight_) { + owner = NetConSave::weight2netcon(weight_); } if (owner && owner->obj_) { ncindex = owner->obj_->index; - // Rebind to long-lived heap if we resolved via index only. - if (!weight_ && owner->weight_) { - weight_ = owner->weight_; - } } fprintf(f, @@ -4371,14 +4371,12 @@ void NetCvode::fornetcon_prepare() { return; } fornetcon_change_cnt_ = structure_change_cnt; + g_fornet_base_to_buf.clear(); if (nrn_fornetcon_cnt_ == 0) { return; } int i, j; - // initialize a map from type to dparam index, -1 means no FOR_NETCONS statement std::vector t2i(n_memb_func, -1); - // create ForNetConsInfo in all the relevant point processes - // and fill in the t2i map. for (i = 0; i < nrn_fornetcon_cnt_; ++i) { int index = nrn_fornetcon_index_[i]; int type = nrn_fornetcon_type_[i]; @@ -4386,12 +4384,9 @@ void NetCvode::fornetcon_prepare() { if (nrn_is_artificial_[type]) { auto* const m = &memb_list[type]; for (j = 0; j < m->nodecount; ++j) { - // Save ForNetConsInfo* as void* to avoid needing to expose the - // definition of ForNetConsInfo to translated MOD file code void** v = &(m->pdata[j][index].literal_value()); _nrn_free_fornetcon(v); - ForNetConsInfo* fnc = new ForNetConsInfo; - *v = fnc; + *v = new ForNetConsInfo; } } else { for (NrnThread* nt: for_threads(nrn_threads, nrn_nthread)) @@ -4401,18 +4396,15 @@ void NetCvode::fornetcon_prepare() { for (j = 0; j < m->nodecount; ++j) { void** v = &(m->pdata[j][index].literal_value()); _nrn_free_fornetcon(v); - ForNetConsInfo* fnc = new ForNetConsInfo; - *v = fnc; + *v = new ForNetConsInfo; } } } } - // two loops over all netcons. one to count, one to fill bases + argslist - // count + // count peers per target if (psl_) for (const PreSyn* ps: *psl_) { - const NetConPList& dil = ps->dil_; - for (const auto& d1: dil) { + for (const auto& d1: ps->dil_) { Point_process* pnt = d1->target_; if (pnt && t2i[pnt->prop->_type] > -1) { auto* fnc = static_cast( @@ -4423,49 +4415,61 @@ void NetCvode::fornetcon_prepare() { } } - // allocate weight_bases + argslist; reset size for fill pass + auto allocate_slots = [](ForNetConsInfo* fnc, int type) { + if (!fnc || fnc->size <= 0) { + return; + } + int const n = fnc->size; + int const arity = pnt_receive_size[type] > 0 ? pnt_receive_size[type] : 1; + fnc->arity = arity; + fnc->weight_bases = new int[n]; + fnc->argslist = new double*[n]; + fnc->weight_storage = new double[static_cast(n) * static_cast(arity)]{}; + for (int s = 0; s < n; ++s) { + fnc->argslist[s] = fnc->weight_storage + static_cast(s) * arity; + fnc->weight_bases[s] = -1; + } + fnc->size = 0; // fill pass + }; + for (i = 0; i < nrn_fornetcon_cnt_; ++i) { int index = nrn_fornetcon_index_[i]; int type = nrn_fornetcon_type_[i]; if (nrn_is_artificial_[type]) { auto* const m = &memb_list[type]; for (j = 0; j < m->nodecount; ++j) { - auto* fnc = static_cast(m->pdata[j][index].get()); - if (fnc->size > 0) { - fnc->weight_bases = new int[fnc->size]; - fnc->argslist = new double*[fnc->size]; - fnc->size = 0; - } + allocate_slots(static_cast(m->pdata[j][index].get()), type); } } else { for (NrnThread* nt: for_threads(nrn_threads, nrn_nthread)) for (tml = nt->tml; tml; tml = tml->next) - if (tml->index == nrn_fornetcon_type_[i]) { + if (tml->index == type) { Memb_list* m = tml->ml; for (j = 0; j < m->nodecount; ++j) { - auto* fnc = static_cast( - m->pdata[j][index].get()); - if (fnc->size > 0) { - fnc->weight_bases = new int[fnc->size]; - fnc->argslist = new double*[fnc->size]; - fnc->size = 0; - } + allocate_slots( + static_cast(m->pdata[j][index].get()), + type); } } } } - // fill weight bases (SoA identity) and dual-write heap pointers + // fill bases + map base → scratch; materialize from SoA if (psl_) { for (const PreSyn* ps: *psl_) { - const NetConPList& dil = ps->dil_; - for (const auto& d1: dil) { + for (const auto& d1: ps->dil_) { Point_process* pnt = d1->target_; if (pnt && t2i[pnt->prop->_type] > -1) { auto* fnc = static_cast( pnt->prop->dparam[t2i[pnt->prop->_type]].get()); int const slot = fnc->size; - fnc->weight_bases[slot] = static_cast(d1->weight_base()); - fnc->argslist[slot] = d1->weight_; + int const base = static_cast(d1->weight_base()); + fnc->weight_bases[slot] = base; + double* buf = fnc->argslist[slot]; + if (base >= 0 && fnc->arity > 0) { + neuron::container::network::SelfEventFields::materialize_weight_block( + base, fnc->arity, buf); + g_fornet_base_to_buf[base] = buf; + } fnc->size = slot + 1; } } @@ -4474,7 +4478,6 @@ void NetCvode::fornetcon_prepare() { } int _nrn_netcon_args(void* v, double*** argslist) { - // Legacy ABI: still used if old generated code is linked; prefer bases API. auto* fnc = static_cast(v); assert(fnc); *argslist = fnc->argslist; @@ -4489,20 +4492,26 @@ int _nrn_netcon_weight_bases(void* v, int** bases) { } double* _nrn_fornetcon_weight(int weight_base) { - // Resolve SoA base → long-lived NetCon weight_ heap (MOD scratch / dual-write). if (weight_base < 0) { return nullptr; } - if (NetCon* nc = NetConSave::weight_index2netcon(weight_base)) { - return nc->weight_; + auto it = g_fornet_base_to_buf.find(weight_base); + if (it != g_fornet_base_to_buf.end()) { + return it->second; } return nullptr; } void _nrn_free_fornetcon(void** v) { if (auto* fnc = static_cast(*v); fnc) { + if (fnc->weight_bases && fnc->size > 0) { + for (int i = 0; i < fnc->size; ++i) { + g_fornet_base_to_buf.erase(fnc->weight_bases[i]); + } + } delete[] std::exchange(fnc->argslist, nullptr); delete[] std::exchange(fnc->weight_bases, nullptr); + delete[] std::exchange(fnc->weight_storage, nullptr); delete fnc; *v = nullptr; } @@ -5009,10 +5018,7 @@ NetCon* NetCvode::install_deliver(neuron::container::data_handle dsrc, ps_thread_link(ps); NetCon* d = new NetCon(ps, target); d->delay_ = delay; - d->weight_[0] = magnitude; - // Dual-write: constructor arg is written to heap; HOC weight[] reads SoA. - // INITIAL (init_events) does soa_to_heap then heap_to_soa — SoA must already - // hold magnitude or weight[0] is wiped to 0. + // HOC weight[] / deliver read Weight SoA only (no weight_ heap). if (d->has_weight_soa()) { d->weight_soa_value(0) = magnitude; } @@ -5101,7 +5107,6 @@ void DiscreteEvent::savestate_write(FILE* f) { void NetCon::allocate_weight_soa(double const* mirror) { weight_block_ = neuron::container::network::Weight::allocate_weight_block(cnt_, mirror); - // Authority: SoA base + count (CoreNEURON-shaped). if (weight_block_) { _soa.weight_index() = weight_block_->base_row(); _soa.weight_count() = weight_block_->size(); @@ -5111,18 +5116,20 @@ void NetCon::allocate_weight_soa(double const* mirror) { } } -void NetCon::weights_soa_to_heap() { - if (!weight_ || !has_weight_soa()) { - return; +double* NetCon::weight_soa_data() { + if (!has_weight_soa() || cnt_ <= 0) { + return nullptr; } - neuron::container::network::Weight::mirror_block_to_heap(*weight_block_, weight_, cnt_); + auto const base = weight_block_->base_row(); + if (base < 0) { + return nullptr; + } + return &neuron::model().weights().get( + static_cast(base)); } -void NetCon::weights_heap_to_soa() { - if (!weight_ || !has_weight_soa()) { - return; - } - neuron::container::network::Weight::mirror_heap_to_block(*weight_block_, weight_, cnt_); +double const* NetCon::weight_soa_data() const { + return const_cast(this)->weight_soa_data(); } void NetCon::soa_sync() { @@ -5130,23 +5137,18 @@ void NetCon::soa_sync() { _soa.active() = active_ ? 1 : 0; _soa.weight_count() = cnt_; if (target_) { - // Point_process dual-write row (Phase 1). _soa.target() = nrn_point_process_soa_row(target_); } else { _soa.target() = -1; } - // Phase 3: reverse edge into PreSyn SoA. if (src_) { _soa.src_presyn() = static_cast(src_->_soa.current_row()); } else { _soa.src_presyn() = -1; } - // Weight block base is SoA authority (refresh after permute/erase). if (has_weight_soa()) { _soa.weight_index() = weight_block_->base_row(); _soa.weight_count() = weight_block_->size(); - // Do not mirror heap → SoA here: HOC weight() writes SoA first; heap is only - // a pnt_receive buffer. heap→SoA after MOD runs (weights_heap_to_soa). } else { _soa.weight_index() = neuron::container::network::invalid_weight_index; } @@ -5166,9 +5168,7 @@ NetCon::NetCon(PreSyn* src, Object* target) { target_ = nullptr; active_ = false; cnt_ = 1; - weight_ = new double[cnt_]; - weight_[0] = 0.0; - allocate_weight_soa(weight_); + allocate_weight_soa(nullptr); soa_sync(); return; } @@ -5181,24 +5181,15 @@ NetCon::NetCon(PreSyn* src, Object* target) { hoc_execerror("No NET_RECEIVE in target PointProcess:", hoc_object_name(target)); } cnt_ = pnt_receive_size[target_->prop->_type]; - weight_ = nullptr; if (cnt_) { - weight_ = new double[cnt_]; - for (int i = 0; i < cnt_; ++i) { - weight_[i] = 0.0; - } - allocate_weight_soa(weight_); + allocate_weight_soa(nullptr); } soa_sync(); } NetCon::~NetCon() { - // printf("~NetCon\n"); NetConSave::invalid(); rmsrc(); - if (cnt_) { - delete[] weight_; - } weight_block_.reset(); // frees Weight SoA block rows // _soa owning_handle frees the NetCon SoA row. #if DISCRETE_EVENT_OBSERVER @@ -5285,6 +5276,15 @@ void NetConSave::invalid() { } NetCon* NetConSave::weight2netcon(double* pd) { + if (!pd) { + return nullptr; + } + // FOR_NETCONS scratch buffers registered during fornetcon_prepare. + for (auto const& kv: g_fornet_base_to_buf) { + if (kv.second == pd) { + return weight_index2netcon(kv.first); + } + } NetCon* nc; if (!wtable_) { hoc_Item* q; @@ -5293,19 +5293,18 @@ NetCon* NetConSave::weight2netcon(double* pd) { ITERATE(q, sym->u.ctemplate->olist) { Object* obj = OBJ(q); nc = (NetCon*) obj->u.this_pointer; - if (nc->weight_) { - (*wtable_)[nc->weight_] = nc; + if (nc) { + if (double* p = nc->weight_soa_data()) { + (*wtable_)[p] = nc; + } } } } auto wti = wtable_->find(pd); if (wti != wtable_->end()) { - nc = wti->second; - assert(nc->weight_ == pd); - return nc; - } else { - return nullptr; + return wti->second; } + return nullptr; } NetCon* NetConSave::index2netcon(long id) { diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index f2b39544d8..cf43dbfc80 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -359,6 +359,8 @@ void sort_network_data(neuron::cache::Model& cache, // Fanout order is NetCon SoA row indices; rebuild after topology/sort. ::PreSyn::mark_fanout_unsorted(); ::PreSyn::ensure_fanout_order(); + // Weight SoA data pointers may have moved; drop weight2netcon cache. + ::NetConSave::invalid(); // Final NetCon reverse-edge refresh (src PreSyn rows may have moved). for (::NetCon* nc: netcons) { diff --git a/src/nrniv/bbsavestate.cpp b/src/nrniv/bbsavestate.cpp index 27668781db..015875699a 100644 --- a/src/nrniv/bbsavestate.cpp +++ b/src/nrniv/bbsavestate.cpp @@ -182,6 +182,7 @@ callback to bbss_early when needed. #include #include #include +#include #include "netcon.h" #include "nrniv_mf.h" @@ -1113,11 +1114,7 @@ static bool selfevent_matches_netcon(SelfEvent const* se, NetCon const* nc) { if (!se || !nc) { return false; } - // Legacy: long-lived heap base pointer identity. - if (se->weight_ && nc->weight_ && se->weight_ == nc->weight_) { - return true; - } - // Dual-write / heap-drop: Weight SoA base row (weight_index). + // Weight SoA base row (heap-free identity). int se_widx = se->weight_index_; if (se_widx < 0) { return false; @@ -1126,7 +1123,7 @@ static bool selfevent_matches_netcon(SelfEvent const* se, NetCon const* nc) { return nc_widx == se_widx; } -/** Bind SelfEvent weight_ / weight_index_ from a NetCon (after BBSaveState restore). */ +/** Bind SelfEvent weight_index_ from a NetCon (after BBSaveState restore). */ static void selfevent_bind_netcon(SelfEvent* se, NetCon* nc) { if (!se) { return; @@ -1136,7 +1133,7 @@ static void selfevent_bind_netcon(SelfEvent* se, NetCon* nc) { se->weight_index_ = -1; return; } - se->weight_ = nc->weight_; + se->weight_ = nullptr; se->weight_index_ = static_cast(nc->weight_base()); } @@ -2236,17 +2233,26 @@ void BBSaveState::netrecv_pp(Point_process* pp) { f->s(buf, 1); for (; dl && dl->de->type() == NetConType; dl = dl->next) { NetCon* nc = (NetCon*) dl->de; - // Dual-write: HOC weight[] is SoA-primary. Materialize SoA → heap before - // OUT/CNT so f->d sees current values; after IN, mirror heap → SoA. - if (f->type() != BBSS_IO::IN) { - nc->weights_soa_to_heap(); - } - f->d(nc->cnt_, nc->weight_); - if (f->type() == BBSS_IO::IN) { - nc->weights_heap_to_soa(); + // Weight SoA only: use contiguous SoA data pointer when available. + double* wptr = nc->weight_soa_data(); + std::vector wtmp; + if (!wptr && nc->cnt_ > 0) { + wtmp.assign(static_cast(nc->cnt_), 0.); if (nc->has_weight_soa()) { - nc->soa_sync(); + int const m = std::min(nc->cnt_, nc->weight_block_->size()); + for (int i = 0; i < m; ++i) { + wtmp[static_cast(i)] = nc->weight_soa_value(i); + } + } + wptr = wtmp.data(); + } + f->d(nc->cnt_, wptr); + if (f->type() == BBSS_IO::IN && nc->has_weight_soa() && wptr) { + int const m = std::min(nc->cnt_, nc->weight_block_->size()); + for (int i = 0; i < m; ++i) { + nc->weight_soa_value(i) = wptr[i]; } + nc->soa_sync(); } if (f->type() != BBSS_IO::IN) { // writing, counting DblList* db = 0; diff --git a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp index 64e10d02ef..76cfa97c88 100644 --- a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp +++ b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp @@ -141,10 +141,13 @@ void nrnthreads_all_weights_return(std::vector& weights) { ith = std::size_t(((NrnThread*) (nc->target_->_vnt))->id); } for (int i = 0; i < nc->cnt_; ++i) { - nc->weight_[i] = weights[ith][iw[ith]++]; + if (nc->has_weight_soa() && i < nc->weight_block_->size()) { + nc->weight_soa_value(i) = weights[ith][iw[ith]++]; + } else { + ++iw[ith]; + } } - // Dual-write: HOC weight[] is SoA-primary; keep Weight SoA in sync. - nc->weights_heap_to_soa(); + nc->soa_sync(); } } @@ -510,10 +513,10 @@ int nrnthread_dat2_3(int tid, int iw = 0; for (int i = 0; i < n; ++i) { NetCon* nc = cg.netcons[i]; - // Dual-write: HOC may have written Weight SoA only; materialize heap. - nc->weights_soa_to_heap(); for (int j = 0; j < nc->cnt_; ++j) { - weights[iw++] = nc->weight_[j]; + weights[iw++] = (nc->has_weight_soa() && j < nc->weight_block_->size()) + ? nc->weight_soa_value(j) + : 0.; } } // alloc a delay array and write netcon delays @@ -936,7 +939,13 @@ static void set_info(TQItem* tqi, Point_process* pnt = se->target_; int type = pnt->prop->_type; int movable_index = type2movable[type]; + // Heap-free: prefer weight_index → SoA data pointer for NetCon match. double* wt = se->weight_; + if (!wt && se->weight_index_ >= 0) { + if (NetCon* nc = NetConSave::weight_index2netcon(se->weight_index_)) { + wt = nc->weight_soa_data(); + } + } core_te->intdata.push_back(type); core_te->dbldata.push_back(se->flag_); @@ -1107,12 +1116,13 @@ NrnCoreTransferEvents* nrn2core_transfer_tqueue(int tid) { assert(iter.second[0] >= NRN_SENTINAL); } - // NEURON SelfEvent weight* into CoreNEURON index into nt.netcons - // On the CoreNEURON side we find the NetCon and then the - // nc.u.weight_index_ + // NEURON SelfEvent weight* / SoA data into CoreNEURON index into nt.netcons for (int i = 0; i < cg.n_netcon; ++i) { NetCon* nc = cg.netcons[i]; - double* wt = nc->weight_; + double* wt = nc->weight_soa_data(); + if (!wt) { + continue; + } auto iter = weight2intdata.find(wt); if (iter != weight2intdata.end()) { for (auto iloc: iter->second) { @@ -1199,7 +1209,7 @@ void core2nrn_SelfEvent_event(int tid, assert(nc->target_ == pnt); #endif - double* weight = nc->weight_; + double* weight = nc->weight_soa_data(); core2nrn_SelfEvent_helper(tid, td, tar_type, tar_index, flag, weight, is_movable); } diff --git a/src/nrniv/prcellstate.cpp b/src/nrniv/prcellstate.cpp index 840d22f306..26ab071b67 100644 --- a/src/nrniv/prcellstate.cpp +++ b/src/nrniv/prcellstate.cpp @@ -76,7 +76,12 @@ static void pr_netcon(NrnThread& nt, FILE* f, const std::map& pnt2in } int wcnt = pnt_receive_size[nc->target_->prop->_type]; for (int k = 0; k < wcnt; ++k) { - fprintf(f, " %.*g", precision, nc->weight_[k]); + fprintf(f, + " %.*g", + precision, + nc->has_weight_soa() && k < nc->weight_block_->size() + ? nc->weight_soa_value(k) + : 0.); } fprintf(f, "\n"); } diff --git a/src/nrniv/savstate.cpp b/src/nrniv/savstate.cpp index 597305d995..1ed24c2d94 100644 --- a/src/nrniv/savstate.cpp +++ b/src/nrniv/savstate.cpp @@ -931,18 +931,14 @@ void SaveState::savenet() { const NetCon* d = (NetCon*) ob->u.this_pointer; int n = ncs_[i].nstate; double* w = ncs_[i].state; - // HOC weight[] is SoA-primary under dual-write; prefer SoA over heap. + // Weight SoA only (heap-free step 6). if (d->has_weight_soa()) { int const m = std::min(n, d->weight_block_->size()); for (int j = 0; j < m; ++j) { w[j] = d->weight_soa_value(j); } for (int j = m; j < n; ++j) { - w[j] = d->weight_ ? d->weight_[j] : 0.; - } - } else if (d->weight_) { - for (int j = 0; j < n; ++j) { - w[j] = d->weight_[j]; + w[j] = 0.; } } ++i; @@ -987,19 +983,13 @@ void SaveState::restorenet() { NetCon* d = (NetCon*) ob->u.this_pointer; int n = ncs_[i].nstate; const double* w = ncs_[i].state; - // Restore into heap buffer (nocmodl / FOR_NETCONS) and Weight SoA so HOC - // dual-write and materialize-on-deliver stay consistent. - if (d->weight_) { - for (int j = 0; j < n; ++j) { - d->weight_[j] = w[j]; - } - } + // Restore into Weight SoA only (heap-free step 6). if (d->has_weight_soa()) { int const m = std::min(n, d->weight_block_->size()); for (int j = 0; j < m; ++j) { d->weight_soa_value(j) = w[j]; } - d->soa_sync(); // WeightIndex / reverse edges + d->soa_sync(); } ++i; } diff --git a/src/nrnoc/nrn_ansi.h b/src/nrnoc/nrn_ansi.h index dec4376c3e..61fc9315d2 100644 --- a/src/nrnoc/nrn_ansi.h +++ b/src/nrnoc/nrn_ansi.h @@ -30,11 +30,11 @@ extern "C" void modl_reg(void); // nrnmech stuff extern void _nrn_free_fornetcon(void**); -/** @brief Legacy FOR_NETCONS: return double** list of NetCon weight_ heaps. */ +/** @brief Legacy FOR_NETCONS: return double** list of owned peer scratch. */ extern int _nrn_netcon_args(void*, double***); -/** @brief FOR_NETCONS by Weight SoA base list (heap-free step 4). */ +/** @brief FOR_NETCONS by Weight SoA base list. */ extern int _nrn_netcon_weight_bases(void*, int**); -/** @brief Resolve weight SoA base → long-lived weight_ for MOD body. */ +/** @brief Resolve weight SoA base → FOR_NETCONS owned scratch for MOD body. */ extern double* _nrn_fornetcon_weight(int weight_base); extern double nrn_call_mech_func(Symbol*, int narg, Prop*, int type); From fd03e70821001c8f4c9a43c42244bc5d7d24d48e Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 11:14:53 -0400 Subject: [PATCH 26/33] =?UTF-8?q?Docs:=20two-epoch=20north=20star=20and=20?= =?UTF-8?q?next=20phase=20after=20heap-free=201=E2=80=936?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Capture sim vs edit epochs: CoreNEURON-shaped hot path plus packing A and reconfigurable nthread; full NEURON construction between runs. Mark steps 1–6 done; outline 6b zero-copy and 7 nocmodl index ABI as the next phase. --- AGENTS.md | 2 +- GROK-NETWORK-SOA.md | 19 ++++--- doc/network-soa/README.md | 21 +++++--- doc/network-soa/dual-write-and-heap.md | 13 +---- doc/network-soa/heap-free.md | 68 ++++++++++++++++++++------ 5 files changed, 81 insertions(+), 42 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 907679bfd8..0dc758b825 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -23,7 +23,7 @@ Prefer CPU-only or default GPU-off builds until integration explicitly needs GPU - Network SoA: `Point_process`, `NetCon`, `PreSyn`, `weights`, `SelfEvent` in `neuron::container` style. - HOC wrappers as permutation-stable handles over backing store — **not** a second pointer graph. -- **Heap-free branch:** follow CoreNEURON as much as feasible; charter in `doc/network-soa/heap-free.md`. +- **Heap-free branch:** CoreNEURON-shaped sim path **plus** host packing / dynamic `nthread` between runs; full NEURON edit epoch. Charter: `doc/network-soa/heap-free.md`. - **Out of scope:** Stage 2/3 GPU `net_buf_receive`, ringtest GPU network buffers (resume after SoA merges to master). ## Execute, don’t delegate diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index c4f4a9c4da..2d18f357f4 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -54,11 +54,14 @@ One CPU backing store for integration-hot network data: - `NrnThread` = lightweight slice (`offset` + count) — **no duplicate CPU copies** - HOC extras in sidecars keyed by handle -CoreNEURON layout is the **integration reference**. +**Two epochs:** **Sim** = CoreNEURON-shaped hot path (indices, packed weights, frozen connectivity) +**plus** host improvements CN may lack (physical weight packing **A**, rebuild after `nthread` change). +**Edit** = full NEURON construction between runs. CoreNEURON is the reference shape, not a ceiling. +Detail: `doc/network-soa/heap-free.md`. --- -## Architecture (current dual-write) +## Architecture ```text HOC / Python @@ -70,7 +73,7 @@ Legacy shells (Point_process*, NetCon, PreSyn) + owning_handle _soa neuron::container::network::{PointProcess,Weight,NetCon,PreSyn} │ ▼ -Hot path: fanout order + weight_index → materialize → pnt_receive(double*) +Hot path: fanout indices + weight_index → (TLS/FOR_NETCONS scratch or SoA double*) → pnt_receive ``` Key paths: @@ -80,8 +83,8 @@ Key paths: | Containers | `src/neuron/container/network/*.hpp` | | Model ownership | `src/neuron/model_data.hpp` | | PP dual-write | `Point_process::_soa`, `nrn_point_process_soa_sync` | -| NetCon / weights | `NetCon::_soa`, `weight_soa_`, `soa_sync` | -| Fanout | `PreSyn::ensure_fanout_order`, global NetCon* order | +| NetCon / weights | `NetCon::_soa`, `weight_block_`, `weight_soa_data()` (no `weight_` heap on heap-free) | +| Fanout | `PreSyn::ensure_fanout_order`, `g_network_fanout_order` (`netcon_index_t`) | | Sort / repack | `network_soa_sort.cpp`, `sort_network_data` in ensure_sorted | | Receive by index | `nrn_pnt_receive_by_weight_index` | @@ -115,9 +118,9 @@ Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep mate ## Recommended next work -1. ~~Sort + SaveState + BBSaveState dual-write~~ (done). -2. ~~PR dual-write green; heap-free charter~~ (`doc/network-soa/heap-free.md`, branch `local/cpu-net-soa-heap-free`). -3. **On heap-free branch:** O(1) NetCon weight-block ownership (SoA base+count); index fanout tables; FOR_NETCONS codegen; drop `weight_` when green. +1. ~~Sort + SaveState + BBSaveState dual-write~~ (done on PR branch). +2. ~~Heap-free steps 1–6~~ (done on `local/cpu-net-soa-heap-free`: no `NetCon::weight_`; charter in `doc/network-soa/heap-free.md`). +3. **Next phase on heap-free:** **6b** zero-copy `pnt_receive` via contiguous `weight_soa_data()`; then **7** nocmodl index ABI (`soaweight[ix+k]`) and drop remaining scratch pools. 4. Keep PR tip rebased on master and green; rebase heap-free onto PR tip after refreshes. 5. GPU Phase 5 only after SoA shape is stable enough to upload the same columns. diff --git a/doc/network-soa/README.md b/doc/network-soa/README.md index 53cdbdc7cc..53954c84cd 100644 --- a/doc/network-soa/README.md +++ b/doc/network-soa/README.md @@ -10,8 +10,8 @@ Docs here are **segregated by height**. Prefer the shortest document that answer |-------|-----|------------------| | **L0 — North star** | this file §North star | One screen: goals, non-goals, current posture | | **L1 — Topology** | [topology.md](topology.md) | PreSyn → NetCon → Point_process; CoreNEURON mapping; fanout authority; rank/thread placement | -| **L2 — Dual-write & heap** | [dual-write-and-heap.md](dual-write-and-heap.md) | What is primary where; why `weight_` still exists; FOR_NETCONS / INITIAL / SaveState | -| **L2b — Heap-free charter** | [heap-free.md](heap-free.md) | Settled decisions; CoreNEURON-as-feasible; queue/structure policy; implementation order | +| **L2 — Dual-write & heap** | [dual-write-and-heap.md](dual-write-and-heap.md) | Dual-write history; PR branch policy | +| **L2b — Heap-free charter** | [heap-free.md](heap-free.md) | Two-epoch north star; settled decisions; steps 1–6 done; next phase 6b–7 | | **L3 — Sort & packing** | [sort-and-packing.md](sort-and-packing.md) | What `nrn_ensure_model_data_are_sorted` does to network containers | | **L4 — Spec detail** | `../network-soa-phase0.md` | Field tags, sidecars, SaveState notes, open questions resolved | | **Ops** | `../../GROK-NETWORK-SOA.md` | Build/test gates, commit status, “what next” | @@ -20,17 +20,26 @@ Docs here are **segregated by height**. Prefer the shortest document that answer ## North star (L0) -**Goal:** One CPU backing store for *integration-hot* network data (columns + indices), with HOC wrappers as permutation-stable handles — same *spirit* as node/mechanism SoA and CoreNEURON’s per-thread flat arrays. +**Goal:** One CPU backing store for *integration-hot* network data (columns + indices), with HOC wrappers as permutation-stable handles. + +**Two epochs:** + +| Epoch | Shape | Flexibility | +|-------|--------|-------------| +| **Sim** | CoreNEURON-shaped (indices, packed weights, frozen connectivity for the run) **plus** host improvements CN may not have (physical weight packing **A**, rebuild after `nthread` change) | No mid-run topology thrash | +| **Edit** | Full NEURON construction/modification between runs | Create/destroy, reconnect, `pc.nthread`, re-sort | + +CoreNEURON is the **integration reference and hot-path shape**, not a performance ceiling. See [heap-free.md](heap-free.md) §North star. **Integration reference:** CoreNEURON (`NrnThread::{pntprocs,netcons,weights,presyns}`, `PreSyn::{nc_index_,nc_cnt_}`, `NetCon::u.weight_index_`). -**Not the first PR’s job (unless we change strategy):** +**Not the dual-write PR’s job:** -- Full free of `NetCon::weight_` under default **nocmodl** ABI - GPU net buffers (sibling track after SoA is usable) - Thin remote `InputPreSyn` / gid→fanout table +- nocmodl native `weight_index` receive (heap-free **next phase** 6b–7) -**Current posture (2026-07):** Dual-write complete on `local/cpu-network-soa` (PR #3822); network sort + SaveState/BBSaveState dual-write green. **`weight_` remains long-lived MOD scratch** on the PR branch. Heap-free work lives on **`local/cpu-net-soa-heap-free`** (no PR): follow CoreNEURON as much as feasible — see [heap-free.md](heap-free.md). +**Current posture (2026-07):** Dual-write on `local/cpu-network-soa` (PR #3822). Heap-free on **`local/cpu-net-soa-heap-free`**: steps 1–6 done (`NetCon::weight_` removed; TLS/FOR_NETCONS scratch remain as MOD bridges). Next: zero-copy SoA `double*` and/or nocmodl index ABI — [heap-free.md](heap-free.md). **PR vs master:** Dual-write PR can gather review without claiming “land for perf.” Heap-free stays private until a green, reviewable slice exists. diff --git a/doc/network-soa/dual-write-and-heap.md b/doc/network-soa/dual-write-and-heap.md index c0fab224c1..51c309c0cf 100644 --- a/doc/network-soa/dual-write-and-heap.md +++ b/doc/network-soa/dual-write-and-heap.md @@ -34,18 +34,9 @@ Passing a **temporary** buffer into `pnt_receive` broke SelfEvent identity: nocm ## 4. Roadmap to full heap free -**Active branch:** `local/cpu-net-soa-heap-free` (no PR). Settled charter: [heap-free.md](heap-free.md). +**Active branch:** `local/cpu-net-soa-heap-free` (no PR). Charter + two-epoch north star: [heap-free.md](heap-free.md). -Rough commit series (each gateable): - -1. **SelfEvent never stores identity-only-as-temp-pointer** — always set `weight_index_` (done in dual-write spirit); never pass non-owner buffers into `pnt_receive` if MOD can net_send. -2. **O(1) NetCon** — one weight block (base+arity on SoA); no per-arg owning handle vector on the shell. -3. **FOR_NETCONS** — packing **A** (target-instance adjacency) + nocmodl index walk; stop requiring foreign NetCon `weight_` bases. -4. **INITIAL** — `pnt_receive_init` by index or materialize into ephemeral/owner scratch only. -5. **Stop allocating `new double[cnt_]`** — optional thread-local scratch of `max(pnt_receive_size)` for MOD only if ABI still needs `double*`. -6. **Delete `weight_` field** — after tests (stdp/FOR_NETCONS, SaveState ring, netrec init, ctest + asan). - -Structure change / NetCon free with a live network queue: **clear queue** (freeze after `finitialize`). See heap-free charter. +**Steps 1–6 done** on that branch (`NetCon::weight_` removed). Remaining: zero-copy SoA `double*` (6b), then nocmodl **weight_index** receive (7) to drop TLS/FOR_NETCONS scratch. Structure change still clears the queue; edit between runs stays fully NEURON-flexible. --- diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index c6f37a3ab5..11c1647435 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -3,14 +3,40 @@ **Branch only — no PR** until a reviewable slice is green. **Base:** keep rebasing onto green `local/cpu-network-soa` (PR #3822) after master merges. -**Charter:** Follow CoreNEURON as much as feasible for the data plane and hot path. -Interpreter, structure change, and queue polymorphism stay NEURON-specific (policies below). +--- + +## North star (two epochs) + +**Sim epoch (CoreNEURON-shaped *plus* host improvements):** +Flat/packed weights, integer bases, source fanout ranges, frozen connectivity for the run. +CoreNEURON is the **integration reference and hot-path shape**, not a performance ceiling +and not a product constraint on NEURON’s lifecycle. + +Improvements CoreNEURON does not (or only partially) provide, that we intentionally pursue: + +| Improvement | Notes | +|-------------|--------| +| **Weight physical packing (A)** | Sort by target thread → target PP instance → …; stronger locality than CN’s construction-order pool + FOR_NETCONS perm | +| **Dynamic `nthread` between runs** | Rebuild sort, thread offsets, fanout, FOR_NETCONS tables — CN setup is fixed | +| **HOC/Python edit between runs** | Handles survive permute; append/invalidate then re-sort | + +**Edit epoch (full NEURON flexibility):** +Create/destroy NetCons, reconnect, change `pc.nthread`, rebuild model. Not mid-`psolve` +with a live queue (structure change clears the queue). + +One line: + +> **Build and edit like NEURON; once the run starts, look like CoreNEURON’s network data plane +> (with better host packing and reconfigurable threads)—then tear down and edit again.** + +**Charter (implementation):** sim path stays index- and SoA-first; interpreter, structure change, +and queue polymorphism stay NEURON-specific (policies below). --- ## Settled decisions -### Data plane (CoreNEURON-shaped) +### Data plane (CoreNEURON-shaped + packing A) | Item | Choice | |------|--------| @@ -38,37 +64,47 @@ Interpreter, structure change, and queue polymorphism stay NEURON-specific (poli | TQueue | Keep `(tdeliver, DiscreteEvent*)` for now; compress later if needed | | GPU | Out of scope on this branch | -### Explicitly not required for v1 +### Explicitly not required for heap-free v1 (steps 1–6) - Arbitrary model edits between `pc.psolve()` with a live queue - Queue scrub or refcount-pin on every NetCon destroy -- Full nocmodl ABI change to `pnt_receive(…, weight_index, flag)` before materialize shim is removable +- Full nocmodl ABI change to `pnt_receive(…, weight_index, flag)` (next phase) - Permanent FOR_NETCONS perm table **if** sort key already groups by target instance (**A**) --- ## Contract (identity) -> A NetCon weight block is identified by a base row in the Weight SoA (arity fixed by target mechanism type). Queued SelfEvents and generated `net_send` carry that base as `int` (or −1). FOR_NETCONS never walks foreign `double*`; it walks a target-local list of bases, contiguous after sort under packing **A**. Per-NetCon `weight_` heap is MOD scratch only until codegen/tests allow deletion. +> A NetCon weight block is identified by a base row in the Weight SoA (arity fixed by target mechanism type). Queued SelfEvents and generated `net_send` carry that base as `int` (or −1). FOR_NETCONS walks a target-local list of bases (owned scratch for MOD `double*`, writeback to SoA). There is **no** per-NetCon `weight_` heap. --- ## Implementation order (gateable) -1. **Docs + index typedefs** — this file; `weight_index_t` / `netcon_index_t`. -2. **O(1) ownership path** — `WeightBlock` off-shell (`unique_ptr`); NetCon SoA base+count authority; HOC `weight_soa_handle(i)`. -3. **Fanout tables as indices** — `g_network_fanout_order` holds `netcon_index_t` (SoA rows); resolve via `g_netcon_by_soa_row`. +### Done on this branch (steps 1–6) + +1. **Docs + index typedefs** — `weight_index_t` / `netcon_index_t`. +2. **O(1) ownership path** — `WeightBlock` off-shell; SoA base+count authority; HOC handles. +3. **Fanout tables as indices** — `g_network_fanout_order` = SoA rows; `g_netcon_by_soa_row` resolve. +4. **nocmodl FOR_NETCONS by bases** — `_nrn_netcon_weight_bases` / `_nrn_fornetcon_weight`; packing A sort key. +5. **Ephemeral MOD scratch** — TLS for non-FOR_NETCONS; active `weight_index` so scratch is never queue identity. +6. **Delete `NetCon::weight_`** — SoA only; FOR_NETCONS per-target scratch; `weight_soa_data()` for legacy double* APIs. -4. **nocmodl FOR_NETCONS** — walk Weight SoA bases (`_nrn_netcon_weight_bases` / - `_nrn_fornetcon_weight`); sort packs by target instance (packing A). +### Process -5. **Ephemeral MOD scratch only** — thread-local buffer for non-FOR_NETCONS - `pnt_receive`; TLS active `weight_index` so `net_send(..., _w)` never queues - scratch as identity. -6. **Delete `NetCon::weight_`** — SoA only; FOR_NETCONS owns per-target scratch - buffers keyed by weight bases; `weight_soa_data()` for contiguous double* APIs. 7. Rebase onto green PR tip after each PR↔master refresh. +### Next phase (after 1–6) — sim path closer to index-native CoreNEURON, still NEURON edit epoch + +| Substep | Scope | Goal | +|---------|--------|------| +| **6b — Zero-copy host receive** | Where block is contiguous, pass `&WeightSoA[base]` into `pnt_receive(double*)` instead of TLS copy | Drop copy cost without nocmodl rewrite; identity stays `weight_index` | +| **7a — nocmodl index ABI** | Generate `NET_RECEIVE` with base index; body names as `soaweight[ix+k]` / `double&` refs | Eliminate primary-edge scratch; CN-shaped interface | +| **7b — FOR_NETCONS without owned double pool** | Loop bases only; write SoA directly (or single shared view) | Drop `ForNetConsInfo::weight_storage` | +| **7c — Cleanup** | Remove TLS scratch + base→buf maps when unused | Host path fully SoA/index | + +Optional later: GPU upload of the same packed columns (sibling track), not a prerequisite for 6b–7. + --- ## Workflow From 220c4144b48d0884066a596371aa5cb45e6f54b6 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 11:21:56 -0400 Subject: [PATCH 27/33] Heap-free 6b: zero-copy pnt_receive into contiguous Weight SoA Prefer data_if_contiguous() / weight_soa_data() for non-FOR_NETCONS deliver and NET_RECEIVE INITIAL so MOD writes SoA in place. TLS materialize remains only when the block is scattered (post-erase, pre-sort). SelfEvent identity stays weight_index. Unit test covers contiguous zero-copy writes. --- GROK-NETWORK-SOA.md | 2 +- doc/network-soa/heap-free.md | 8 +-- src/neuron/container/network/weight_block.hpp | 28 ++++++++++ src/nrncvode/netcon.h | 7 ++- src/nrncvode/netcvode.cpp | 55 +++++++++++++------ test/unit_tests/container/network.cpp | 9 +++ 6 files changed, 85 insertions(+), 24 deletions(-) diff --git a/GROK-NETWORK-SOA.md b/GROK-NETWORK-SOA.md index 2d18f357f4..d346711bb5 100644 --- a/GROK-NETWORK-SOA.md +++ b/GROK-NETWORK-SOA.md @@ -120,7 +120,7 @@ Native nocmodl `weight_index` ABI is **out of scope** for this branch; keep mate 1. ~~Sort + SaveState + BBSaveState dual-write~~ (done on PR branch). 2. ~~Heap-free steps 1–6~~ (done on `local/cpu-net-soa-heap-free`: no `NetCon::weight_`; charter in `doc/network-soa/heap-free.md`). -3. **Next phase on heap-free:** **6b** zero-copy `pnt_receive` via contiguous `weight_soa_data()`; then **7** nocmodl index ABI (`soaweight[ix+k]`) and drop remaining scratch pools. +3. **Next phase on heap-free:** ~~**6b** zero-copy contiguous SoA `double*`~~ (done); **7** nocmodl index ABI (`soaweight[ix+k]`) and drop remaining scratch pools. 4. Keep PR tip rebased on master and green; rebase heap-free onto PR tip after refreshes. 5. GPU Phase 5 only after SoA shape is stable enough to upload the same columns. diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index 11c1647435..f5d4746f65 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -94,16 +94,16 @@ and queue polymorphism stay NEURON-specific (policies below). 7. Rebase onto green PR tip after each PR↔master refresh. -### Next phase (after 1–6) — sim path closer to index-native CoreNEURON, still NEURON edit epoch +### Next phase — sim path closer to index-native CoreNEURON, still NEURON edit epoch | Substep | Scope | Goal | |---------|--------|------| -| **6b — Zero-copy host receive** | Where block is contiguous, pass `&WeightSoA[base]` into `pnt_receive(double*)` instead of TLS copy | Drop copy cost without nocmodl rewrite; identity stays `weight_index` | +| **6b — Zero-copy host receive** | **Done:** contiguous block → `pnt_receive` via `data_if_contiguous()` / `weight_soa_data()`; TLS materialize only if scattered | Drop copy when packed; identity stays `weight_index` | | **7a — nocmodl index ABI** | Generate `NET_RECEIVE` with base index; body names as `soaweight[ix+k]` / `double&` refs | Eliminate primary-edge scratch; CN-shaped interface | -| **7b — FOR_NETCONS without owned double pool** | Loop bases only; write SoA directly (or single shared view) | Drop `ForNetConsInfo::weight_storage` | +| **7b — FOR_NETCONS without owned double pool** | Loop bases only; write SoA directly (or one shared view) | Drop `ForNetConsInfo::weight_storage` | | **7c — Cleanup** | Remove TLS scratch + base→buf maps when unused | Host path fully SoA/index | -Optional later: GPU upload of the same packed columns (sibling track), not a prerequisite for 6b–7. +Optional later: GPU upload of the same packed columns (sibling track), not a prerequisite for 7. --- diff --git a/src/neuron/container/network/weight_block.hpp b/src/neuron/container/network/weight_block.hpp index 5cb57f48ea..c4f59b7bb3 100644 --- a/src/neuron/container/network/weight_block.hpp +++ b/src/neuron/container/network/weight_block.hpp @@ -62,6 +62,34 @@ struct WeightBlock { assert(i >= 0 && i < size()); return rows[static_cast(i)].value_handle(); } + + /** + * @brief Pointer to first value if rows are contiguous in storage (base..base+n-1). + * + * True after allocate and after network weight sort; may be false after + * unrelated erases until the next repack. Used for zero-copy pnt_receive (6b). + */ + [[nodiscard]] double* data_if_contiguous() { + if (rows.empty()) { + return nullptr; + } + auto& store = neuron::model().weights(); + auto const r0 = rows.front().current_row(); + auto const n = rows.size(); + if (r0 + n > store.size()) { + return nullptr; + } + for (std::size_t i = 1; i < n; ++i) { + if (!rows[i].id() || rows[i].current_row() != r0 + i) { + return nullptr; + } + } + return &store.get(r0); + } + + [[nodiscard]] double const* data_if_contiguous() const { + return const_cast(this)->data_if_contiguous(); + } }; /** diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index 11882c813a..d41bb7c8c3 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -148,9 +148,10 @@ class NetCon: public DiscreteEvent { /** * @brief Pointer to first Weight SoA value of this edge (cnt_ consecutive rows). * - * Valid while the model is not mid-permute and the block remains contiguous - * (true after allocate and after network sort). Used for CoreNEURON export - * and legacy double* APIs — not for SelfEvent identity. + * Valid while the block remains contiguous (after allocate / network sort). + * May be null if rows are scattered (post-erase, pre-sort). Used for + * zero-copy pnt_receive (6b), CoreNEURON export, legacy double* APIs — + * not for SelfEvent identity. */ [[nodiscard]] double* weight_soa_data(); [[nodiscard]] double const* weight_soa_data() const; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 1cfd655f3a..fa73e62a2e 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2426,15 +2426,37 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, return; } - // Non-FOR_NETCONS: thread-local scratch only (never SelfEvent identity). + // Non-FOR_NETCONS (heap-free 6b): prefer zero-copy into contiguous Weight SoA. + // Fallback TLS materialize if block is scattered (post-erase, pre-sort). + // Never treat buf as SelfEvent identity (tls_guard + nrn_net_send policy). if (weight_index < 0 || n <= 0) { POINT_RECEIVE(type, pnt, nullptr, flag); return; } - double* buf = mod_weight_scratch(n); - neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, n, buf); + double* buf = nullptr; + bool zero_copy = false; + if (NetCon* owner = NetConSave::weight_index2netcon(weight_index)) { + if (owner->has_weight_soa()) { + buf = owner->weight_block_->data_if_contiguous(); + zero_copy = (buf != nullptr); + } + } + if (!buf && weight_heap) { + // Caller-supplied buffer (e.g. SoA data pointer already resolved). + buf = weight_heap; + zero_copy = true; // assume caller points at durable SoA or fornet scratch + } + if (!buf) { + buf = mod_weight_scratch(n); + neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, + n, + buf); + zero_copy = false; + } POINT_RECEIVE(type, pnt, buf, flag); - neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); + if (!zero_copy) { + neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); + } } // for threads, revised net_send to use absolute time (in the @@ -3002,13 +3024,18 @@ void NetCvode::init_events() { int const n = d->cnt_; if (pnt_receive_init[type]) { double* buf = nullptr; - if (widx >= 0 && n > 0) { + bool zero_copy = false; + if (d->has_weight_soa()) { + buf = d->weight_block_->data_if_contiguous(); + zero_copy = (buf != nullptr); + } + if (!buf && widx >= 0 && n > 0) { buf = mod_weight_scratch(n); neuron::container::network::SelfEventFields::materialize_weight_block( widx, n, buf); } (*pnt_receive_init[type])(d->target_, buf, 0); - if (widx >= 0 && n > 0 && buf) { + if (!zero_copy && widx >= 0 && n > 0 && buf) { neuron::container::network::SelfEventFields::store_weight_block(widx, n, buf); } } else if (d->has_weight_soa()) { @@ -3194,13 +3221,13 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); - // Heap-free step 5: deliver by weight_index; MOD sees ephemeral scratch - // (or FOR_NETCONS long-lived heaps). SelfEvent identity is weight_index. + // Heap-free 6b: deliver by weight_index; prefer contiguous SoA double*. int widx = _soa.weight_index(); if (widx < 0) { widx = static_cast(weight_base()); } - nrn_pnt_receive_by_weight_index(target_, widx, 0., /*weight_heap*/ nullptr); + double* wptr = has_weight_soa() ? weight_block_->data_if_contiguous() : nullptr; + nrn_pnt_receive_by_weight_index(target_, widx, 0., wptr); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3220,7 +3247,8 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { if (widx < 0) { widx = static_cast(weight_base()); } - nrn_pnt_receive_by_weight_index(target_, widx, 0., /*weight_heap*/ nullptr); + double* wptr = has_weight_soa() ? weight_block_->data_if_contiguous() : nullptr; + nrn_pnt_receive_by_weight_index(target_, widx, 0., wptr); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -5120,12 +5148,7 @@ double* NetCon::weight_soa_data() { if (!has_weight_soa() || cnt_ <= 0) { return nullptr; } - auto const base = weight_block_->base_row(); - if (base < 0) { - return nullptr; - } - return &neuron::model().weights().get( - static_cast(base)); + return weight_block_->data_if_contiguous(); } double const* NetCon::weight_soa_data() const { diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index 2cf8bd5890..edf12887e3 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -205,6 +205,15 @@ TEST_CASE("SOA-backed Weight structure", "[Neuron][data_structures][network][wei REQUIRE(block->value(2) == 3.5); REQUIRE(block->base_row() >= 0); } + THEN("data_if_contiguous supports zero-copy pnt_receive (6b)") { + double* p = block->data_if_contiguous(); + REQUIRE(p != nullptr); + REQUIRE(p[0] == 1.5); + REQUIRE(p[1] == 2.5); + REQUIRE(p[2] == 3.5); + p[1] = 7.0; + REQUIRE(block->value(1) == 7.0); + } WHEN("heap is updated and remirrored") { heap[1] = 9.0; Weight::mirror_heap_to_block(*block, heap, 3); From 6aa7a10446f7d6c2770101c27a97d0d6bd7885e4 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 11:53:55 -0400 Subject: [PATCH 28/33] Heap-free 7a: pnt_receive / net_send weight_index ABI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change pnt_receive_t and pnt_receive_init_t to (Point_process*, int weight_index, double flag). Generated NET_RECEIVE (nocmodl + NMODL neuron) keeps _args[i] via _nrn_netrec_wsoa / _nrn_netrec_wsoa_done; net_send / artcell_net_send take the SoA base index (−1 if none). Always commit wsoa_done so NET_RECEIVE INITIAL values reach Weight SoA when fornetcon_prepare has already mapped FOR_NETCONS scratch. Verified: build 225/225, build-asan 223/223. --- doc/network-soa/heap-free.md | 3 +- .../codegen/codegen_neuron_cpp_visitor.cpp | 15 +- src/nocmodl/nocpout.cpp | 39 ++- src/nocmodl/parsact.cpp | 5 +- src/nrncvode/netcvode.cpp | 225 +++++++----------- src/nrncvode/netcvode.h | 2 +- src/nrniv/bbsavestate.cpp | 6 +- .../callbacks/nrncore_callbacks.cpp | 12 +- src/nrniv/savstate.cpp | 3 +- src/nrnoc/nrniv_mf.h | 30 +-- 10 files changed, 160 insertions(+), 180 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index f5d4746f65..6c643c46b2 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -68,7 +68,6 @@ and queue polymorphism stay NEURON-specific (policies below). - Arbitrary model edits between `pc.psolve()` with a live queue - Queue scrub or refcount-pin on every NetCon destroy -- Full nocmodl ABI change to `pnt_receive(…, weight_index, flag)` (next phase) - Permanent FOR_NETCONS perm table **if** sort key already groups by target instance (**A**) --- @@ -99,7 +98,7 @@ and queue polymorphism stay NEURON-specific (policies below). | Substep | Scope | Goal | |---------|--------|------| | **6b — Zero-copy host receive** | **Done:** contiguous block → `pnt_receive` via `data_if_contiguous()` / `weight_soa_data()`; TLS materialize only if scattered | Drop copy when packed; identity stays `weight_index` | -| **7a — nocmodl index ABI** | Generate `NET_RECEIVE` with base index; body names as `soaweight[ix+k]` / `double&` refs | Eliminate primary-edge scratch; CN-shaped interface | +| **7a — nocmodl index ABI** | **Done:** `pnt_receive_t(Point_process*, int weight_index, double flag)`; generated body keeps `_args[i]` via `_nrn_netrec_wsoa` / `_done`; `net_send` / `artcell_net_send` take index (−1 if none) | CN-shaped interface; identity is index only | | **7b — FOR_NETCONS without owned double pool** | Loop bases only; write SoA directly (or one shared view) | Drop `ForNetConsInfo::weight_storage` | | **7c — Cleanup** | Remove TLS scratch + base→buf maps when unused | Host path fully SoA/index | diff --git a/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp b/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp index 2a799d06a2..f44546a3d7 100644 --- a/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp +++ b/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp @@ -2738,7 +2738,8 @@ void CodegenNeuronCppVisitor::print_net_send_call(const ast::FunctionCall& node) throw std::runtime_error("Not implemented. [jfiwoei]"); } - std::string weight_pointer = "nullptr"; + // Heap-free 7a: Weight SoA base index (−1 outside NET_RECEIVE). + std::string weight_index = printing_net_receive ? "_weight_index" : "-1"; auto point_process = get_variable_name(naming::POINT_PROCESS_VARIABLE, /* use_instance */ false); if (!printing_net_receive) { @@ -2749,7 +2750,7 @@ void CodegenNeuronCppVisitor::print_net_send_call(const ast::FunctionCall& node) printer->fmt_text("{}(/* tqitem */ &{}, {}, {}, {} + ", info.artificial_cell ? "artcell_net_send" : "net_send", tqitem, - weight_pointer, + weight_index, point_process, get_variable_name("t")); print_vector_elements(arguments, ", "); @@ -2817,8 +2818,9 @@ static void rename_net_receive_arguments(const ast::NetReceiveBlock& net_receive CodegenNeuronCppVisitor::ParamVector CodegenNeuronCppVisitor::net_receive_args() { + // Heap-free 7a: match pnt_receive_t (Point_process*, int weight_index, double flag). return {{"", "Point_process*", "", "_pnt"}, - {"", "double*", "", "_args"}, + {"", "int", "", "_weight_index"}, {"", "double", "", "flag"}}; } @@ -2827,6 +2829,9 @@ void CodegenNeuronCppVisitor::print_net_receive_common_code() { printer->add_line("_nrn_mechanism_cache_instance _lmc{_pnt->prop};"); printer->add_line("auto * nt = static_cast(_pnt->_vnt);"); printer->add_line("auto * _ppvar = _nrn_mechanism_access_dparam(_pnt->prop);"); + // Body still uses _args[i]; resolve Weight SoA (zero-copy) or TLS. + printer->fmt_line("double* _args = _nrn_netrec_wsoa(_weight_index, {});", + info.num_net_receive_parameters); printer->fmt_line("auto inst = make_instance_{}(&_lmc);", info.mod_suffix); if (!info.artificial_cell) { @@ -2978,6 +2983,8 @@ void CodegenNeuronCppVisitor::print_net_receive() { print_statement_block(*node->get_statement_block(), false, false); + printer->fmt_line("_nrn_netrec_wsoa_done(_weight_index, {}, _args);", + info.num_net_receive_parameters); printer->add_newline(); printer->pop_block(); printing_net_receive = false; @@ -3000,6 +3007,8 @@ void CodegenNeuronCppVisitor::print_net_init() { if (!block->get_statements().empty()) { print_net_receive_common_code(); print_statement_block(*block, false, false); + printer->fmt_line("_nrn_netrec_wsoa_done(_weight_index, {}, _args);", + info.num_net_receive_parameters); } printer->pop_block(); printing_net_init = false; diff --git a/src/nocmodl/nocpout.cpp b/src/nocmodl/nocpout.cpp index f91509902f..aacd219bcf 100644 --- a/src/nocmodl/nocpout.cpp +++ b/src/nocmodl/nocpout.cpp @@ -1174,16 +1174,17 @@ static const char *_mechanism[] = {\n\ } } if (net_receive_) { - Lappendstr(defs_list, "static void _net_receive(Point_process*, double*, double);\n"); + // Heap-free 7a: weight identity is SoA base index, not double*. + Lappendstr(defs_list, "static void _net_receive(Point_process*, int, double);\n"); + Lappendstr(defs_list, "extern double* _nrn_netrec_wsoa(int, int);\n"); + Lappendstr(defs_list, "extern void _nrn_netrec_wsoa_done(int, int, double*);\n"); if (for_netcons_) { - // Heap-free step 4: walk by Weight SoA bases; resolve double* for MOD. Lappendstr(defs_list, "extern int _nrn_netcon_weight_bases(void*, int**);\n"); Lappendstr(defs_list, "extern double* _nrn_fornetcon_weight(int);\n"); - // Legacy dual-write ABI (still available for hand-written code). Lappendstr(defs_list, "extern int _nrn_netcon_args(void*, double***);\n"); } if (net_init_q1_) { - Lappendstr(defs_list, "static void _net_init(Point_process*, double*, double);\n"); + Lappendstr(defs_list, "static void _net_init(Point_process*, int, double);\n"); } } if (vectorize && thread_mem_init_list->next != thread_mem_init_list) { @@ -3054,7 +3055,8 @@ void net_receive(Item* qarg, Item* qp1, Item* qp2, Item* qstmt, Item* qend) { } net_receive_ = 1; deltokens(qp1, qp2); - insertstr(qstmt, "(Point_process* _pnt, double* _args, double _lflag)"); + /* Heap-free 7a: index ABI; body keeps _args[i] via SoA/TLS helper (like #define w soa[ix]). */ + insertstr(qstmt, "(Point_process* _pnt, int _weight_index, double _lflag)"); i = 0; ITERATE(q1, qarg) if (q1->next != qarg) { /* skip last "flag" arg */ s = SYM(q1); @@ -3072,6 +3074,14 @@ void net_receive(Item* qarg, Item* qp1, Item* qp2, Item* qstmt, Item* qend) { if (watch_seen_) { insertstr(qstmt, " int _watch_rm = 0;\n"); } + /* Resolve Weight SoA (zero-copy if contiguous) or TLS into _args for body. */ + { + char wbuf[256]; + Sprintf(wbuf, + " double* _args = _nrn_netrec_wsoa(_weight_index, %d);\n", + i > 0 ? i : 0); + insertstr(qstmt, wbuf); + } vectorize_substitute( insertstr(qstmt, " neuron::legacy::set_globals_from_prop(_pnt->_prop, _ml_real, _ml, _iml);\n"), @@ -3092,7 +3102,11 @@ void net_receive(Item* qarg, Item* qp1, Item* qp2, Item* qstmt, Item* qend) { "NetCon.delay\");}\n _tsav = t;"); } } - insertstr(qend, "}"); + { + char wbuf[256]; + Sprintf(wbuf, "\n _nrn_netrec_wsoa_done(_weight_index, %d, _args);\n}", i > 0 ? i : 0); + insertstr(qend, wbuf); + } if (!artificial_cell) { Symbol* ions[10]; int j, nion = 0; @@ -3165,10 +3179,12 @@ void net_receive(Item* qarg, Item* qp1, Item* qp2, Item* qstmt, Item* qend) { void net_init(Item* qinit, Item* qp2) { /* qinit=INITIAL { stmtlist qp2=} */ - replacstr(qinit, "\nstatic void _net_init(Point_process* _pnt, double* _args, double _lflag)"); + replacstr(qinit, "\nstatic void _net_init(Point_process* _pnt, int _weight_index, double _lflag)"); + /* Arity from this mech's pnt_receive_size[_mechtype] (avoid incomplete Prop::_type). */ Sprintf(buf, " neuron::legacy::set_globals_from_prop(_pnt->_prop, _ml_real, _ml, _iml);\n" - " _ppvar = _nrn_mechanism_access_dparam(_pnt->_prop);\n"); + " _ppvar = _nrn_mechanism_access_dparam(_pnt->_prop);\n" + " double* _args = _nrn_netrec_wsoa(_weight_index, pnt_receive_size[_mechtype]);\n"); vectorize_substitute(insertstr(qinit->next->next, buf), " _nrn_mechanism_cache_instance _ml_real{_pnt->_prop};\n" " auto* const _ml = &_ml_real;\n" @@ -3176,7 +3192,12 @@ void net_init(Item* qinit, Item* qp2) { " Datum* _ppvar = _nrn_mechanism_access_dparam(_pnt->_prop);\n" " Datum* _thread = nullptr;\n" " double* _globals = nullptr;\n" - " NrnThread* _nt = (NrnThread*)_pnt->_vnt;\n"); + " NrnThread* _nt = (NrnThread*)_pnt->_vnt;\n" + " double* _args = _nrn_netrec_wsoa(_weight_index, " + "pnt_receive_size[_mechtype]);\n"); + /* commit after INITIAL body: insert before closing brace qp2 */ + insertstr(qp2, + " _nrn_netrec_wsoa_done(_weight_index, pnt_receive_size[_mechtype], _args);\n"); if (net_init_q1_) { diag("NET_RECEIVE block can contain only one INITIAL block", (char*) 0); } diff --git a/src/nocmodl/parsact.cpp b/src/nocmodl/parsact.cpp index 17cd0dbdc8..f3851a7e42 100644 --- a/src/nocmodl/parsact.cpp +++ b/src/nocmodl/parsact.cpp @@ -902,9 +902,10 @@ void vectorize_use_func(Item* qname, Item* qpar1, Item* qexpr, Item* qpar2, int } Insertstr(qexpr, "t + "); if (blocktype == NETRECEIVE) { - Insertstr(qpar1->next, "_tqitem, _args, _pnt,"); + /* Heap-free 7a: pass Weight SoA base index, not double*. */ + Insertstr(qpar1->next, "_tqitem, _weight_index, _pnt,"); } else if (blocktype == INITIAL1) { - Insertstr(qpar1->next, "_tqitem, nullptr, _ppvar[1].get(),"); + Insertstr(qpar1->next, "_tqitem, -1, _ppvar[1].get(),"); } else { diag("net_send allowed only in INITIAL and NET_RECEIVE blocks", (char*) 0); } diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index fa73e62a2e..dae9bb2ded 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -59,7 +59,7 @@ typedef void (*ReceiveFunc)(Point_process*, double*, double); #define PP2t(pp) (PP2NT(pp)->_t) // classical and when DiscreteEvent::deliver is already in the right thread // via a future thread instance of NrnNetItem with its own tqe. -#define POINT_RECEIVE(type, tar, w, f) (*pnt_receive[type])(tar, w, f) +#define POINT_RECEIVE(type, tar, widx, f) (*pnt_receive[type])(tar, widx, f) // when global tqe is managed by master thread and the correct thread // needs to be fired to execute the NET_RECEIVE block. //#define POINT_RECEIVE(type, tar, w, f) ns->point_receive(type, tar, w, f) @@ -714,8 +714,8 @@ static double nc_event(void* v) { hoc_execerror("Can only send fake self-events to ARTIFICIAL_CELLs", 0); } auto* pq = pnt->prop->dparam + nrn_artcell_qindex_[type]; - // Pass Weight SoA base pointer when present so weight2netcon can bind. - nrn_net_send(pq, d->weight_soa_data(), pnt, td, flag); + // Heap-free 7a: pass Weight SoA base index as SelfEvent identity. + nrn_net_send(pq, static_cast(d->weight_base()), pnt, td, flag); } else { net_cvode_instance->event(td, d, PP2NT(d->target_)); } @@ -2293,6 +2293,8 @@ namespace { thread_local std::vector g_mod_weight_scratch; /** @brief Weight SoA base of the NET_RECEIVE currently on the stack (−1 if none). */ thread_local int g_tls_receive_weight_index = -1; +/** @brief 1 if _nrn_netrec_wsoa returned TLS (needs commit). */ +thread_local int g_netrec_wsoa_is_tmp = 0; double* mod_weight_scratch(int n) { if (n <= 0) { @@ -2314,19 +2316,53 @@ struct ReceiveWeightIndexGuard { g_tls_receive_weight_index = prev; } }; +} // namespace -/** Phase 4/5: SelfEvent identity is Weight SoA base (and optional long-lived heap). */ -void selfevent_set_indices(SelfEvent* se, Point_process* pnt, double* weight) { - se->target_row_ = nrn_point_process_soa_row(pnt); - se->weight_index_ = -1; - if (weight) { - if (NetCon* nc = NetConSave::weight2netcon(weight)) { - se->weight_index_ = static_cast(nc->weight_base()); - return; +double* _nrn_netrec_wsoa(int weight_index, int count) { + g_netrec_wsoa_is_tmp = 0; + if (weight_index < 0 || count <= 0) { + return nullptr; + } + // FOR_NETCONS peer/primary scratch (already SoA-synced around receive). + auto it = g_fornet_base_to_buf.find(weight_index); + if (it != g_fornet_base_to_buf.end()) { + return it->second; + } + // Zero-copy into contiguous Weight SoA block. + if (NetCon* nc = NetConSave::weight_index2netcon(weight_index)) { + if (nc->has_weight_soa()) { + if (double* p = nc->weight_block_->data_if_contiguous()) { + return p; + } } } - // net_send(..., _w) during NET_RECEIVE with ephemeral scratch: inherit active receive base. - if (g_tls_receive_weight_index >= 0) { + // Scattered: TLS materialize (commit in _nrn_netrec_wsoa_done). + double* buf = mod_weight_scratch(count); + neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, + count, + buf); + g_netrec_wsoa_is_tmp = 1; + return buf; +} + +void _nrn_netrec_wsoa_done(int weight_index, int count, double* buf) { + // Always commit when we have a buffer: TLS materialize, FOR_NETCONS owned + // scratch (fornetcon_prepare runs before init_events, so INITIAL often writes + // scratch not SoA), or no-op-ish self-copy when buf already aliases SoA. + // Without this, INITIAL values in fornet scratch were wiped on first deliver + // (soa_to_buf from still-zero SoA). + if (buf && weight_index >= 0 && count > 0) { + neuron::container::network::SelfEventFields::store_weight_block(weight_index, count, buf); + } + g_netrec_wsoa_is_tmp = 0; +} + +namespace { +/** SelfEvent identity from weight_index (preferred) or legacy pointer. */ +void selfevent_set_indices(SelfEvent* se, Point_process* pnt, int weight_index) { + se->target_row_ = nrn_point_process_soa_row(pnt); + se->weight_index_ = weight_index; + if (se->weight_index_ < 0 && g_tls_receive_weight_index >= 0) { se->weight_index_ = g_tls_receive_weight_index; } } @@ -2383,10 +2419,7 @@ void sync_netcon_weights_for_target(Point_process* pnt, bool soa_to_heap) { } } // namespace -void nrn_pnt_receive_by_weight_index(Point_process* pnt, - int weight_index, - double flag, - double* weight_heap) { +void nrn_pnt_receive_by_weight_index(Point_process* pnt, int weight_index, double flag) { if (!pnt || !pnt->prop) { return; } @@ -2394,88 +2427,33 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, if (!pnt_receive[type]) { return; } - int const n = pnt_receive_size[type]; - bool const fornet = type_has_fornetcon(type); - // Publish active base so net_send(..., _w) binds SelfEvent by weight_index. + // Active base for net_send identity (generated code passes _weight_index). ReceiveWeightIndexGuard const tls_guard(weight_index); - // FOR_NETCONS: materialize all peer edges into owned scratch, run MOD, write back. + bool const fornet = type_has_fornetcon(type); if (fornet) { sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ true); - double* buf = nullptr; - if (weight_index >= 0) { - auto it = g_fornet_base_to_buf.find(weight_index); - if (it != g_fornet_base_to_buf.end()) { - buf = it->second; - } - } - if (!buf && weight_heap) { - buf = weight_heap; - } - if (!buf && weight_index >= 0 && n > 0) { - buf = mod_weight_scratch(n); - neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, - n, - buf); - } - POINT_RECEIVE(type, pnt, buf, flag); - if (weight_index >= 0 && n > 0 && buf == mod_weight_scratch(n)) { - neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); - } - sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ false); - return; - } - - // Non-FOR_NETCONS (heap-free 6b): prefer zero-copy into contiguous Weight SoA. - // Fallback TLS materialize if block is scattered (post-erase, pre-sort). - // Never treat buf as SelfEvent identity (tls_guard + nrn_net_send policy). - if (weight_index < 0 || n <= 0) { - POINT_RECEIVE(type, pnt, nullptr, flag); - return; } - double* buf = nullptr; - bool zero_copy = false; - if (NetCon* owner = NetConSave::weight_index2netcon(weight_index)) { - if (owner->has_weight_soa()) { - buf = owner->weight_block_->data_if_contiguous(); - zero_copy = (buf != nullptr); - } - } - if (!buf && weight_heap) { - // Caller-supplied buffer (e.g. SoA data pointer already resolved). - buf = weight_heap; - zero_copy = true; // assume caller points at durable SoA or fornet scratch - } - if (!buf) { - buf = mod_weight_scratch(n); - neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, - n, - buf); - zero_copy = false; - } - POINT_RECEIVE(type, pnt, buf, flag); - if (!zero_copy) { - neuron::container::network::SelfEventFields::store_weight_block(weight_index, n, buf); + // Generated NET_RECEIVE obtains _args via _nrn_netrec_wsoa(_weight_index, n). + POINT_RECEIVE(type, pnt, weight_index, flag); + if (fornet) { + sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ false); } } // for threads, revised net_send to use absolute time (in the // mod file we add the thread time when we call it). -void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, double flag) { +// Heap-free 7a: weight identity is SoA base index (−1 if none). +void nrn_net_send(Datum* v, int weight_index, Point_process* pnt, double td, double flag) { STATISTICS(SelfEvent::selfevent_send_); NrnThread* nt = PP2NT(pnt); NetCvodeThreadData& p = net_cvode_instance->p[nt->id]; SelfEvent* se = p.sepool_->alloc(); se->flag_ = flag; se->target_ = pnt; - // Identity is weight_index only (heap-free step 6: no NetCon::weight_). - selfevent_set_indices(se, pnt, weight); + // Identity is weight_index only (no NetCon::weight_ heap; no double* identity). + selfevent_set_indices(se, pnt, weight_index); se->weight_ = nullptr; - if (weight && se->weight_index_ < 0) { - if (NetCon* nc = NetConSave::weight2netcon(weight)) { - se->weight_index_ = static_cast(nc->weight_base()); - } - } se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -2494,7 +2472,7 @@ void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, doubl // printf("net_send %g %s %g %p\n", td, hoc_object_name(pnt->ob), flag, *v); } -void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, double flag) { +void artcell_net_send(Datum* v, int weight_index, Point_process* pnt, double td, double flag) { if (nrn_use_selfqueue_ && flag == 1.0) { STATISTICS(SelfEvent::selfevent_send_); NrnThread* nt = PP2NT(pnt); @@ -2502,13 +2480,8 @@ void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, d SelfEvent* se = p.sepool_->alloc(); se->flag_ = flag; se->target_ = pnt; - selfevent_set_indices(se, pnt, weight); + selfevent_set_indices(se, pnt, weight_index); se->weight_ = nullptr; - if (weight && se->weight_index_ < 0) { - if (NetCon* nc = NetConSave::weight2netcon(weight)) { - se->weight_index_ = static_cast(nc->weight_base()); - } - } se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -2530,17 +2503,17 @@ void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, d se->deliver(td, net_cvode_instance, nt); } } else { - nrn_net_send(v, weight, pnt, td, flag); + nrn_net_send(v, weight_index, pnt, td, flag); } } // Deprecated overloads for backwards compatibility -void artcell_net_send(void* v, double* weight, Point_process* pnt, double td, double flag) { - artcell_net_send(static_cast(v), weight, pnt, td, flag); +void artcell_net_send(void* v, int weight_index, Point_process* pnt, double td, double flag) { + artcell_net_send(static_cast(v), weight_index, pnt, td, flag); } -void nrn_net_send(void* v, double* weight, Point_process* pnt, double td, double flag) { - nrn_net_send(static_cast(v), weight, pnt, td, flag); +void nrn_net_send(void* v, int weight_index, Point_process* pnt, double td, double flag) { + nrn_net_send(static_cast(v), weight_index, pnt, td, flag); } void net_event(Point_process* pnt, double time) { @@ -2841,10 +2814,12 @@ void NetCvode::allthread_handle(double tt, HocEvent* he, NrnThread* nt) { } #if 0 +// Dead alternate path: global TQE + per-thread NET_RECEIVE marshalling. +// Kept for reference; signature matches heap-free 7a index ABI. struct PPArgs { int type; Point_process* pp; - double* w; + int widx; double f; }; @@ -2852,21 +2827,21 @@ static PPArgs* ppargs; static void point_receive_job(NrnThread* nt) { PPArgs* p = ppargs + nt->id; - (*pnt_receive[p->_type])(p->pp, p->w, p->f); + (*pnt_receive[p->_type])(p->pp, p->widx, p->f); } -void NetCvode::point_receive(int type, Point_process* pp, double* w, double f) { +void NetCvode::point_receive(int type, Point_process* pp, int widx, double f) { // this is the master thread. need to execute the thread associated // with the pp. int id = PP2NT(pp)->id; if (id == 0) { // execute on this, the master thread - (*pnt_receive[type])(pp, w, f); + (*pnt_receive[type])(pp, widx, f); }else{ // marshall the args PPArgs* p = ppargs + id; p->_type = type; p->pp = pp; - p->w = w; + p->widx = widx; p->f = f; nrn_onethread_job(id, point_receive_job); } @@ -3018,26 +2993,13 @@ void NetCvode::init_events() { // target_ may outlive Prop (unlocated / free_one_point); skip dead targets. if (d->target_ && d->target_->prop) { int type = d->target_->prop->_type; - // Dual-write: INITIAL and HOC weight[] must share one value stream. - // SoA is HOC-primary; heap is the buffer for generated pnt_receive_init. + // Heap-free 7a: INITIAL receives weight_index; generated body uses + // _nrn_netrec_wsoa / _nrn_netrec_wsoa_done for SoA materialize. int const widx = static_cast(d->weight_base()); int const n = d->cnt_; if (pnt_receive_init[type]) { - double* buf = nullptr; - bool zero_copy = false; - if (d->has_weight_soa()) { - buf = d->weight_block_->data_if_contiguous(); - zero_copy = (buf != nullptr); - } - if (!buf && widx >= 0 && n > 0) { - buf = mod_weight_scratch(n); - neuron::container::network::SelfEventFields::materialize_weight_block( - widx, n, buf); - } - (*pnt_receive_init[type])(d->target_, buf, 0); - if (!zero_copy && widx >= 0 && n > 0 && buf) { - neuron::container::network::SelfEventFields::store_weight_block(widx, n, buf); - } + ReceiveWeightIndexGuard const tls_guard(widx); + (*pnt_receive_init[type])(d->target_, widx, 0); } else if (d->has_weight_soa()) { // not the first weight: zero weight[1..] in SoA for (j = n - 1; j > 0; --j) { @@ -3221,13 +3183,12 @@ void NetCon::deliver(double tt, NetCvode* ns, NrnThread* nt) { // printf("NetCon::deliver t=%g tt=%g %s\n", t, tt, hoc_object_name(target_->ob)); STATISTICS(netcon_deliver_); - // Heap-free 6b: deliver by weight_index; prefer contiguous SoA double*. + // Heap-free 7a: deliver by weight_index only (generated NET_RECEIVE resolves SoA). int widx = _soa.weight_index(); if (widx < 0) { widx = static_cast(weight_base()); } - double* wptr = has_weight_soa() ? weight_block_->data_if_contiguous() : nullptr; - nrn_pnt_receive_by_weight_index(target_, widx, 0., wptr); + nrn_pnt_receive_by_weight_index(target_, widx, 0.); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3247,8 +3208,7 @@ void NetCon::pgvts_deliver(double tt, NetCvode* ns) { if (widx < 0) { widx = static_cast(weight_base()); } - double* wptr = has_weight_soa() ? weight_block_->data_if_contiguous() : nullptr; - nrn_pnt_receive_by_weight_index(target_, widx, 0., wptr); + nrn_pnt_receive_by_weight_index(target_, widx, 0.); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during NetCon deliver to NET_RECEIVE", (char*) 0); @@ -3552,14 +3512,8 @@ DiscreteEvent* SelfEvent::savestate_save() { void SelfEvent::savestate_restore(double tt, NetCvode* nc) { // pr("savestate_restore", tt, nc); - // Rebind via weight_index only (no NetCon::weight_ heap). - double* w = nullptr; - if (weight_index_ >= 0) { - if (NetCon* owner = NetConSave::weight_index2netcon(weight_index_)) { - w = owner->weight_soa_data(); - } - } - nrn_net_send(movable_, w, target_, tt, flag_); + // Rebind via weight_index only (heap-free 7a). + nrn_net_send(movable_, weight_index_, target_, tt, flag_); } DiscreteEvent* SelfEvent::savestate_read(FILE* f) { @@ -3679,9 +3633,8 @@ void SelfEvent::pgvts_deliver(double tt, NetCvode* ns) { } void SelfEvent::call_net_receive(NetCvode* ns) { STATISTICS(selfevent_deliver_); - // Deliver by weight_index; weight_ is optional long-lived NetCon heap only. - // Ephemeral scratch is never stored on SelfEvent (heap-free step 5). - nrn_pnt_receive_by_weight_index(target_, weight_index_, flag_, weight_); + // Deliver by weight_index (heap-free 7a); ephemeral scratch is never queue identity. + nrn_pnt_receive_by_weight_index(target_, weight_index_, flag_); if (errno) { if (nrn_errno_check(target_->prop->_type)) { hoc_warning("errno set during SelfEvent deliver to NET_RECEIVE", (char*) 0); @@ -5857,12 +5810,12 @@ void WatchCondition::deliver(double tt, NetCvode* ns, NrnThread* nt) { PP2t(pnt_) = tt; } STATISTICS(watch_deliver_); - // WATCH-driven NET_RECEIVE (e.g. flag=2) may run FOR_NETCONS, which mutates - // every NetCon weight_ heap sharing this target — keep SoA dual-write coherent. + // WATCH-driven NET_RECEIVE (e.g. flag=2): no NetCon weight edge (−1). + // FOR_NETCONS still needs per-target scratch ↔ SoA sync. if (type_has_fornetcon(type)) { sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ true); } - POINT_RECEIVE(type, pnt_, nullptr, nrflag_); + POINT_RECEIVE(type, pnt_, -1, nrflag_); if (type_has_fornetcon(type)) { sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ false); } @@ -5959,7 +5912,7 @@ void WatchCondition::pgvts_deliver(double tt, NetCvode* ns) { if (type_has_fornetcon(type)) { sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ true); } - POINT_RECEIVE(type, pnt_, nullptr, nrflag_); + POINT_RECEIVE(type, pnt_, -1, nrflag_); if (type_has_fornetcon(type)) { sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ false); } diff --git a/src/nrncvode/netcvode.h b/src/nrncvode/netcvode.h index ebdfa14b97..39c5c30a17 100644 --- a/src/nrncvode/netcvode.h +++ b/src/nrncvode/netcvode.h @@ -188,7 +188,7 @@ class NetCvode { const char* sym2name(Symbol*); int pgvts(double tstop); void update_ps2nt(); - void point_receive(int, Point_process*, double*, double); + void point_receive(int, Point_process*, int /*weight_index*/, double); bool deliver_event(double til, NrnThread*); // uses TQueue atomically bool empty_; void delete_list(); diff --git a/src/nrniv/bbsavestate.cpp b/src/nrniv/bbsavestate.cpp index 015875699a..5d9d343456 100644 --- a/src/nrniv/bbsavestate.cpp +++ b/src/nrniv/bbsavestate.cpp @@ -198,13 +198,11 @@ extern bool nrn_use_bin_queue_; extern void (*nrn_binq_enqueue_error_handler)(double, TQItem*); static void bbss_early(double td, TQItem* tq); -typedef void (*ReceiveFunc)(Point_process*, double*, double); - #include "membfunc.h" extern int section_count; extern "C" void nrn_shape_update(); extern Section** secorder; -extern ReceiveFunc* pnt_receive; +// pnt_receive declared in nrniv_mf.h (heap-free 7a: weight_index ABI) extern NetCvode* net_cvode_instance; extern TQueue* net_cvode_instance_event_queue(NrnThread*); extern cTemplate** nrn_pnt_template_; @@ -2339,7 +2337,7 @@ void BBSaveState::netrecv_pp(Point_process* pp) { // new SelfEvent item mostly filled in. // But starting out with NULL weight vector and // flag=1 so that tqi->data is the new SelfEvent - nrn_net_send(&tqi_datum, nullptr, pp, tt, 1.0); + nrn_net_send(&tqi_datum, -1, pp, tt, 1.0); auto* tqi = tqi_datum.get(); assert(tqi && tqi->data_ && static_cast(tqi->data_)->type() == SelfEventType); diff --git a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp index 76cfa97c88..499fafcb70 100644 --- a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp +++ b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp @@ -1161,7 +1161,7 @@ static void core2nrn_SelfEvent_helper(int tid, int tar_type, int tar_index, double flag, - double* weight, + int weight_index, int is_movable) { if (type2movable.empty()) { setup_type2semantics(); @@ -1186,7 +1186,8 @@ static void core2nrn_SelfEvent_helper(int tid, int const movable_index = type2movable[tar_type]; auto* const movable_arg = pnt->prop->dparam + movable_index; auto* const old_movable_arg = (*movable_arg).get(); - nrn_net_send(movable_arg, weight, pnt, td, flag); + // Heap-free 7a: SelfEvent identity is Weight SoA base index. + nrn_net_send(movable_arg, weight_index, pnt, td, flag); if (!is_movable) { *movable_arg = old_movable_arg; } @@ -1209,8 +1210,8 @@ void core2nrn_SelfEvent_event(int tid, assert(nc->target_ == pnt); #endif - double* weight = nc->weight_soa_data(); - core2nrn_SelfEvent_helper(tid, td, tar_type, tar_index, flag, weight, is_movable); + int const weight_index = static_cast(nc->weight_base()); + core2nrn_SelfEvent_helper(tid, td, tar_type, tar_index, flag, weight_index, is_movable); } void core2nrn_SelfEvent_event_noweight(int tid, @@ -1220,8 +1221,7 @@ void core2nrn_SelfEvent_event_noweight(int tid, double flag, int is_movable) { assert(tid < nrn_nthread); - double* weight = NULL; - core2nrn_SelfEvent_helper(tid, td, tar_type, tar_index, flag, weight, is_movable); + core2nrn_SelfEvent_helper(tid, td, tar_type, tar_index, flag, -1, is_movable); } // Set of the voltage indices in which PreSyn.flag_ == true diff --git a/src/nrniv/savstate.cpp b/src/nrniv/savstate.cpp index 1ed24c2d94..2e55b18f7f 100644 --- a/src/nrniv/savstate.cpp +++ b/src/nrniv/savstate.cpp @@ -16,13 +16,12 @@ #include -typedef void (*ReceiveFunc)(Point_process*, double*, double); +// pnt_receive declared in nrniv_mf.h (heap-free 7a: weight_index ABI) #include "membfunc.h" extern int section_count; extern "C" void nrn_shape_update(); extern Section** secorder; -extern ReceiveFunc* pnt_receive; extern NetCvode* net_cvode_instance; extern TQueue* net_cvode_instance_event_queue(NrnThread*); extern std::vector* net_cvode_instance_psl(); diff --git a/src/nrnoc/nrniv_mf.h b/src/nrnoc/nrniv_mf.h index b22b18b1b6..eb7db27a69 100644 --- a/src/nrnoc/nrniv_mf.h +++ b/src/nrnoc/nrniv_mf.h @@ -19,37 +19,37 @@ using ldifusfunc3_t = double (*)(int, using ldifusfunc2_t = void(int, ldifusfunc3_t, void**, int, int, int, neuron::model_sorted_token const&, NrnThread&); using ldifusfunc_t = void (*)(ldifusfunc2_t, neuron::model_sorted_token const&, NrnThread&); -typedef void (*pnt_receive_t)(Point_process*, double*, double); -typedef void (*pnt_receive_init_t)(Point_process*, double*, double); +/** @brief NET_RECEIVE callback: weight identity is SoA base (−1 if none). */ +typedef void (*pnt_receive_t)(Point_process*, int /*weight_index*/, double /*flag*/); +typedef void (*pnt_receive_init_t)(Point_process*, int /*weight_index*/, double /*flag*/); /** - * @brief Phase 4 dual-write: call pnt_receive after materializing Weight SoA → heap buffer. - * - * Generated MOD still uses (Point_process*, double*, double). weight_index is the - * base row in neuron::model().weights(); count is pnt_receive_size[type]. - * If weight_heap is non-null it is used as the buffer (and written back to SoA after). + * @brief Deliver NET_RECEIVE by Weight SoA base index (heap-free 7a). */ -void nrn_pnt_receive_by_weight_index(Point_process* pnt, - int weight_index, - double flag, - double* weight_heap = nullptr); +void nrn_pnt_receive_by_weight_index(Point_process* pnt, int weight_index, double flag); + +/** @brief Resolve base → double* for generated NET_RECEIVE body (_args[i]). */ +double* _nrn_netrec_wsoa(int weight_index, int count); +/** @brief Write back if _nrn_netrec_wsoa used TLS (no-op for zero-copy SoA). */ +void _nrn_netrec_wsoa_done(int weight_index, int count, double* buf); extern Prop* need_memb_cl(Symbol*, int*, int*); extern Prop* prop_alloc(Prop**, int, Node*); void prop_update_ion_variables(Prop*, Node*); +/** @brief net_send: weight identity is SoA base index (−1 if none). */ [[deprecated("non-void* overloads are preferred")]] void artcell_net_send(void* v, - double* weight, + int weight_index, Point_process* pnt, double td, double flag); -void artcell_net_send(Datum* v, double* weight, Point_process* pnt, double td, double flag); +void artcell_net_send(Datum* v, int weight_index, Point_process* pnt, double td, double flag); [[deprecated("non-void* overloads are preferred")]] void nrn_net_send(void* v, - double* weight, + int weight_index, Point_process* pnt, double td, double flag); -void nrn_net_send(Datum* v, double* weight, Point_process* pnt, double td, double flag); +void nrn_net_send(Datum* v, int weight_index, Point_process* pnt, double td, double flag); extern double nrn_ion_charge(Symbol*); extern Point_process* ob2pntproc(Object*); From 4c53220b25b8d09ac099a4bd4c577ce20305c47b Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 12:08:29 -0400 Subject: [PATCH 29/33] Heap-free 7b: FOR_NETCONS bases only (drop weight_storage) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ForNetConsInfo now holds only Weight SoA base indices. Generated FOR_NETCONS resolves each peer via _nrn_fornetcon_weight: zero-copy into a contiguous SoA block when packed, otherwise a shared TLS view committed on the next peer or at receive end. Removes argslist, weight_storage, and the base→buf map; receive path no longer bulk- syncs a per-target double pool. Verified: build 225/225, build-asan 223/223. --- doc/network-soa/heap-free.md | 4 +- .../codegen/codegen_neuron_cpp_visitor.cpp | 12 +- src/nocmodl/nocpout.cpp | 2 +- src/nrncvode/netcvode.cpp | 206 +++++++----------- src/nrnoc/nrn_ansi.h | 10 +- 5 files changed, 94 insertions(+), 140 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index 6c643c46b2..06373d240f 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -99,8 +99,8 @@ and queue polymorphism stay NEURON-specific (policies below). |---------|--------|------| | **6b — Zero-copy host receive** | **Done:** contiguous block → `pnt_receive` via `data_if_contiguous()` / `weight_soa_data()`; TLS materialize only if scattered | Drop copy when packed; identity stays `weight_index` | | **7a — nocmodl index ABI** | **Done:** `pnt_receive_t(Point_process*, int weight_index, double flag)`; generated body keeps `_args[i]` via `_nrn_netrec_wsoa` / `_done`; `net_send` / `artcell_net_send` take index (−1 if none) | CN-shaped interface; identity is index only | -| **7b — FOR_NETCONS without owned double pool** | Loop bases only; write SoA directly (or one shared view) | Drop `ForNetConsInfo::weight_storage` | -| **7c — Cleanup** | Remove TLS scratch + base→buf maps when unused | Host path fully SoA/index | +| **7b — FOR_NETCONS without owned double pool** | **Done:** `ForNetConsInfo` = bases only; `_nrn_fornetcon_weight` zero-copy SoA or shared TLS view; no `weight_storage` / `argslist` / base→buf map | Drop per-target peer double pool | +| **7c — Cleanup** | Remove remaining TLS scratch when unused; drop dead `_nrn_netcon_args` | Host path fully SoA/index | Optional later: GPU upload of the same packed columns (sibling track), not a prerequisite for 7. diff --git a/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp b/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp index f44546a3d7..8e169a7013 100644 --- a/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp +++ b/src/nmodl/codegen/codegen_neuron_cpp_visitor.cpp @@ -2617,7 +2617,8 @@ void CodegenNeuronCppVisitor::print_mechanism_variables_macros() { // for CVODE printer->add_line("extern void _cvode_abstol(Symbol**, double*, int);"); if (info.for_netcon_used) { - printer->add_line("int _nrn_netcon_args(void*, double***);"); + printer->add_line("int _nrn_netcon_weight_bases(void*, int**);"); + printer->add_line("double* _nrn_fornetcon_weight(int);"); } } @@ -3070,16 +3071,17 @@ void CodegenNeuronCppVisitor::visit_for_netcon(const ast::ForNetcon& node) { int dparam_index = dparam_it->index; auto netcon_var = get_name(codegen_int_variables[dparam_index]); + // Heap-free 7b: walk Weight SoA bases; resolve double* per edge (SoA/TLS). // This is called from `print_statement_block` which pre-indents the // current line. Hence `add_text` only. - printer->add_text("double ** _fornetcon_data;"); + printer->add_text("int* _fornetcon_bases;"); printer->add_newline(); - printer->fmt_line("int _n_netcons = _nrn_netcon_args({}, &_fornetcon_data);", + printer->fmt_line("int _n_netcons = _nrn_netcon_weight_bases({}, &_fornetcon_bases);", get_variable_name(netcon_var, false)); - printer->push_block("for (size_t _i = 0; _i < _n_netcons; ++_i)"); - printer->add_line("double * _netcon_data = _fornetcon_data[_i];"); + printer->push_block("for (size_t _i = 0; _i < static_cast(_n_netcons); ++_i)"); + printer->add_line("double* _netcon_data = _nrn_fornetcon_weight(_fornetcon_bases[_i]);"); print_statement_block(*statement_block, false, false); printer->pop_block(); } diff --git a/src/nocmodl/nocpout.cpp b/src/nocmodl/nocpout.cpp index aacd219bcf..dfc45f4221 100644 --- a/src/nocmodl/nocpout.cpp +++ b/src/nocmodl/nocpout.cpp @@ -1179,9 +1179,9 @@ static const char *_mechanism[] = {\n\ Lappendstr(defs_list, "extern double* _nrn_netrec_wsoa(int, int);\n"); Lappendstr(defs_list, "extern void _nrn_netrec_wsoa_done(int, int, double*);\n"); if (for_netcons_) { + // Heap-free 7b: bases + per-edge SoA/TLS view (no owned double pool). Lappendstr(defs_list, "extern int _nrn_netcon_weight_bases(void*, int**);\n"); Lappendstr(defs_list, "extern double* _nrn_fornetcon_weight(int);\n"); - Lappendstr(defs_list, "extern int _nrn_netcon_args(void*, double***);\n"); } if (net_init_q1_) { Lappendstr(defs_list, "static void _net_init(Point_process*, int, double);\n"); diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index dae9bb2ded..02fcda4ada 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -235,28 +235,19 @@ extern void nrn_outputevent(unsigned char, double); #endif /** - * @brief Per-target FOR_NETCONS peer list (heap-free steps 4–6). + * @brief Per-target FOR_NETCONS peer list (heap-free 7b). * - * Identity is Weight SoA bases. argslist[i] points into owned scratch of length - * arity (not NetCon::weight_ — that field is gone). Generated code uses - * weight_bases + _nrn_fornetcon_weight. + * Identity is Weight SoA bases only — no owned double pool. + * Generated code: `_nrn_netcon_weight_bases` + `_nrn_fornetcon_weight(base)` + * (zero-copy into contiguous SoA, else shared TLS view + commit). */ struct ForNetConsInfo { int size{}; int arity{}; /** @brief Weight SoA base row for each NetCon targeting this PP. */ int* weight_bases{}; - /** @brief MOD-facing double* list (owned scratch, same order as bases). */ - double** argslist{}; - /** @brief Flat storage for argslist[i][0..arity). */ - double* weight_storage{}; }; -/** @brief base → FOR_NETCONS scratch (rebuilt in fornetcon_prepare). */ -std::unordered_map g_fornet_base_to_buf; -/** @brief base → NetCon* for weight2netcon when pointer is SoA data or fornet buf. */ -std::unordered_map g_weight_ptr_to_netcon; - static unsigned long deliver_cnt_, net_event_cnt_; unsigned long DiscreteEvent::discretevent_send_; unsigned long DiscreteEvent::discretevent_deliver_; @@ -2287,15 +2278,23 @@ void NetCvode::remove_event(TQItem* q, int tid) { namespace { /** - * Heap-free step 5: ephemeral MOD scratch for pnt_receive double* ABI. + * Heap-free step 5 / 7b: ephemeral MOD scratch for double* ABI. * Never use this buffer as SelfEvent / net_send identity — only weight_index. + * Primary NET_RECEIVE and FOR_NETCONS use separate TLS so they do not clobber. */ thread_local std::vector g_mod_weight_scratch; +thread_local std::vector g_fornet_weight_scratch; /** @brief Weight SoA base of the NET_RECEIVE currently on the stack (−1 if none). */ thread_local int g_tls_receive_weight_index = -1; /** @brief 1 if _nrn_netrec_wsoa returned TLS (needs commit). */ thread_local int g_netrec_wsoa_is_tmp = 0; +/** Active FOR_NETCONS edge view (commit on next peer or flush). */ +thread_local int g_fornet_active_base = -1; +thread_local int g_fornet_active_arity = 0; +thread_local double* g_fornet_active_buf = nullptr; +thread_local int g_fornet_active_is_tmp = 0; + double* mod_weight_scratch(int n) { if (n <= 0) { return nullptr; @@ -2306,6 +2305,29 @@ double* mod_weight_scratch(int n) { return g_mod_weight_scratch.data(); } +double* fornet_weight_scratch(int n) { + if (n <= 0) { + return nullptr; + } + if (static_cast(g_fornet_weight_scratch.size()) < n) { + g_fornet_weight_scratch.resize(static_cast(n)); + } + return g_fornet_weight_scratch.data(); +} + +void fornet_weight_commit() { + if (g_fornet_active_is_tmp && g_fornet_active_buf && g_fornet_active_base >= 0 && + g_fornet_active_arity > 0) { + neuron::container::network::SelfEventFields::store_weight_block(g_fornet_active_base, + g_fornet_active_arity, + g_fornet_active_buf); + } + g_fornet_active_is_tmp = 0; + g_fornet_active_base = -1; + g_fornet_active_arity = 0; + g_fornet_active_buf = nullptr; +} + struct ReceiveWeightIndexGuard { int prev; explicit ReceiveWeightIndexGuard(int idx) @@ -2323,11 +2345,6 @@ double* _nrn_netrec_wsoa(int weight_index, int count) { if (weight_index < 0 || count <= 0) { return nullptr; } - // FOR_NETCONS peer/primary scratch (already SoA-synced around receive). - auto it = g_fornet_base_to_buf.find(weight_index); - if (it != g_fornet_base_to_buf.end()) { - return it->second; - } // Zero-copy into contiguous Weight SoA block. if (NetCon* nc = NetConSave::weight_index2netcon(weight_index)) { if (nc->has_weight_soa()) { @@ -2346,15 +2363,13 @@ double* _nrn_netrec_wsoa(int weight_index, int count) { } void _nrn_netrec_wsoa_done(int weight_index, int count, double* buf) { - // Always commit when we have a buffer: TLS materialize, FOR_NETCONS owned - // scratch (fornetcon_prepare runs before init_events, so INITIAL often writes - // scratch not SoA), or no-op-ish self-copy when buf already aliases SoA. - // Without this, INITIAL values in fornet scratch were wiped on first deliver - // (soa_to_buf from still-zero SoA). + // Commit primary-edge TLS (no-op-ish self-copy when buf already aliases SoA). + // Also flush any pending FOR_NETCONS peer TLS from a nested loop. if (buf && weight_index >= 0 && count > 0) { neuron::container::network::SelfEventFields::store_weight_block(weight_index, count, buf); } g_netrec_wsoa_is_tmp = 0; + fornet_weight_commit(); } namespace { @@ -2366,57 +2381,7 @@ void selfevent_set_indices(SelfEvent* se, Point_process* pnt, int weight_index) se->weight_index_ = g_tls_receive_weight_index; } } -} // namespace -namespace { -bool type_has_fornetcon(int type) { - for (int i = 0; i < nrn_fornetcon_cnt_; ++i) { - if (nrn_fornetcon_type_[i] == type) { - return true; - } - } - return false; -} - -/** Sync FOR_NETCONS owned scratch buffers ↔ Weight SoA for one target PP. */ -void sync_fornetcon_buffers_for_target(Point_process* pnt, bool soa_to_buf) { - if (!pnt || !pnt->prop) { - return; - } - int const type = pnt->prop->_type; - int fn_slot = -1; - for (int i = 0; i < nrn_fornetcon_cnt_; ++i) { - if (nrn_fornetcon_type_[i] == type) { - fn_slot = nrn_fornetcon_index_[i]; - break; - } - } - if (fn_slot < 0) { - return; - } - auto* fnc = static_cast(pnt->prop->dparam[fn_slot].get()); - if (!fnc || fnc->size <= 0 || !fnc->weight_bases || !fnc->argslist) { - return; - } - int const arity = fnc->arity > 0 ? fnc->arity : pnt_receive_size[type]; - for (int i = 0; i < fnc->size; ++i) { - int const base = fnc->weight_bases[i]; - double* buf = fnc->argslist[i]; - if (base < 0 || !buf || arity <= 0) { - continue; - } - if (soa_to_buf) { - neuron::container::network::SelfEventFields::materialize_weight_block(base, arity, buf); - } else { - neuron::container::network::SelfEventFields::store_weight_block(base, arity, buf); - } - } -} - -// WatchCondition / legacy name: SoA ↔ FOR_NETCONS owned scratch. -void sync_netcon_weights_for_target(Point_process* pnt, bool soa_to_heap) { - sync_fornetcon_buffers_for_target(pnt, soa_to_heap); -} } // namespace void nrn_pnt_receive_by_weight_index(Point_process* pnt, int weight_index, double flag) { @@ -2430,15 +2395,9 @@ void nrn_pnt_receive_by_weight_index(Point_process* pnt, int weight_index, doubl // Active base for net_send identity (generated code passes _weight_index). ReceiveWeightIndexGuard const tls_guard(weight_index); - bool const fornet = type_has_fornetcon(type); - if (fornet) { - sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ true); - } - // Generated NET_RECEIVE obtains _args via _nrn_netrec_wsoa(_weight_index, n). + // Heap-free 7b: no FOR_NETCONS owned pool to sync — SoA / per-edge TLS only. POINT_RECEIVE(type, pnt, weight_index, flag); - if (fornet) { - sync_fornetcon_buffers_for_target(pnt, /*soa_to_buf*/ false); - } + fornet_weight_commit(); } // for threads, revised net_send to use absolute time (in the @@ -4352,7 +4311,6 @@ void NetCvode::fornetcon_prepare() { return; } fornetcon_change_cnt_ = structure_change_cnt; - g_fornet_base_to_buf.clear(); if (nrn_fornetcon_cnt_ == 0) { return; } @@ -4396,6 +4354,7 @@ void NetCvode::fornetcon_prepare() { } } + // Heap-free 7b: allocate bases only (no owned double pool). auto allocate_slots = [](ForNetConsInfo* fnc, int type) { if (!fnc || fnc->size <= 0) { return; @@ -4404,10 +4363,7 @@ void NetCvode::fornetcon_prepare() { int const arity = pnt_receive_size[type] > 0 ? pnt_receive_size[type] : 1; fnc->arity = arity; fnc->weight_bases = new int[n]; - fnc->argslist = new double*[n]; - fnc->weight_storage = new double[static_cast(n) * static_cast(arity)]{}; for (int s = 0; s < n; ++s) { - fnc->argslist[s] = fnc->weight_storage + static_cast(s) * arity; fnc->weight_bases[s] = -1; } fnc->size = 0; // fill pass @@ -4434,7 +4390,7 @@ void NetCvode::fornetcon_prepare() { } } } - // fill bases + map base → scratch; materialize from SoA + // fill Weight SoA bases only if (psl_) { for (const PreSyn* ps: *psl_) { for (const auto& d1: ps->dil_) { @@ -4443,14 +4399,7 @@ void NetCvode::fornetcon_prepare() { auto* fnc = static_cast( pnt->prop->dparam[t2i[pnt->prop->_type]].get()); int const slot = fnc->size; - int const base = static_cast(d1->weight_base()); - fnc->weight_bases[slot] = base; - double* buf = fnc->argslist[slot]; - if (base >= 0 && fnc->arity > 0) { - neuron::container::network::SelfEventFields::materialize_weight_block( - base, fnc->arity, buf); - g_fornet_base_to_buf[base] = buf; - } + fnc->weight_bases[slot] = static_cast(d1->weight_base()); fnc->size = slot + 1; } } @@ -4458,11 +4407,13 @@ void NetCvode::fornetcon_prepare() { } } -int _nrn_netcon_args(void* v, double*** argslist) { - auto* fnc = static_cast(v); - assert(fnc); - *argslist = fnc->argslist; - return fnc->size; +int _nrn_netcon_args(void* /*v*/, double*** argslist) { + // Heap-free 7b: owned peer double pool removed. Use bases + _nrn_fornetcon_weight. + // Legacy API kept so old object files fail closed rather than crash on null. + if (argslist) { + *argslist = nullptr; + } + return 0; } int _nrn_netcon_weight_bases(void* v, int** bases) { @@ -4473,26 +4424,37 @@ int _nrn_netcon_weight_bases(void* v, int** bases) { } double* _nrn_fornetcon_weight(int weight_base) { + // Commit previous peer edge (shared TLS view) before opening a new one. + fornet_weight_commit(); if (weight_base < 0) { return nullptr; } - auto it = g_fornet_base_to_buf.find(weight_base); - if (it != g_fornet_base_to_buf.end()) { - return it->second; + NetCon* nc = NetConSave::weight_index2netcon(weight_base); + if (!nc || !nc->has_weight_soa()) { + return nullptr; } - return nullptr; + int const arity = nc->cnt_ > 0 ? nc->cnt_ : 1; + // Zero-copy when the weight block is contiguous in SoA (packing A / allocate). + if (double* p = nc->weight_block_->data_if_contiguous()) { + g_fornet_active_base = weight_base; + g_fornet_active_arity = arity; + g_fornet_active_buf = p; + g_fornet_active_is_tmp = 0; + return p; + } + // Shared TLS view for one edge (not O(peers) owned pool). + double* buf = fornet_weight_scratch(arity); + neuron::container::network::SelfEventFields::materialize_weight_block(weight_base, arity, buf); + g_fornet_active_base = weight_base; + g_fornet_active_arity = arity; + g_fornet_active_buf = buf; + g_fornet_active_is_tmp = 1; + return buf; } void _nrn_free_fornetcon(void** v) { if (auto* fnc = static_cast(*v); fnc) { - if (fnc->weight_bases && fnc->size > 0) { - for (int i = 0; i < fnc->size; ++i) { - g_fornet_base_to_buf.erase(fnc->weight_bases[i]); - } - } - delete[] std::exchange(fnc->argslist, nullptr); delete[] std::exchange(fnc->weight_bases, nullptr); - delete[] std::exchange(fnc->weight_storage, nullptr); delete fnc; *v = nullptr; } @@ -5255,12 +5217,6 @@ NetCon* NetConSave::weight2netcon(double* pd) { if (!pd) { return nullptr; } - // FOR_NETCONS scratch buffers registered during fornetcon_prepare. - for (auto const& kv: g_fornet_base_to_buf) { - if (kv.second == pd) { - return weight_index2netcon(kv.first); - } - } NetCon* nc; if (!wtable_) { hoc_Item* q; @@ -5811,14 +5767,9 @@ void WatchCondition::deliver(double tt, NetCvode* ns, NrnThread* nt) { } STATISTICS(watch_deliver_); // WATCH-driven NET_RECEIVE (e.g. flag=2): no NetCon weight edge (−1). - // FOR_NETCONS still needs per-target scratch ↔ SoA sync. - if (type_has_fornetcon(type)) { - sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ true); - } + // Heap-free 7b: FOR_NETCONS mutates SoA / peer TLS; flush after body. POINT_RECEIVE(type, pnt_, -1, nrflag_); - if (type_has_fornetcon(type)) { - sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ false); - } + fornet_weight_commit(); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during WatchCondition deliver to NET_RECEIVE", (char*) 0); @@ -5909,13 +5860,8 @@ void WatchCondition::pgvts_deliver(double tt, NetCvode* ns) { } int type = pnt_->prop->_type; STATISTICS(watch_deliver_); - if (type_has_fornetcon(type)) { - sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ true); - } POINT_RECEIVE(type, pnt_, -1, nrflag_); - if (type_has_fornetcon(type)) { - sync_netcon_weights_for_target(pnt_, /*soa_to_heap*/ false); - } + fornet_weight_commit(); if (errno) { if (nrn_errno_check(type)) { hoc_warning("errno set during WatchCondition deliver to NET_RECEIVE", (char*) 0); diff --git a/src/nrnoc/nrn_ansi.h b/src/nrnoc/nrn_ansi.h index 61fc9315d2..103178814e 100644 --- a/src/nrnoc/nrn_ansi.h +++ b/src/nrnoc/nrn_ansi.h @@ -30,11 +30,17 @@ extern "C" void modl_reg(void); // nrnmech stuff extern void _nrn_free_fornetcon(void**); -/** @brief Legacy FOR_NETCONS: return double** list of owned peer scratch. */ +/** + * @brief Legacy FOR_NETCONS double** API (heap-free 7b: always empty). + * Prefer _nrn_netcon_weight_bases + _nrn_fornetcon_weight. + */ extern int _nrn_netcon_args(void*, double***); /** @brief FOR_NETCONS by Weight SoA base list. */ extern int _nrn_netcon_weight_bases(void*, int**); -/** @brief Resolve weight SoA base → FOR_NETCONS owned scratch for MOD body. */ +/** + * @brief Resolve weight SoA base → mutable double* for MOD FOR_NETCONS body. + * Zero-copy if contiguous; else shared TLS view (commit on next peer / receive end). + */ extern double* _nrn_fornetcon_weight(int weight_base); extern double nrn_call_mech_func(Symbol*, int narg, Prop*, int type); From f62f75f5b486c09f8e6515d449c25b8cbc1cb471 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 12:27:11 -0400 Subject: [PATCH 30/33] Heap-free 7c: drop SelfEvent::weight_ and dead FOR_NETCONS API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove SelfEvent::weight_ (queue identity is weight_index only) and _nrn_netcon_args (bases + _nrn_fornetcon_weight only). TLS commit on _nrn_netrec_wsoa_done only when the primary edge used materialize; zero-copy SoA needs no writeback. Document post-7a–7c host path shape. Verified: build 225/225, build-asan 223/223. --- doc/network-soa/heap-free.md | 15 ++++++++-- src/nrncvode/netcon.h | 17 +++-------- src/nrncvode/netcvode.cpp | 28 ++++--------------- src/nrniv/bbsavestate.cpp | 6 ++-- .../callbacks/nrncore_callbacks.cpp | 6 ++-- src/nrnoc/nrn_ansi.h | 5 ---- 6 files changed, 28 insertions(+), 49 deletions(-) diff --git a/doc/network-soa/heap-free.md b/doc/network-soa/heap-free.md index 06373d240f..f9578be6cd 100644 --- a/doc/network-soa/heap-free.md +++ b/doc/network-soa/heap-free.md @@ -74,7 +74,7 @@ and queue polymorphism stay NEURON-specific (policies below). ## Contract (identity) -> A NetCon weight block is identified by a base row in the Weight SoA (arity fixed by target mechanism type). Queued SelfEvents and generated `net_send` carry that base as `int` (or −1). FOR_NETCONS walks a target-local list of bases (owned scratch for MOD `double*`, writeback to SoA). There is **no** per-NetCon `weight_` heap. +> A NetCon weight block is identified by a base row in the Weight SoA (arity fixed by target mechanism type). Queued SelfEvents and generated `net_send` carry that base as `int` (or −1). `NET_RECEIVE` / FOR_NETCONS resolve MOD `double*` via contiguous SoA zero-copy, or TLS materialize when scattered. There is **no** per-NetCon weight heap and **no** `SelfEvent::weight_` pointer. --- @@ -100,10 +100,21 @@ and queue polymorphism stay NEURON-specific (policies below). | **6b — Zero-copy host receive** | **Done:** contiguous block → `pnt_receive` via `data_if_contiguous()` / `weight_soa_data()`; TLS materialize only if scattered | Drop copy when packed; identity stays `weight_index` | | **7a — nocmodl index ABI** | **Done:** `pnt_receive_t(Point_process*, int weight_index, double flag)`; generated body keeps `_args[i]` via `_nrn_netrec_wsoa` / `_done`; `net_send` / `artcell_net_send` take index (−1 if none) | CN-shaped interface; identity is index only | | **7b — FOR_NETCONS without owned double pool** | **Done:** `ForNetConsInfo` = bases only; `_nrn_fornetcon_weight` zero-copy SoA or shared TLS view; no `weight_storage` / `argslist` / base→buf map | Drop per-target peer double pool | -| **7c — Cleanup** | Remove remaining TLS scratch when unused; drop dead `_nrn_netcon_args` | Host path fully SoA/index | +| **7c — Cleanup** | **Done:** drop `SelfEvent::weight_`, drop `_nrn_netcon_args`; TLS only for scattered blocks (zero-copy when packed); commit only when materialize was used | Host path SoA/index; no dead double* identity | Optional later: GPU upload of the same packed columns (sibling track), not a prerequisite for 7. +### Host path shape after 7a–7c + +| Item | State | +|------|--------| +| NetCon weight storage | Weight SoA only (`WeightBlock`) | +| SelfEvent / net_send identity | `int weight_index` (−1 if none) | +| `pnt_receive` / INITIAL | `(Point_process*, int, double)` | +| Primary edge `_args[i]` | Contiguous SoA zero-copy, else TLS + commit | +| FOR_NETCONS | Bases list; per-edge SoA/TLS (no owned peer pool) | +| TLS | Kept as **scattered-block fallback** only (packing A + sort → usually zero-copy) | + --- ## Workflow diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index d41bb7c8c3..72c357cf66 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -187,19 +187,11 @@ class NetConSave: public DiscreteEvent { NetCon* netcon_; static void invalid(); - /** - * @brief Map double* → NetCon* if it is a FOR_NETCONS scratch buffer or - * Weight SoA data pointer known for a live edge. Prefer weight_index2netcon. - */ + /** @brief Map Weight SoA data pointer → NetCon* (legacy; prefer weight_index2netcon). */ static NetCon* weight2netcon(double*); /** @brief Map HOC NetCon object index → NetCon*. */ static NetCon* index2netcon(long); - /** - * @brief Map Weight SoA base row (weight_index) → NetCon*. - * - * Used when SelfEvent identity is carried as weight_index_ without a - * reliable weight_ heap pointer (dual-write / heap-drop path). - */ + /** @brief Map Weight SoA base row (weight_index) → NetCon*. */ static NetCon* weight_index2netcon(int weight_index); private: @@ -231,11 +223,10 @@ class SelfEvent: public DiscreteEvent { double flag_; Point_process* target_; - double* weight_; Datum* movable_; // pointed-to Datum holds TQItem* - /** @brief Phase 4: base row in Weight SoA (-1 if unknown / null weights). */ + /** @brief Weight SoA base row (−1 if no NetCon / null weights). Queue identity. */ int weight_index_{-1}; - /** @brief Phase 4: PointProcess SoA row of target_ (-1 if unknown). */ + /** @brief PointProcess SoA row of target_ (−1 if unknown). */ int target_row_{-1}; static unsigned long selfevent_send_; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 02fcda4ada..683e83f6fb 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2363,9 +2363,9 @@ double* _nrn_netrec_wsoa(int weight_index, int count) { } void _nrn_netrec_wsoa_done(int weight_index, int count, double* buf) { - // Commit primary-edge TLS (no-op-ish self-copy when buf already aliases SoA). - // Also flush any pending FOR_NETCONS peer TLS from a nested loop. - if (buf && weight_index >= 0 && count > 0) { + // Commit only if _nrn_netrec_wsoa returned TLS (scattered block). + // Zero-copy SoA pointers need no writeback. Flush pending FOR_NETCONS peer TLS. + if (g_netrec_wsoa_is_tmp && buf && weight_index >= 0 && count > 0) { neuron::container::network::SelfEventFields::store_weight_block(weight_index, count, buf); } g_netrec_wsoa_is_tmp = 0; @@ -2373,7 +2373,7 @@ void _nrn_netrec_wsoa_done(int weight_index, int count, double* buf) { } namespace { -/** SelfEvent identity from weight_index (preferred) or legacy pointer. */ +/** SelfEvent identity: weight_index (−1 if none); TLS receive base as fallback. */ void selfevent_set_indices(SelfEvent* se, Point_process* pnt, int weight_index) { se->target_row_ = nrn_point_process_soa_row(pnt); se->weight_index_ = weight_index; @@ -2410,9 +2410,8 @@ void nrn_net_send(Datum* v, int weight_index, Point_process* pnt, double td, dou SelfEvent* se = p.sepool_->alloc(); se->flag_ = flag; se->target_ = pnt; - // Identity is weight_index only (no NetCon::weight_ heap; no double* identity). + // Identity is weight_index only (heap-free 7c: no SelfEvent::weight_). selfevent_set_indices(se, pnt, weight_index); - se->weight_ = nullptr; se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -2440,7 +2439,6 @@ void artcell_net_send(Datum* v, int weight_index, Point_process* pnt, double td, se->flag_ = flag; se->target_ = pnt; selfevent_set_indices(se, pnt, weight_index); - se->weight_ = nullptr; se->movable_ = v; // needed for SaveState assert(net_cvode_instance); ++p.unreffed_event_cnt_; @@ -3462,7 +3460,6 @@ DiscreteEvent* SelfEvent::savestate_save() { SelfEvent* se = new SelfEvent(); se->flag_ = flag_; se->target_ = target_; - se->weight_ = weight_; se->weight_index_ = weight_index_; se->target_row_ = target_row_; se->movable_ = movable_; @@ -3486,14 +3483,12 @@ DiscreteEvent* SelfEvent::savestate_read(FILE* f) { sscanf(buf, "%s %d %d %d %d %lf\n", ppname, &ppindex, &pptype, &ncindex, &moff, &flag) == 6); se->target_ = SelfEvent::index2pp(pptype, ppindex); - se->weight_ = nullptr; se->weight_index_ = -1; se->target_row_ = nrn_point_process_soa_row(se->target_); if (ncindex >= 0) { - // File identity is NetCon object index (not weight_ pointer). + // File identity is NetCon object index → Weight SoA base. NetCon* nc = NetConSave::index2netcon(ncindex); assert(nc); - se->weight_ = nullptr; se->weight_index_ = static_cast(nc->weight_base()); } se->flag_ = flag; @@ -3537,8 +3532,6 @@ void SelfEvent::savestate_write(FILE* f) { NetCon* owner = nullptr; if (weight_index_ >= 0) { owner = NetConSave::weight_index2netcon(weight_index_); - } else if (weight_) { - owner = NetConSave::weight2netcon(weight_); } if (owner && owner->obj_) { ncindex = owner->obj_->index; @@ -4407,15 +4400,6 @@ void NetCvode::fornetcon_prepare() { } } -int _nrn_netcon_args(void* /*v*/, double*** argslist) { - // Heap-free 7b: owned peer double pool removed. Use bases + _nrn_fornetcon_weight. - // Legacy API kept so old object files fail closed rather than crash on null. - if (argslist) { - *argslist = nullptr; - } - return 0; -} - int _nrn_netcon_weight_bases(void* v, int** bases) { auto* fnc = static_cast(v); assert(fnc); diff --git a/src/nrniv/bbsavestate.cpp b/src/nrniv/bbsavestate.cpp index 5d9d343456..4417200fb7 100644 --- a/src/nrniv/bbsavestate.cpp +++ b/src/nrniv/bbsavestate.cpp @@ -1127,11 +1127,9 @@ static void selfevent_bind_netcon(SelfEvent* se, NetCon* nc) { return; } if (!nc) { - se->weight_ = nullptr; se->weight_index_ = -1; return; } - se->weight_ = nullptr; se->weight_index_ = static_cast(nc->weight_base()); } @@ -1139,8 +1137,8 @@ SEWrap::SEWrap(const TQItem* tq, DEList* dl) { tt = tq->t_; se = (SelfEvent*) tq->data_; // SelfEvent identity for BBSaveState: index into target's NetCon DEList - // (same policy as SaveState NetCon object index), not weight_* alone. - if (se->weight_ || se->weight_index_ >= 0) { + // via weight_index_ (heap-free 7c: no SelfEvent::weight_). + if (se->weight_index_ >= 0) { ncindex = 0; for (; dl && dl->de && dl->de->type() == NetConType; dl = dl->next, ++ncindex) { if (selfevent_matches_netcon(se, static_cast(dl->de))) { diff --git a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp index 499fafcb70..96c0fe303e 100644 --- a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp +++ b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp @@ -939,9 +939,9 @@ static void set_info(TQItem* tqi, Point_process* pnt = se->target_; int type = pnt->prop->_type; int movable_index = type2movable[type]; - // Heap-free: prefer weight_index → SoA data pointer for NetCon match. - double* wt = se->weight_; - if (!wt && se->weight_index_ >= 0) { + // Heap-free 7c: SelfEvent identity is weight_index only. + double* wt = nullptr; + if (se->weight_index_ >= 0) { if (NetCon* nc = NetConSave::weight_index2netcon(se->weight_index_)) { wt = nc->weight_soa_data(); } diff --git a/src/nrnoc/nrn_ansi.h b/src/nrnoc/nrn_ansi.h index 103178814e..6cdd012826 100644 --- a/src/nrnoc/nrn_ansi.h +++ b/src/nrnoc/nrn_ansi.h @@ -30,11 +30,6 @@ extern "C" void modl_reg(void); // nrnmech stuff extern void _nrn_free_fornetcon(void**); -/** - * @brief Legacy FOR_NETCONS double** API (heap-free 7b: always empty). - * Prefer _nrn_netcon_weight_bases + _nrn_fornetcon_weight. - */ -extern int _nrn_netcon_args(void*, double***); /** @brief FOR_NETCONS by Weight SoA base list. */ extern int _nrn_netcon_weight_bases(void*, int**); /** From 9eebc43d0fdc24ded7d316612f705899e0ced68e Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Fri, 24 Jul 2026 12:30:30 -0400 Subject: [PATCH 31/33] format-pr --- src/nocmodl/nocpout.cpp | 10 ++++------ src/nrncvode/netcvode.cpp | 13 ++++++------- test/unit_tests/container/network.cpp | 3 +-- 3 files changed, 11 insertions(+), 15 deletions(-) diff --git a/src/nocmodl/nocpout.cpp b/src/nocmodl/nocpout.cpp index dfc45f4221..3c90f2fc1e 100644 --- a/src/nocmodl/nocpout.cpp +++ b/src/nocmodl/nocpout.cpp @@ -3077,9 +3077,7 @@ void net_receive(Item* qarg, Item* qp1, Item* qp2, Item* qstmt, Item* qend) { /* Resolve Weight SoA (zero-copy if contiguous) or TLS into _args for body. */ { char wbuf[256]; - Sprintf(wbuf, - " double* _args = _nrn_netrec_wsoa(_weight_index, %d);\n", - i > 0 ? i : 0); + Sprintf(wbuf, " double* _args = _nrn_netrec_wsoa(_weight_index, %d);\n", i > 0 ? i : 0); insertstr(qstmt, wbuf); } vectorize_substitute( @@ -3179,7 +3177,8 @@ void net_receive(Item* qarg, Item* qp1, Item* qp2, Item* qstmt, Item* qend) { void net_init(Item* qinit, Item* qp2) { /* qinit=INITIAL { stmtlist qp2=} */ - replacstr(qinit, "\nstatic void _net_init(Point_process* _pnt, int _weight_index, double _lflag)"); + replacstr(qinit, + "\nstatic void _net_init(Point_process* _pnt, int _weight_index, double _lflag)"); /* Arity from this mech's pnt_receive_size[_mechtype] (avoid incomplete Prop::_type). */ Sprintf(buf, " neuron::legacy::set_globals_from_prop(_pnt->_prop, _ml_real, _ml, _iml);\n" @@ -3196,8 +3195,7 @@ void net_init(Item* qinit, Item* qp2) { " double* _args = _nrn_netrec_wsoa(_weight_index, " "pnt_receive_size[_mechtype]);\n"); /* commit after INITIAL body: insert before closing brace qp2 */ - insertstr(qp2, - " _nrn_netrec_wsoa_done(_weight_index, pnt_receive_size[_mechtype], _args);\n"); + insertstr(qp2, " _nrn_netrec_wsoa_done(_weight_index, pnt_receive_size[_mechtype], _args);\n"); if (net_init_q1_) { diag("NET_RECEIVE block can contain only one INITIAL block", (char*) 0); } diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index 683e83f6fb..abb37ddc56 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2355,9 +2355,7 @@ double* _nrn_netrec_wsoa(int weight_index, int count) { } // Scattered: TLS materialize (commit in _nrn_netrec_wsoa_done). double* buf = mod_weight_scratch(count); - neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, - count, - buf); + neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, count, buf); g_netrec_wsoa_is_tmp = 1; return buf; } @@ -3231,7 +3229,8 @@ void for_each_fanout_netcon(PreSyn* ps, F&& fn) { if (base >= 0 && cnt == static_cast(ps->dil_.size()) && base + cnt <= static_cast(g_network_fanout_order.size())) { for (int i = 0; i < cnt; ++i) { - NetCon* d = netcon_from_soa_row(g_network_fanout_order[static_cast(base + i)]); + NetCon* d = netcon_from_soa_row( + g_network_fanout_order[static_cast(base + i)]); if (d) { fn(d); } @@ -4376,9 +4375,9 @@ void NetCvode::fornetcon_prepare() { if (tml->index == type) { Memb_list* m = tml->ml; for (j = 0; j < m->nodecount; ++j) { - allocate_slots( - static_cast(m->pdata[j][index].get()), - type); + allocate_slots(static_cast( + m->pdata[j][index].get()), + type); } } } diff --git a/test/unit_tests/container/network.cpp b/test/unit_tests/container/network.cpp index edf12887e3..46d78fd0da 100644 --- a/test/unit_tests/container/network.cpp +++ b/test/unit_tests/container/network.cpp @@ -361,8 +361,7 @@ TEST_CASE("Fanout order stores SoA indices not pointers", void* b = reinterpret_cast(static_cast(0x2000)); shells[0] = a; shells[2] = b; - REQUIRE(sizeof(netcon_index_t) * fanout_order.size() < - sizeof(void*) * fanout_order.size()); + REQUIRE(sizeof(netcon_index_t) * fanout_order.size() < sizeof(void*) * fanout_order.size()); REQUIRE(shells[fanout_order[0]] == a); REQUIRE(shells[fanout_order[1]] == b); REQUIRE(shells[fanout_order[2]] == nullptr); From dcdbf97fe8d09b5435abf4d4d73886f46944abe9 Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Sat, 25 Jul 2026 19:21:22 -0400 Subject: [PATCH 32/33] Heap-free: resolve Weight SoA by index (drop reverse NetCon map) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sim path was looking up NetCon* from weight_index on every NET_RECEIVE (_nrn_netrec_wsoa / FOR_NETCONS), which is CoreNEURON-unlike and O(N) over HOC NetCons — Traub 1/10 hung after first spikes until nrn_timeout. Address model().weights()[base] directly (weight_soa_ptr), TLS materialize only if out of range. Remove weight_index2netcon; NetCon→base stays forward-only (owning handles / WeightIndex on sort). SaveState SelfEvent write uses a cold O(N) scan for HOC object index only. Verified: Traub one_tenth no-gap, nthread=1, pc.timeout(1); 100 ms nocmodl and NMODL both complete with exact 4474 spikes. --- src/neuron/container/network/self_event.hpp | 27 +++++++- src/nrncvode/netcon.h | 6 +- src/nrncvode/netcvode.cpp | 69 +++++++------------ src/nrncvode/network_soa_sort.cpp | 3 +- .../callbacks/nrncore_callbacks.cpp | 8 +-- 5 files changed, 59 insertions(+), 54 deletions(-) diff --git a/src/neuron/container/network/self_event.hpp b/src/neuron/container/network/self_event.hpp index 3bc9444d31..c01cbd2317 100644 --- a/src/neuron/container/network/self_event.hpp +++ b/src/neuron/container/network/self_event.hpp @@ -42,11 +42,34 @@ struct Flag { }; } // namespace field +/** + * @brief CoreNEURON-style zero-copy pointer into Weight SoA at base @p weight_index. + * + * Under packing A (post-sort / sim freeze), a NetCon block of @p count is + * base..base+count-1 consecutive rows. No NetCon* reverse lookup: the index is + * the address (cf. NrnThread::weights + weight_index). + * + * @return nullptr if out of range (caller may TLS-materialize). + */ +inline double* weight_soa_ptr(int weight_index, int count) { + if (weight_index < 0 || count <= 0) { + return nullptr; + } + auto& store = neuron::model().weights(); + auto const n = store.size(); + auto const base = static_cast(weight_index); + auto const need = static_cast(count); + if (base >= n || base + need > n) { + return nullptr; + } + return &store.get(base); +} + /** * @brief Copy @p count consecutive Weight SoA rows starting at @p weight_index into @p out. * - * Contiguity is the dual-write invariant from weight block allocation (Phase 1–2). - * After a full weight repack/sort this remains valid for a NetCon's block. + * Used when zero-copy is unavailable (scattered rows before pack). After weight + * repack/sort, prefer weight_soa_ptr. */ inline void materialize_weight_block(int weight_index, int count, double* out) { if (!out || count <= 0 || weight_index < 0) { diff --git a/src/nrncvode/netcon.h b/src/nrncvode/netcon.h index 72c357cf66..02e8d83ba4 100644 --- a/src/nrncvode/netcon.h +++ b/src/nrncvode/netcon.h @@ -187,12 +187,10 @@ class NetConSave: public DiscreteEvent { NetCon* netcon_; static void invalid(); - /** @brief Map Weight SoA data pointer → NetCon* (legacy; prefer weight_index2netcon). */ + /** @brief Map Weight SoA data pointer → NetCon* (legacy cold helpers / queue tools). */ static NetCon* weight2netcon(double*); - /** @brief Map HOC NetCon object index → NetCon*. */ + /** @brief Map HOC NetCon object index → NetCon* (SaveState). */ static NetCon* index2netcon(long); - /** @brief Map Weight SoA base row (weight_index) → NetCon*. */ - static NetCon* weight_index2netcon(int weight_index); private: static NetConSaveWeightTable* wtable_; diff --git a/src/nrncvode/netcvode.cpp b/src/nrncvode/netcvode.cpp index abb37ddc56..a653602a60 100644 --- a/src/nrncvode/netcvode.cpp +++ b/src/nrncvode/netcvode.cpp @@ -2294,6 +2294,8 @@ thread_local int g_fornet_active_base = -1; thread_local int g_fornet_active_arity = 0; thread_local double* g_fornet_active_buf = nullptr; thread_local int g_fornet_active_is_tmp = 0; +/** @brief Arity from last _nrn_netcon_weight_bases (FOR_NETCONS walk). */ +thread_local int g_fornet_list_arity = 1; double* mod_weight_scratch(int n) { if (n <= 0) { @@ -2341,19 +2343,16 @@ struct ReceiveWeightIndexGuard { } // namespace double* _nrn_netrec_wsoa(int weight_index, int count) { + // CoreNEURON shape: weight_index addresses Weight SoA directly (no NetCon*). g_netrec_wsoa_is_tmp = 0; if (weight_index < 0 || count <= 0) { return nullptr; } - // Zero-copy into contiguous Weight SoA block. - if (NetCon* nc = NetConSave::weight_index2netcon(weight_index)) { - if (nc->has_weight_soa()) { - if (double* p = nc->weight_block_->data_if_contiguous()) { - return p; - } - } + if (double* p = neuron::container::network::SelfEventFields::weight_soa_ptr(weight_index, + count)) { + return p; } - // Scattered: TLS materialize (commit in _nrn_netrec_wsoa_done). + // Scattered / incomplete pack: TLS materialize (commit in _nrn_netrec_wsoa_done). double* buf = mod_weight_scratch(count); neuron::container::network::SelfEventFields::materialize_weight_block(weight_index, count, buf); g_netrec_wsoa_is_tmp = 1; @@ -3526,14 +3525,21 @@ void SelfEvent::savestate_free() { void SelfEvent::savestate_write(FILE* f) { fprintf(f, "%d\n", SelfEventType); int const moff = movable_ ? (movable_ - target_->prop->dparam) : -1; + // File format stores HOC NetCon object index (stable across weight repack). + // Cold path only: O(N) scan — not on the sim delivery path. int ncindex = -1; - // SelfEvent identity for SaveState: NetCon object index via weight_index. - NetCon* owner = nullptr; if (weight_index_ >= 0) { - owner = NetConSave::weight_index2netcon(weight_index_); - } - if (owner && owner->obj_) { - ncindex = owner->obj_->index; + Symbol* sym = hoc_lookup("NetCon"); + if (sym && sym->u.ctemplate && sym->u.ctemplate->olist) { + hoc_Item* q = nullptr; + ITERATE(q, sym->u.ctemplate->olist) { + auto* nc = static_cast(OBJ(q)->u.this_pointer); + if (nc && static_cast(nc->weight_base()) == weight_index_ && nc->obj_) { + ncindex = nc->obj_->index; + break; + } + } + } } fprintf(f, @@ -4403,6 +4409,8 @@ int _nrn_netcon_weight_bases(void* v, int** bases) { auto* fnc = static_cast(v); assert(fnc); *bases = fnc->weight_bases; + // Arity is per-mechanism (same for all peers); stash for _nrn_fornetcon_weight. + g_fornet_list_arity = fnc->arity > 0 ? fnc->arity : 1; return fnc->size; } @@ -4412,13 +4420,10 @@ double* _nrn_fornetcon_weight(int weight_base) { if (weight_base < 0) { return nullptr; } - NetCon* nc = NetConSave::weight_index2netcon(weight_base); - if (!nc || !nc->has_weight_soa()) { - return nullptr; - } - int const arity = nc->cnt_ > 0 ? nc->cnt_ : 1; - // Zero-copy when the weight block is contiguous in SoA (packing A / allocate). - if (double* p = nc->weight_block_->data_if_contiguous()) { + int const arity = g_fornet_list_arity > 0 ? g_fornet_list_arity : 1; + // Direct SoA address (CoreNEURON shape); no NetCon reverse lookup. + if (double* p = neuron::container::network::SelfEventFields::weight_soa_ptr(weight_base, + arity)) { g_fornet_active_base = weight_base; g_fornet_active_arity = arity; g_fornet_active_buf = p; @@ -5248,28 +5253,6 @@ NetCon* NetConSave::index2netcon(long id) { } } -NetCon* NetConSave::weight_index2netcon(int weight_index) { - if (weight_index < 0) { - return nullptr; - } - Symbol* sym = hoc_lookup("NetCon"); - if (!sym || !sym->u.ctemplate || !sym->u.ctemplate->olist) { - return nullptr; - } - hoc_Item* q = nullptr; - ITERATE(q, sym->u.ctemplate->olist) { - auto* nc = static_cast(OBJ(q)->u.this_pointer); - if (!nc) { - continue; - } - int const base = static_cast(nc->weight_base()); - if (base == weight_index) { - return nc; - } - } - return nullptr; -} - void nrn_update_ps2nt() { net_cvode_instance->update_ps2nt(); } diff --git a/src/nrncvode/network_soa_sort.cpp b/src/nrncvode/network_soa_sort.cpp index cf43dbfc80..347f3110c3 100644 --- a/src/nrncvode/network_soa_sort.cpp +++ b/src/nrncvode/network_soa_sort.cpp @@ -359,7 +359,8 @@ void sort_network_data(neuron::cache::Model& cache, // Fanout order is NetCon SoA row indices; rebuild after topology/sort. ::PreSyn::mark_fanout_unsorted(); ::PreSyn::ensure_fanout_order(); - // Weight SoA data pointers may have moved; drop weight2netcon cache. + // Weight SoA data pointers may have moved; drop SaveState helper maps + // (weight2netcon / index2netcon). Sim path resolves weights by SoA index only. ::NetConSave::invalid(); // Final NetCon reverse-edge refresh (src PreSyn rows may have moved). diff --git a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp index 96c0fe303e..0496fa4edf 100644 --- a/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp +++ b/src/nrniv/nrncore_write/callbacks/nrncore_callbacks.cpp @@ -5,6 +5,7 @@ #include "nrnmpi.h" #include "section.h" #include "netcon.h" +#include "neuron/container/network/self_event.hpp" #include "nrncvode.h" #include "nrniv_mf.h" #include "hocdec.h" @@ -939,12 +940,11 @@ static void set_info(TQItem* tqi, Point_process* pnt = se->target_; int type = pnt->prop->_type; int movable_index = type2movable[type]; - // Heap-free 7c: SelfEvent identity is weight_index only. + // Heap-free 7c: SelfEvent identity is weight_index only — address SoA by index + // (CoreNEURON shape; no NetCon reverse lookup). double* wt = nullptr; if (se->weight_index_ >= 0) { - if (NetCon* nc = NetConSave::weight_index2netcon(se->weight_index_)) { - wt = nc->weight_soa_data(); - } + wt = neuron::container::network::SelfEventFields::weight_soa_ptr(se->weight_index_, 1); } core_te->intdata.push_back(type); From a49fbad5700ffe948df92cc59fe1c019da07181e Mon Sep 17 00:00:00 2001 From: Michael Hines Date: Sat, 25 Jul 2026 19:21:22 -0400 Subject: [PATCH 33/33] nrnivmodl: default NMODL_PYLIB and NMODLHOME when unset Classic nrnivmodl did not export NMODL_PYLIB or NMODLHOME, so nmodl aborted when NRN_ENABLE_NMODL is ON (or -nmodl). Set NMODL_PYLIB from configure-time PYTHON_LIBRARY and NMODLHOME from the install prefix. Harmless with nocmodl. --- bin/nrnivmodl.in | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/bin/nrnivmodl.in b/bin/nrnivmodl.in index d858beeeb2..b8f7f6144f 100755 --- a/bin/nrnivmodl.in +++ b/bin/nrnivmodl.in @@ -35,6 +35,21 @@ export prefix export bindir export libdir +# NMODL (default when NRN_ENABLE_NMODL=ON, or via -nmodl) embeds Python and +# requires NMODL_PYLIB + NMODLHOME. Prefer existing env values; otherwise use +# the Python library from configure and the install/build prefix of this +# script. Harmless when using nocmodl. +if [ -z "${NMODL_PYLIB:-}" ]; then + _nrn_nmodl_pylib="@PYTHON_LIBRARY@" + if [ -n "${_nrn_nmodl_pylib}" ]; then + export NMODL_PYLIB="${_nrn_nmodl_pylib}" + fi + unset _nrn_nmodl_pylib +fi +if [ -z "${NMODLHOME:-}" ]; then + export NMODLHOME="$(cd "$(dirname "${0}")/.." && pwd)" +fi + if command -v xcrun >/dev/null 2>&1; then @NRN_OSX_BUILD_TRUE@export SDKROOT=$(xcrun --sdk macosx --show-sdk-path) @NRN_OSX_BUILD_TRUE@export MACOSX_DEPLOYMENT_TARGET="@CMAKE_OSX_DEPLOYMENT_TARGET@"