Merge pull request #2 from justinlietz93/02-experimental

02 experimental

Author: justinlietz93

.gitignore

@@ -9,7 +9,6 @@ venv/
 runs/
 *.log
 .DS_Store
-reports
 .arachnid_scans
 
 # Experiments

README.md

@@ -54,10 +54,10 @@ python -m vdm_rt.run_nexus --neurons 800 --hz 10 --domain biology_consciousness
 Artifacts land in `runs/<timestamp>/` by default:
 
 ```text
-events.jsonl          structured runtime logs
-utd_events.jsonl      explicit motor-event audit records when a motor port emits
-phase.json           optional external control-plane input when present
-state_<step>.h5       checkpoint when --checkpoint-every is enabled
+events.jsonl.zst            internal runtime dynamics as compressed JSONL
+motor_traces.jsonl.zst      UTE, efferent, afferent, actuator, witness, and UTD trace rows
+phase.json                  optional external control-plane input when present
+state_<step>.h5             checkpoint when --checkpoint-every is enabled
 ```
 
 ## Runtime boundaries

config/README.md

@@ -24,8 +24,7 @@ Current files:
 - `learning.toml` - experimental REVGSP/GDSP adapter controls; the default SparseConnectome path does not yet satisfy their required substrate contract.
 - `scouts.toml` - void-walker scout budgets and per-scout enable flags.
 - `io.toml` - receptor queue, HTTP status, and Redis status output.
-- `logging.toml` - JSONL and zip spool limits. `events.jsonl` uses the
-  `events_*` limits, names containing `utd` use `utd_*`, and other JSONL files
-  use `log_*`; `roll_check_every = 1` enforces active-file caps after each write.
-  `zip_spool = true` instead selects the zip-buffer strategy and uses
-  `zip_buffer_bytes` and `zip_ring_bytes` for those standard writers.
+- `logging.toml` - JSONL-in-zstd runtime log limits. `events.jsonl.zst` uses
+  the `events_*` limits, `motor_traces.jsonl.zst` uses `motor_trace_*`, and
+  other compressed JSONL files use `log_*`; the MiB caps are enforced against
+  the uncompressed JSONL payload size.

config/logging.toml

@@ -1,32 +1,31 @@
-# Runtime file logging and spool limits.
+# Runtime file logging and compressed JSONL segment limits.
 
 [logging]
-# Choose the active-file strategy for standard event and UTD writers.
-# true: use zip_buffer_bytes/zip_ring_bytes and append rotated buffers to a zip.
-# false: use the category-specific rolling limits below and archive old lines.
-zip_spool = true
+# Runtime logs are JSONL records compressed immediately with zstd. Active files
+# use the .jsonl.zst suffix; archived rotated segments keep the same suffix.
+# Runtime records use tick for endogenous model time, and wall_time_s plus
+# run_elapsed_s for external clock time.
+compression = "zstd"
+zstd_level = 3
 
-# Rolling JSONL caps, in MiB where the name says mb. Zero line caps mean
-# "unset; use byte caps". Active files are checked after every write by
-# default, so small diagnostic runs honor these limits immediately.
+# Active compressed streams rotate when their uncompressed JSONL payload reaches
+# the configured MiB cap. Zero line caps mean "unset; use byte caps".
 #
-# Category selection is filename-based: events.jsonl uses events_*, files whose
-# names contain "utd" use utd_*, and all other JSONL files use log_*.
+# Category selection is filename-based: events.jsonl uses events_*,
+# motor_traces.jsonl uses motor_trace_*, and all other JSONL files use log_*.
 events_max_mb = 256
 events_max_lines = 0
+motor_trace_max_mb = 256
+motor_trace_max_lines = 0
+log_max_mb = 256
+log_max_lines = 0
+
+# Retained for the plain JSONL debug writer only; the default zstd writer checks
+# rotation after every record.
 events_archive_segment_mb = 512
 events_archive_segment_lines = 0
-utd_max_mb = 256
-utd_max_lines = 0
-utd_archive_segment_mb = 512
-utd_archive_segment_lines = 0
-log_max_mb = 128
-log_max_lines = 0
+motor_trace_archive_segment_mb = 512
+motor_trace_archive_segment_lines = 0
 log_archive_segment_mb = 256
 log_archive_segment_lines = 0
 roll_check_every = 1
-
-# Zip spooler byte caps. Values are exact positive byte limits; use a small
-# value for a quick local rotation test and a larger value for normal runs.
-zip_buffer_bytes = 1048576
-zip_ring_bytes = 65536

docs/contracts/runtime-cleanup-classification.yml

@@ -197,7 +197,8 @@ entries:
     evidence:
       - The mapping skipped repeated symbols before creating indices.
       - The file was deleted from the importable runtime tree.
-      - runtime/helpers/ingest.py no longer maps text into connectome indices.
+      - runtime/helpers/ingest.py no longer maps text into connectome indices; it accepts only explicit receptor/stimulation node-index fields.
+      - runtime/loop/main.py records successful explicit-index stimulation under trace_kind="stimulation" in motor_traces.jsonl.zst.
       - The source authority requires sequential, repetition-preserving receptor slices.
     follow_up: Replace with a bounded receptor-slice port when implementing the receptor rewrite; do not preserve the unique-symbol mapping.
 
@@ -214,10 +215,11 @@ entries:
   - path: io/utd.py
     class: port
     state: motor_event_boundary_only
-    role: Explicit motor-event audit boundary.
+    role: Explicit UTD actuation producer for the sensorimotor trace.
     evidence:
       - Text emission, macro registration, macro persistence, and input echo behavior were removed.
-      - UTD now records explicit motor events only.
+      - UTD now records explicit actuation rows only, under trace_kind="utd_actuation" in motor_traces.jsonl.zst.
+      - runtime/loop/main.py can forward emitted attached-actuator witness events through UTD; no default actuator implementation is installed here.
       - The source authority requires fixed actuator primitives and actuator events, not a text decoder.
     follow_up: Replace with an actuator-transducer contract during the motor rewrite.
 

docs/pages/definitions/vdm-definitions.md

@@ -0,0 +1,27 @@
+# Definitions
+
+<!-- TODO: Definitions are stale here and should be expanded with links to the code files each metric is defined at -->
+
+| Metric | Role in the runtime | Current source |
+| --- | --- | --- |
+| `active_edges` | structural load / active synaptic edge count | events, dashboard, H5-derived analyses |
+| `active_synapses` | structural activity proxy closely related to `active_edges` | events |
+| `connectome_entropy` | graph-distribution entropy / structural disorder metric | events, dashboard |
+| `vt_coverage` | traversal coverage fraction | events, dashboard |
+| `vt_entropy` | traversal/topic entropy | events, dashboard |
+| `b1_z` | boundary pulse / motor-gate proxy | events, dashboard, UTD `why` |
+| `adc_territories` | active hierarchy/territory count | events, dashboard, H5 `adc_json` |
+| `adc_boundaries` | boundary count in ADC summary | events |
+| `cohesion_components` | topological integration/fragmentation state | events, UTD `why` |
+| `sie_total_reward` | SIE-v1 aggregate reward-like signal | events |
+| `sie_valence_01` | normalized SIE-v1 valence | events, UTD `why` |
+| `sie_td_error` | SIE-v1 TD-like error channel | events |
+| `sie_v2_reward_mean` | SIE-v2 reward mean | events |
+| `sie_v2_valence_01` | normalized SIE-v2 valence | events, dashboard, UTD `why` |
+| `a_mean` | mean `a`/alpha-like field term | events |
+| `omega_mean` | mean omega-like field term | events |
+| `homeostasis_pruned` | number pruned in current step | events |
+| `homeostasis_bridged` | number bridged in current step | events |
+| `phase` | runtime phase label in event/UTD output | events, UTD `why` |
+| `ute_in_count` / `ute_text_count` | ingress counts | events, UTD |
+| `did_say` | offline say-event marker from UTD alignment | derived tick table |

docs/pages/getting-started/runtime-quickstart.md

@@ -33,8 +33,8 @@ PYTHONPATH=. pytest -q vdm_rt/tests
 A run directory contains structured runtime artifacts such as:
 
 ```text
-events.jsonl
-utd_events.jsonl
+events.jsonl.zst
+motor_traces.jsonl.zst
 phase.json
 state_<step>.h5
 ```

docs/roadmap/motor-learning-system/IO_AUDIT.md

@@ -27,9 +27,9 @@ they must not be hidden by decoder removal.
 | `runtime/helpers/speak.py`, lexical state in `nexus.py`, and the lexicon save call in `runtime/loop/main.py` | deleted or removed | These callers made the prohibited decoder live. The release gate no longer composes content. |
 | `io/actuators/macros.py`, `runtime/emitters.py`, `runtime/helpers/macro_board.py`, and macro smoke output | deleted from the live output path | The macro registry and its canned text emitters were not an actuator basis. |
 | `io/actuators/thoughts.py` and thought smoke output | deleted | It had no runtime producer beyond optional smoke material and was not required provenance logging. |
-| `io/cognition/stimulus.py` and the text portion of `runtime/helpers/ingest.py` | deleted or disabled | The unique-symbol mapping is removed. `process_messages()` now counts messages only; it does not stimulate the connectome or echo through UTD. |
+| `io/cognition/stimulus.py` and the text portion of `runtime/helpers/ingest.py` | deleted or disabled | The unique-symbol mapping is removed. `process_messages()` does not map text. It accepts only explicit receptor/stimulation node-index fields, which the loop can pass to `SparseConnectome.stimulate_indices()`, and it does not echo through UTD. |
 | `io/ute.py` | boundary port only | UTE now exposes an explicit queue and no stdin, chat-inbox, or synthetic ticker source. |
-| `io/utd.py` and `runtime/helpers/emission.py` | boundary port only | The status/text/macro emission helper is deleted. UTD has no text or macro API and records only explicit motor events. |
+| `io/utd.py` and `runtime/helpers/emission.py` | boundary port only | The status/text/macro emission helper is deleted. UTD has no text or macro API and records only explicit `utd_actuation` rows in `motor_traces.jsonl.zst`. |
 | `io/logging/rolling_jsonl.py` | keep | Bounded external audit logging is allowed when it does not feed cognition or stand in for reafference. |
 
 ## Static Reachability
@@ -54,7 +54,11 @@ authoring under live runtime roots. It proves:
 1. No live output path imports the composer, speaker, lexicon, phrase bank, or macro emitter.
 2. The runtime has no release-time sentence composition or `say` macro path.
 3. Input receipt never causes outbound UTD emission.
-4. The remaining UTE and UTD surfaces are documented as incomplete receptor and actuator ports, not compliant motor-learning implementations.
+4. Explicit receptor-node indices can stimulate the connectome without restoring text mapping.
+5. The remaining UTE and UTD surfaces are documented as incomplete receptor and actuator ports, not compliant motor-learning implementations.
 
 This audit does not claim that the future motor actuator, articulation trace,
-or sequential receptor is implemented. Those remain roadmap work.
+or sequential receptor is implemented. The live loop now records stimulation,
+efferent observation-node summaries, and optional attached-actuator witness/UTD
+handoffs into `motor_traces.jsonl.zst`; a default actuator implementation
+remains roadmap work.