Skip to content

poc: portable network state serialization (.brian format)#1799

Draft
Sanchit2662 wants to merge 1 commit into
brian-team:masterfrom
Sanchit2662:poc/serialization-brian-format
Draft

poc: portable network state serialization (.brian format)#1799
Sanchit2662 wants to merge 1 commit into
brian-team:masterfrom
Sanchit2662:poc/serialization-brian-format

Conversation

@Sanchit2662
Copy link
Copy Markdown
Contributor

@Sanchit2662 Sanchit2662 commented Mar 21, 2026

What this is

This is a proof-of-concept for the GSoC 2026 project on serialization/deserialization for Brian2.
I'm submitting it early to get feedback on the core approach before building the full implementation on top of it.


What I built

Three new files:

brian2/serialization/brian_format.py

The core module. Two functions:

  • serialize_network_state(net, filepath) , takes a running Network, walks all its objects, and writes a .brian ZIP archive to disk
  • restore_network_state(net, filepath) , reads that archive and restores every object back to the exact state it was in

examples/serialization/brian_format_demo.py

A runnable end-to-end demo.

It builds a LIF network (NeuronGroup + Synapses with per-neuron delays + StateMonitor + SpikeMonitor), runs 5 ms, serializes the state, resets the network to t=0 using the existing pickle store/restore, restores from the .brian archive to t=5 ms, runs another 5 ms, and checks that the final traces are numerically identical to a reference run that ran straight through to 10 ms.

Max error is 0.00e+00.

README_POC.md

Explains what the PoC covers, what it doesn't, and how to run it.


How it works

The existing Network.store() already has a method _full_state() that walks all objects in the network and collects their array state. The current code just pickles that and writes it to disk.

I'm calling the same _full_state() but writing the output differently.

The .brian archive is a ZIP with three files inside:

metadata.json

Human-readable.

Contains the simulation time, and for every object and every variable:

  • shape
  • dtype
  • SI dimension as a 7-element list of exponents (from Dimension._dims the 7 SI base units)

This means the archive carries unit information that pickle never exposed.

arrays.npz

All the actual numerical data.

  • Every array is stored under a flat key "ObjectName__varname"
  • StateMonitor's 2D recorded arrays are handled correctly (size is stored as a tuple, not an int)

spikequeues.json

This one is important.

SynapticPathway._full_state() returns the Cython SpikeQueue state as (offset, list_of_lists) the in-flight spikes that are mid-delay at the moment of serialization.

I convert this to a JSON dict so it survives the round-trip without pickle.

Without this, restoring a network mid-simulation would silently drop spikes that haven't been delivered yet.


For restore:

  • Reconstruct the state dict that _restore_from_full_state() expects
  • Load the npz arrays and spike queue data
  • Walk the live network objects the same way _full_state() does
  • Call obj._restore_from_full_state() on each one

No changes to any existing Brian2 internals needed.


What this isn't

This is only the RuntimeDevice path.

It does not yet handle CPPStandaloneDevice that's where the real NotImplementedError lives.

The plan for that is the same format, but arrays are read from:

results/<filename>

via get_array_filename(var) after the binary has run, instead of from live NumPy arrays.

The archive layout stays identical.


This also does not store model structure equations, thresholds, connectivity conditions.

The network objects have to already exist before you can restore into them.
That's the same contract as the existing Network.store/restore.

The structural side (BrianExporter, BrianImporter) is a separate deliverable.


Questions for reviewers

  1. Is brian2/serialization/ the right place for this, or should it live inside brian2/importexport/ next to the existing ImportExport registry?

  2. I'm reading Dimension._dims directly to serialize unit information.
    Is there a more stable public-facing API I should use instead?

  3. For the CPP standalone path is reading from results/ after has_been_run is set the right hook point, or is there somewhere better in the device lifecycle to intercept this?

Signed-off-by: Sanchit2662 <sanchit2662@gmail.com>
@mstimberg
Copy link
Copy Markdown
Member

Hi @Sanchit2662, given that this project is not part of this year's GSoC I'll assume that we can close the PR for now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants