Version 2.4.0

This branch of releases (brainpy==2.4.x) are going to support the large-scale modeling for brain dynamics.

As the start, this release provides support for automatic object-oriented (OO) transformations.

What's New

1. Automatic OO transformations on longer need to take dyn_vars or child_objs information.
   These transformations are capable of automatically inferring the underlying dynamical variables.
   Specifically, they include:

   • brainpy.math.grad and other autograd functionalities
   • brainpy.math.jit
   • brainpy.math.for_loop
   • brainpy.math.while_loop
   • brainpy.math.ifelse
   • brainpy.math.cond

2. Update documentation

3. Fix several bugs

What's Changed

• reorganize operators in brainpy.math by @chaoming0625 in #357
• Automatic transformations any function/object using brainpy.math.Variable by @chaoming0625 in #358
• New OO transforms support jax.disable_jit mode by @chaoming0625 in #359
• [oo transform] Enable new style of jit transformation to support static_argnums and static_argnames by @chaoming0625 in #360
• [documentation] update documentation to brainpy>=2.4.0 by @chaoming0625 in #361

Full Changelog: V2.3.8...V2.4.0

Version 2.3.8

This release continues to add support for improving the usability of BrainPy.

New Features

1. New data structures for object-oriented transformations.
   • NodeList and NodeDict for a list/tuple/dict of BrainPyObject instances.
   • ListVar and DictVar for a list/tuple/dict of brainpy data.
2. Clip transformation for brainpy initializers.
3. All brainpylib operators are accessible in brainpy.math module. Especially there are some dedicated operators for scaling up the million-level neuron networks. For an example, see example in Simulating 1-million-neuron networks with 1GB GPU memory
4. Enable monitoring GPU models on CPU when setting DSRunner(..., memory_efficient=True). This setting can usually reduce so much memory usage.
5. brainpylib wheels on the Linux platform support the GPU operators. Users can install GPU version of brainpylib (require brainpylib>=0.1.7) directly by pip install brainpylib. @ztqakita

What's Changed

• Fix bugs and add more variable structures: ListVar and DictVar by @chaoming0625 in #345
• add CI for testing various models by @chaoming0625 in #346
• Update docs and tests by @chaoming0625 in #347
• Fix `Runner(jit=False)`` bug by @chaoming0625 in #348
• Compatible with jax>=0.4.7 by @chaoming0625 in #349
• Updates by @chaoming0625 in #350
• reconstruct BrainPy by merging brainpylib by @ztqakita in #351
• Intergate brainpylib operators into brainpy by @chaoming0625 in #352
• fix brainpylib call bug by @chaoming0625 in #354
• Enable memory-efficient DSRunner by @chaoming0625 in #355
• fix Array transform bug by @chaoming0625 in #356

Full Changelog: V2.3.7...V2.3.8

Version 2.3.7

• Fix bugs on population models in brainpy.rate module
• Fix bug on brainpy.LoopOverTime
• Add more synaptic models including DualExpoenetial model and Alpha model in brainpy.experimental module
• Support call a module through right shift, such as data >> module1 >> module2

Version 2.3.6

This release continues to add support for brain-inspired computation.

New Features

More flexible customization of surrogate gradient functions.

• brainpy.math.surrogate.Sigmoid
• brainpy.math.surrogate.PiecewiseQuadratic
• brainpy.math.surrogate.PiecewiseExp
• brainpy.math.surrogate.SoftSign
• brainpy.math.surrogate.Arctan
• brainpy.math.surrogate.NonzeroSignLog
• brainpy.math.surrogate.ERF
• brainpy.math.surrogate.PiecewiseLeakyRelu
• brainpy.math.surrogate.SquarewaveFourierSeries
• brainpy.math.surrogate.S2NN
• brainpy.math.surrogate.QPseudoSpike
• brainpy.math.surrogate.LeakyRelu
• brainpy.math.surrogate.LogTailedRelu
• brainpy.math.surrogate.ReluGrad
• brainpy.math.surrogate.GaussianGrad
• brainpy.math.surrogate.InvSquareGrad
• brainpy.math.surrogate.MultiGaussianGrad
• brainpy.math.surrogate.SlayerGrad

Fix bugs

• brainpy.LoopOverTime

Version 2.3.5

This release continues to add support for brain-inspired computation.

New Features

1. brainpy.share for sharing data across submodules

In this release, we abstract the shared data as a brainpy.share object.

This object together with brainpy.Delay we will introduce below constitutes the support that enables us to define SNN models like ANN ones.

2. brainpy.Delay for delay processing

Delay is abstracted as a dynamical system, which can be updated/retrieved by users.

import brainpy as bp

class EINet(bp.DynamicalSystemNS):
  def __init__(self, scale=1.0, e_input=20., i_input=20., delay=None):
    super().__init__()

    self.bg_exc = e_input
    self.bg_inh = i_input

    # network size
    num_exc = int(3200 * scale)
    num_inh = int(800 * scale)

    # neurons
    pars = dict(V_rest=-60., V_th=-50., V_reset=-60., tau=20., tau_ref=5.,
                V_initializer=bp.init.Normal(-55., 2.), input_var=False)
    self.E = bp.neurons.LIF(num_exc, **pars)
    self.I = bp.neurons.LIF(num_inh, **pars)

    # synapses
    we = 0.6 / scale  # excitatory synaptic weight (voltage)
    wi = 6.7 / scale  # inhibitory synaptic weight
    self.E2E = bp.experimental.Exponential(
      bp.conn.FixedProb(0.02, pre=self.E.size, post=self.E.size),
      g_max=we, tau=5., out=bp.experimental.COBA(E=0.)
    )
    self.E2I = bp.experimental.Exponential(
      bp.conn.FixedProb(0.02, pre=self.E.size, post=self.I.size, ),
      g_max=we, tau=5., out=bp.experimental.COBA(E=0.)
    )
    self.I2E = bp.experimental.Exponential(
      bp.conn.FixedProb(0.02, pre=self.I.size, post=self.E.size),
      g_max=wi, tau=10., out=bp.experimental.COBA(E=-80.)
    )
    self.I2I = bp.experimental.Exponential(
      bp.conn.FixedProb(0.02, pre=self.I.size, post=self.I.size),
      g_max=wi, tau=10., out=bp.experimental.COBA(E=-80.)
    )
    self.delayE = bp.Delay(self.E.spike, entries={'E': delay})
    self.delayI = bp.Delay(self.I.spike, entries={'I': delay})

  def update(self):
    e_spike = self.delayE.at('E')
    i_spike = self.delayI.at('I')
    e_inp = self.E2E(e_spike, self.E.V) + self.I2E(i_spike, self.E.V) + self.bg_exc
    i_inp = self.I2I(i_spike, self.I.V) + self.E2I(e_spike, self.I.V) + self.bg_inh
    self.delayE(self.E(e_inp))
    self.delayI(self.I(i_inp))

3. brainpy.checkpoints.save_pytree and brainpy.checkpoints.load_pytree for saving/loading target from the filename

Now we can directly use brainpy.checkpoints.save_pytree to save a network state into the file path we specified.

Similarly, we can use brainpy.checkpoints.load_pytree to load states from the given file path.

4. More ANN layers

• brainpy.layers.ConvTranspose1d
• brainpy.layers.ConvTranspose2d
• brainpy.layers.ConvTranspose3d
• brainpy.layers.Conv1dLSTMCell
• brainpy.layers.Conv2dLSTMCell
• brainpy.layers.Conv3dLSTMCell

5. More compatible dense operators

PyTorch operators:

• brainpy.math.Tensor
• brainpy.math.flatten
• brainpy.math.cat
• brainpy.math.abs
• brainpy.math.absolute
• brainpy.math.acos
• brainpy.math.arccos
• brainpy.math.acosh
• brainpy.math.arccosh
• brainpy.math.add
• brainpy.math.addcdiv
• brainpy.math.addcmul
• brainpy.math.angle
• brainpy.math.asin
• brainpy.math.arcsin
• brainpy.math.asinh
• brainpy.math.arcsin
• brainpy.math.atan
• brainpy.math.arctan
• brainpy.math.atan2
• brainpy.math.atanh

TensorFlow operators:

• brainpy.math.concat
• brainpy.math.reduce_sum
• brainpy.math.reduce_max
• brainpy.math.reduce_min
• brainpy.math.reduce_mean
• brainpy.math.reduce_all
• brainpy.math.reduce_any
• brainpy.math.reduce_logsumexp
• brainpy.math.reduce_prod
• brainpy.math.reduce_std
• brainpy.math.reduce_variance
• brainpy.math.reduce_euclidean_norm
• brainpy.math.unsorted_segment_sqrt_n
• brainpy.math.segment_mean
• brainpy.math.unsorted_segment_sum
• brainpy.math.unsorted_segment_prod
• brainpy.math.unsorted_segment_max
• brainpy.math.unsorted_segment_min
• brainpy.math.unsorted_segment_mean
• brainpy.math.segment_sum
• brainpy.math.segment_prod
• brainpy.math.segment_max
• brainpy.math.segment_min
• brainpy.math.clip_by_value
• brainpy.math.cast

Others

• Remove the hard requirements of brainpylib and numba.

Version 2.3.4

This release mainly focuses on the compatibility with other frameworks:

1. Fix Jax import error when jax>=0.4.2
2. Backward compatibility of brainpy.dyn module
3. Start to implement and be compatible with operators in pytorch and tensorflow, so that user's pytorch/tensorflow models can be easily migrated to brainpy

Full Changelog: V2.3.3...V2.3.4

Version 2.3.3

Improve backward compatibility:

• monitors and inputs in DSRunner
• models in brainpy.dyn
• constants and function in brainpy.analysis

Version 2.3.2

This release (under the branch of brainpy=2.3.x) continues to add support for brain-inspired computation.

New Features

1. New package structure for stable API release

Unstable APIs are all hosted in brainpy._src module.
Other APIs are stable and will be maintained for a long time.

2. New schedulers

• brainpy.optim.CosineAnnealingWarmRestarts
• brainpy.optim.CosineAnnealingLR
• brainpy.optim.ExponentialLR
• brainpy.optim.MultiStepLR
• brainpy.optim.StepLR

3. Others

• support static_argnums in brainpy.math.jit
• fix bugs of reset_state() and clear_input() in brainpy.channels
• fix jit error checking

Version 2.3.1

This release (under the release branch of brainpy=2.3.x) continues to add supports for brain-inspired computation.

import brainpy as bp
import brainpy.math as bm

Backwards Incompatible Changes

1. Error: module 'brainpy' has no attribute 'datasets'

brainpy.datasets module is now published as an independent package brainpy_datasets.

Please change your dataset access from

bp.datasets.xxxxx

to

import brainpy_datasets as bp_data

bp_data.chaos.XXX
bp_data.vision.XXX

For a chaotic data series,

# old version
data = bp.datasets.double_scroll_series(t_warmup + t_train + t_test, dt=dt)
x_var = data['x']
y_var = data['y']
z_var = data['z']

# new version
data = bd.chaos.DoubleScrollEq(t_warmup + t_train + t_test, dt=dt)
x_var = data.xs
y_var = data.ys
z_var = data.zs

For a vision dataset,

# old version
dataset = bp.datasets.FashionMNIST(root, train=True, download=True)

# new version
dataset = bd.vision.FashionMNIST(root, split='train', download=True)

2. Error: DSTrainer must receive an instance with BatchingMode

This error will happen when using brainpy.OnlineTrainer , brainpy.OfflineTrainer, brainpy.BPTT , brainpy.BPFF.

From version 2.3.1, BrainPy explicitly consider the computing mode of each model. For trainers, all training target should be a model with BatchingMode or TrainingMode.

If you are training model with OnlineTrainer or OfflineTrainer,

# old version
class NGRC(bp.DynamicalSystem):
  def __init__(self, num_in):
    super(NGRC, self).__init__()
    self.r = bp.layers.NVAR(num_in, delay=2, order=3)
    self.di = bp.layers.Dense(self.r.num_out, num_in)

  def update(self, sha, x):
    di = self.di(sha, self.r(sha, x))
    return x + di


# new version
bm.set_enviroment(mode=bm.batching_mode)

class NGRC(bp.DynamicalSystem):
  def __init__(self, num_in):
    super(NGRC, self).__init__()
    self.r = bp.layers.NVAR(num_in, delay=2, order=3)
    self.di = bp.layers.Dense(self.r.num_out, num_in, mode=bm.training_mode)

  def update(self, sha, x):
    di = self.di(sha, self.r(sha, x))
    return x + di

If you are training models with BPTrainer, adding the following line at the top of the script,

bm.set_enviroment(mode=bm.training_mode)

3. Error: inputs_are_batching is no longer supported.

This is because if the training target is in batching mode, this has already indicated that the inputs should be batching.

Simple remove the inputs_are_batching from your functional call of .predict() will solve the issue.

New Features

1. brainpy.math module upgrade

brainpy.math.surrogate module for surrogate gradient functions.

Currently, we support

• brainpy.math.surrogate.arctan
• brainpy.math.surrogate.erf
• brainpy.math.surrogate.gaussian_grad
• brainpy.math.surrogate.inv_square_grad
• brainpy.math.surrogate.leaky_relu
• brainpy.math.surrogate.log_tailed_relu
• brainpy.math.surrogate.multi_gaussian_grad
• brainpy.math.surrogate.nonzero_sign_log
• brainpy.math.surrogate.one_input
• brainpy.math.surrogate.piecewise_exp
• brainpy.math.surrogate.piecewise_leaky_relu
• brainpy.math.surrogate.piecewise_quadratic
• brainpy.math.surrogate.q_pseudo_spike
• brainpy.math.surrogate.relu_grad
• brainpy.math.surrogate.s2nn
• brainpy.math.surrogate.sigmoid
• brainpy.math.surrogate.slayer_grad
• brainpy.math.surrogate.soft_sign
• brainpy.math.surrogate.squarewave_fourier_series

New transformation function brainpy.math.to_dynsys

New transformation function brainpy.math.to_dynsys supports to transform a pure Python function into a DynamicalSystem. This will be useful when running a DynamicalSystem with arbitrary customized inputs.

import brainpy.math as bm

hh = bp.neurons.HH(1)

@bm.to_dynsys(child_objs=hh)
def run_hh(tdi, x=None):
    if x is not None:
	    hh.input += x
    
runner = bp.DSRunner(run_hhh, monitors={'v': hh.V})
runner.run(inputs=bm.random.uniform(3, 6, 1000))

Default data types

Default data types brainpy.math.int_, brainpy.math.float_ and brainpy.math.complex_ are initialized according to the default x64 settings. Then, these data types can be set or get by brainpy.math.set_* or brainpy.math.get_* syntaxes.

Take default integer type int_ as an example,

# set the default integer type
bm.set_int_(jax.numpy.int64)

# get the default integer type
a1 = bm.asarray([1], dtype=bm.int_)
a2 = bm.asarray([1], dtype=bm.get_int()) # equivalent

Default data types are changed according to the x64 setting of JAX. For instance,

bm.enable_x64()
assert bm.int_ == jax.numpy.int64
bm.disable_x64()
assert bm.int_ == jax.numpy.int32

brainpy.math.float_ and brainpy.math.complex_ behaves similarly with brainpy.math.int_.

Environment context manager

This release introduces a new concept computing environment in BrainPy. Computing environment is a default setting for current computation jobs, including the default data type (int_, float_, complex_), the default numerical integration precision (dt), the default computing mode (mode). All models, arrays, and computations using the default setting will be carried out under the environment setting.

Users can set a default environment through

brainpy.math.set_environment(mode, dt, x64)

However, ones can also construct models or perform computation through a temporal environment context manager, this can be implemented through:

# constructing a HH model with dt=0.1 and x64 precision
with bm.environment(mode, dt=0.1, x64=True):
    hh1 = bp.neurons.HH(1)
    
# constructing a HH model with dt=0.05 and x32 precision
with bm.environment(mode, dt=0.05, x64=False):
    hh2 = bp.neuron.HH(1)

Usually, users construct models for either brain-inspired computing (training mode) or brain simulation (nonbatching mode), therefore, there are shortcut context manager for setting a training environment or batching environment:

with bm.training_environment(dt, x64):
    pass

with bm.batching_environment(dt, x64):
    pass

2. brainpy.dyn module

brainpy.dyn.transfom module for transforming a DynamicalSystem instance to a callable BrainPyObject.

Specifically, we provide

• LoopOverTime for unrolling a dynamical system over time.
• NoSharedArg for removing the dependency of shared arguments.

3. Running supports in BrainPy

All brainpy.Runner now are subclasses of BrainPyObject

This means that all brainpy.Runner can be used as a part of the high-level program or transformation.

Enable the continuous running of a differential equation (ODE, SDE, FDE, DDE, etc.) with IntegratorRunner.

For example,

import brainpy as bp

# differential equation
a, b, tau = 0.7, 0.8, 12.5
dV = lambda V, t, w, Iext: V - V * V * V / 3 - w + Iext
dw = lambda w, t, V: (V + a - b * w) / tau
fhn = bp.odeint(bp.JointEq([dV, dw]), method='rk4', dt=0.1)

# differential integrator runner
runner = bp.IntegratorRunner(fhn, monitors=['V', 'w'], inits=[1., 1.])

# run 1
Iext, duration = bp.inputs.section_input([0., 1., 0.5], [200, 200, 200], return_length=True)
runner.run(duration, dyn_args=dict(Iext=Iext))
bp.visualize.line_plot(runner.mon.ts, runner.mon['V'], legend='V')

# run 2
Iext, duration = bp.inputs.section_input([0.5], [200], return_length=True)
runner.run(duration, dyn_args=dict(Iext=Iext))
bp.visualize.line_plot(runner.mon.ts, runner.mon['V'], legend='V-run2', show=True)

Enable call a customized function during fitting of brainpy.BPTrainer.

This customized function (provided through fun_after_report) will be useful to save a checkpoint during the training. For instance,

class CheckPoint:
    def __init__(self, path='path/to/directory/'):
        self.max_acc = 0.
        self.path = path
        
    def __call__(self, idx, metrics, phase):
        if phase == 'test' and metrics['acc'] > self.max_acc:
            self.max_acc = matrics['acc']
            bp.checkpoints.save(self.path, net.state_dict(), idx)

trainer = bp.BPTT()
trainer.fit(..., fun_after_report=CheckPoint())    

Enable data with data_first_axis format when predicting or fitting in a brainpy.DSRunner and brainpy.DSTrainer.

Previous version of BrainPy only supports data with the batch dimension at the first axis. Currently, brainpy.DSRunner and brainpy.DSTrainer can support the data with the time dimension at the first axis. This can be set through data_first_axis='T' when initializing a runner or trainer.

runner = bp.DSRunner(..., data_first_axis='T')
trainer = bp.DSTrainer(..., data_first_axis='T')

4. Utility in BrainPy

brainpy.encoding module for encoding rate values into spike trains

Currently, we support

• brainpy.encoding.LatencyEncoder
• brainpy.encoding.PoissonEncoder
• brainpy.encoding.WeightedPhaseEncoder

brainpy.checkpoints module for model state serialization.

This version of BrainPy supports to save a checkpoint of the model into the physical disk. Inspired from the Flax API, we provide the following checkpoint APIs:

• brainpy.checkpoints.save() for saving a checkpoint of the model.
• brainpy.checkpoints.multiprocess_save() for saving a checkpoint of the model in multi-process environment.
• brainpy.checkpoints.load() for loadin...

Version 2.3.0

This branch of releases aims to provide a unified computing framework for brain simulation and brain-inspired computing.

New features

1. brainpy.BPTT supports train_data and test_data with general Python iterators. For instance, one can train a model with PyTorch dataloader or TensorFlow datasets.

import torchvision
from torch.utils.data import DataLoader
data = torchvision.datasets.CIFAR10("./CIFAR10", train=False, transform=torchvision.transforms.ToTensor())
loader = DataLoader(dataset=data, batch_size=4, shuffle=True, num_workers=0, drop_last=False)

# any generator can be used for train_data or test_data
trainer = bp.BPTT()
trainer.fit(loader)

2. Consolidated object-oriented transformation in brainpy.math.object_transform module. All brainpy transformations generate a new BrainPyObject instance so that objects in brainpy can be composed hierarchically. brainpy.math.to_object() transformation transforms a pure Python function into a BrainPyObject.

3. New documentation is currently online for introducing the consolidated BrainPy concept of object-oriented transformation.

4. Change brainpy.math.JaxArray to brainpy.math.Array.

Deprecations

1. brainpy.datasets module is no longer supported. New APIs will be moved into brainpy-datasets package.
2. brainpy.train.BPTT no longer support to receive the train data [X, Y]. Instead, users should provide a data generator such like pytorch dataset or tensorflow dataset.
3. The update function of brainpy.math.TimeDealy does not support receiving a time index. Instead, one can update the new data by directly using TimeDealy.update(data) instead of TimeDealy.update(time, data).
4. Fix the monitoring error of delay differential equations with brainpy.integrators.IntegratorRunner.

Bug Fixes

1. Fix the bug on One2One connection.
2. Fix the bug in eprop example.
3. Fix ij2csr transformation error.
4. Fix test bugs

What's Changed

• fix eprop example error by @chaoming0625 in #305
• minor updates on API and DOC by @chaoming0625 in #306
• Add new optimizers by @chaoming0625 in #307
• add documentation of for random number generation by @chaoming0625 in #308
• consolidate the concept of OO transformation by @chaoming0625 in #309
• Upgrade documetations by @chaoming0625 in #310
• Ready for publish by @chaoming0625 in #311

Full Changelog: V2.2.4.0...V2.3.0