Skip to content

Commit 988ef66

Browse files
authored
Update user_documentation.rst - sections 7 to 10
1 parent 07ea610 commit 988ef66

File tree

1 file changed

+9
-11
lines changed

1 file changed

+9
-11
lines changed

doc/source/user_documentation.rst

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3191,10 +3191,7 @@ Currently, the following argument pairs are acceptable for ``job_type`` and ``co
31913191
31923192
7. Batch searches on the Rosenbrock function (some simple examples)
31933193
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3194-
The ``examples`` directory `on the NetPyNE github <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples/rosenbrock>`_ contains multiple methods of performing automatic parameter search of a
3195-
2 dimensional Rosenbrock function. These examples are used to quickly demonstrate some of the functionality of batch communications rather than the full process of running parameter searches on a detailed
3196-
NEURON simulation (see 7. Performing parameter optimization searches (CA3 example)) and therefore only contain the a `batch.py` file containing the script detailing the parameter space and search method, and a
3197-
`rosenbrock.py` file containing the function to explore, and the appropriate declarations and calls for batch automation and communication (rather than the traditional `cfg.py`, `netParams.py`, and `init.py` files).
3194+
The ``examples`` directory `on the NetPyNE github <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples/rosenbrock>`_ contains multiple methods of performing automatic parameter search of a 2-dimensional Rosenbrock function. These examples are used to quickly demonstrate some of the functionality of batch communications rather than the full process of running parameter searches on a detailed NEURON simulation (see 7. Performing parameter optimization searches (CA3 example)) and therefore only contain the a `batch.py` file containing the script detailing the parameter space and search method, and a `rosenbrock.py` file containing the function to explore, and the appropriate declarations and calls for batch automation and communication (rather than the traditional `cfg.py`, `netParams.py`, and `init.py` files).
31983195
31993196
1. `basic_rosenbrock <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples/rosenbrock/basic_rosenbrock>`_
32003197
@@ -3267,7 +3264,9 @@ By using ``xn.0`` and ``xn.1`` we can reference the 0th and 1st elements of the
32673264
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
32683265
The ``examples`` directory `on the NetPyNE github <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples>`_ shows both a ``grid`` based search as well as an ``optuna`` based optimization.
32693266
3270-
In the ``CA3`` example, we tune the ``PYR->BC`` ``NMDA`` and ``AMPA`` synaptic weights, as well as the ``BC->PYR`` ``GABA`` synaptic weight. Note the search space is defined
3267+
In the ``CA3`` example, we tune the ``PYR->BC`` ``NMDA`` and ``AMPA`` synaptic weights, as well as the ``BC->PYR`` ``GABA`` synaptic weight.
3268+
3269+
For the optuna-based parameter optimization, the search space upper and lower bounds are defined in ``optuna_search.py`` as:
32713270
32723271
.. code-block:: python
32733272
@@ -3277,7 +3276,7 @@ In the ``CA3`` example, we tune the ``PYR->BC`` ``NMDA`` and ``AMPA`` synaptic w
32773276
'gaba.BC->PYR' : [0.4e-3, 1.0e-3],
32783277
}
32793278
3280-
in both ``optuna_search.py``, defining the upper and lower bounds of the search space, while in ``grid_search.py`` the search space is defined
3279+
In contrast, for the grid-based parameter explorations, the ``3x3x3`` specific values to search over are defined in ``grid_search.py`` as:
32813280
32823281
.. code-block:: python
32833282
@@ -3287,9 +3286,7 @@ in both ``optuna_search.py``, defining the upper and lower bounds of the search
32873286
'gaba.BC->PYR' : numpy.linspace(0.4e-3, 1.0e-3, 3),
32883287
}
32893288
3290-
which defines ``3x3x3`` specific values to search over
3291-
3292-
Note that the ``metric`` specifies a specific ``string`` (``loss``) to report and optimize around. This value is generated and ``sent`` by the ``init.py`` simulation
3289+
Note that the ``metric`` sets a specific ``string`` (``loss``) to report and optimize around. This value is generated and sent by the ``init.py`` simulation:
32933290
32943291
.. code-block:: python
32953292
@@ -3311,17 +3308,18 @@ In a multi-objective optimization, the relevant ``PYR_loss``, ``BC_loss``, and `
33113308
9. Ray Checkpointing and Resuming Interrupted Searches
33123309
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
33133310
A new feature in this beta release is the checkpointing and saving of search progress via the ``ray`` backend. This data is saved in the ``checkpoint_path`` directory specified in the ``search`` function, (which defaults to a newly created ``checkpoint`` folder within the source directory, and the default behavior of ``search`` is to automatically attempt a restore if the batch job is interrupted.
3311+
33143312
Upon successful completion of the search, the default behavior is to delete these checkpoint files. If the user manually ends the search due to coding error and wishes to restart the search, the ``checkpoint_path`` should be deleted first.
33153313
33163314
10. Parameter Importance Evaluation Using fANOVA (unstable)
33173315
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3318-
A new feature in this beta release is the ability to evaluate parameter importance using a functional ANOVA inspired algorithm via the ``Optuna`` and ``scikit-learn`` libraries.
3316+
Another new feature in this beta release is the ability to evaluate parameter importance using a functional ANOVA inspired algorithm via the ``Optuna`` and ``scikit-learn`` libraries.
33193317
(See `the original Hutter paper <http://proceedings.mlr.press/v32/hutter14.pdf>`_ and its `citation <https://automl.github.io/fanova/cite.html>`_)
33203318
33213319
Currently, only unpaired single parameter importance to a single metric score is supported through the ``NetPyNE.batchtools.analysis`` ``Analyzer`` object, with an example of its usage
33223320
`here <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples/rosenbrock/fanova_rosenbrock>`_:
33233321
3324-
To run the example, generate an output ``grid.csv`` using ``batch.py``, then loading that ``grid.csv`` into the ``Analyzer`` object. Then, using ``run_analysis`` will generate, per parameter, a single score indicative of the estimated ``importance`` of the parameter: that is, the estimated effect on the total variance of the model within the given bounds.
3322+
To run the example, generate an output ``grid.csv`` using ``batch.py``, and then load that ``grid.csv`` into the ``Analyzer`` object. Finally, executing ``run_analysis`` will generate a single score per parameter indicative of the estimated ``importance`` of the parameter: that is, the estimated effect on the total variance of the model within the given bounds:
33253323
33263324
.. code-block:: python
33273325

0 commit comments

Comments
 (0)