Merge pull request #142 from austin-hilberg/master

dimitri-yatsenko · web-flow · commit ca427b9fd732 · 2018-12-19T13:22:04.000-06:00
Documentation content additions and revisions for M2
diff --git a/docs-parts/admin/5-blob-config_lang1.rst b/docs-parts/admin/5-blob-config_lang1.rst
@@ -0,0 +1,4 @@
+.. note::
+
+  External storage is not yet implemented in MATLAB.
+  The feature will be added in an upcoming release: https://github.com/datajoint/datajoint-matlab/issues/143
diff --git a/docs-parts/admin/5-blob-config_lang2.rst b/docs-parts/admin/5-blob-config_lang2.rst
@@ -0,0 +1 @@
+Use ``dj.set`` for configuration.
diff --git a/docs-parts/admin/5-blob-config_lang3.rst b/docs-parts/admin/5-blob-config_lang3.rst
@@ -0,0 +1,4 @@
+.. note::
+
+  The cache folder is not yet implemented in MATLAB.
+  The feature will be added in an upcoming release: https://github.com/datajoint/datajoint-matlab/issues/143
diff --git a/docs-parts/admin/5-blob-config_lang4.rst b/docs-parts/admin/5-blob-config_lang4.rst
@@ -0,0 +1,4 @@
+.. note::
+
+  External storage is not yet implemented in MATLAB.
+  The feature will be added in an upcoming release: https://github.com/datajoint/datajoint-matlab/issues/143
diff --git a/docs-parts/admin/5-blob-config_lang5.rst b/docs-parts/admin/5-blob-config_lang5.rst
@@ -0,0 +1,4 @@
+.. note::
+
+  External storage is not yet implemented in MATLAB.
+  The feature will be added in an upcoming release: https://github.com/datajoint/datajoint-matlab/issues/143
diff --git a/docs-parts/computation/01-autopopulate_lang3.rst b/docs-parts/computation/01-autopopulate_lang3.rst
@@ -0,0 +1,3 @@
+Behavior of the ``populate`` method depends on the number of output arguments requested in the function call.
+When no output arguments are requested, errors will halt population.
+With two output arguments (``failedKeys`` and ``errors``), ``populate`` will catch any encountered errors and return them along with the offending keys.
diff --git a/docs-parts/computation/01-autopopulate_lang4.rst b/docs-parts/computation/01-autopopulate_lang4.rst
@@ -0,0 +1,28 @@
+The function ``parpopulate`` works identically to ``populate`` except that it uses a job reservation mechanism to allow multiple processes to populate the same table in parallel without collision.
+When running ``parpopulate`` for the first time, DataJoint will create a job reservation table and its class ``<package>.Jobs`` with the following declaration:
+
+.. code-block:: matlab
+
+  {%
+    # the job reservation table
+    table_name      : varchar(255)          # className of the table
+    key_hash        : char(32)              # key hash
+    ---
+    status            : enum('reserved','error','ignore')# if tuple is missing, the job is available
+    key=null          : blob                  # structure containing the key
+    error_message=""  : varchar(1023)         # error message returned if failed
+    error_stack=null  : blob                  # error stack if failed
+    host=""           : varchar(255)          # system hostname
+    pid=0             : int unsigned          # system process id
+    timestamp=CURRENT_TIMESTAMP : timestamp    # automatic timestamp
+  %}
+
+A job is considered to be available when ``<package>.Jobs`` contains no matching entry.
+
+For each ``make`` call, ``parpopulate`` sets the job status to ``reserved``.
+When the job is completed, the record is removed.
+If the job results in error, the job record is left in place with the status set to ``error`` and the error message and error stacks saved.
+Consequently, jobs that ended in error during the last execution will not be attempted again until you delete the corresponding entities from ``<package>.Jobs``.
+
+The primary key of the jobs table comprises the name of the class and a 32-character hash of the job's primary key.
+However, the key is saved in a separate field for error debugging purposes.
diff --git a/docs-parts/computation/02-keysource_lang1.rst b/docs-parts/computation/02-keysource_lang1.rst
@@ -0,0 +1 @@
+A custom key source can be configured by setting the ``keySource`` property within a table's ``classdef`` block, using MATLAB's `dependent properties <https://www.mathworks.com/help/matlab/matlab_oop/access-methods-for-dependent-properties.html>`_ syntax.
diff --git a/docs-parts/computation/02-keysource_lang2.rst b/docs-parts/computation/02-keysource_lang2.rst
@@ -0,0 +1,17 @@
+.. code-block:: matlab
+
+  %{
+  -> Recording
+  ---
+  sample_rate : float
+  eeg_data : longblob
+  %}
+  classdef EEG < dj.Imported
+
+    methods
+      function q = get.keySource(self)
+        q = ephys.Recording & 'recording_type = "EEG"'
+      end
+    end
+
+  end
diff --git a/docs-parts/computation/04-master-part_lang1.rst b/docs-parts/computation/04-master-part_lang1.rst
@@ -26,7 +26,7 @@ The part table must declare the property ``master`` containing an object of the
 
    %{
    # Region of interest resulting from segmentation
-   -> Segmentation
+   -> test.Segmentation
    roi  : smallint   # roi number
    ---
    roi_pixels  : longblob   #  indices of pixels
diff --git a/docs-parts/computation/04-master-part_lang3.rst b/docs-parts/computation/04-master-part_lang3.rst
@@ -1,4 +1,51 @@
 
+``+test/ArrayResponse.m``
+
+.. code-block:: matlab
+
+  %{
+  -> Probe
+  array: int
+  %}
+  classdef ArrayResponse < dj.Computed
+      methods(Access=protected)
+          function make(self, key)
+              self.insert(key)
+              make(test.ArrayResponseElectrodeResponse, key)
+          end
+      end
+  end
+
+``+test/ArrayResponseElectrodeResponse.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.ArrayResponse
+  electrode : int % electrode number on the probe
+  %}
+  classdef ArrayResponseElectrodeResponse < dj.Part
+      methods(SetAccess=protected)
+          function make(self, key)
+              self.insert(key)
+          end
+      end
+  end
+
+``+test/ArrayResponseChannelResponse.m``
+
 .. code-block:: matlab
 
-    
+  %{
+  -> test.ArrayResponseElectrodeResponse
+  channel: int
+  ---
+  response: longblob  % response of a channel
+  %}
+  classdef ArrayResponseChannelResponse < dj.Part
+      methods(SetAccess=protected)
+          function make(self, key)
+              self.insert(key)
+          end
+      end
+  end
diff --git a/docs-parts/concepts/04-Integrity_lang1.rst b/docs-parts/concepts/04-Integrity_lang1.rst
@@ -0,0 +1,25 @@
+``+test/Mouse.m``
+
+.. code-block:: matlab
+
+  %{
+  mouse_name : varchar(64)
+  ---
+  mouse_dob : datetime
+  %}
+
+  classdef Mouse < dj.Manual
+  end
+
+``+test/MouseDeath.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.Mouse
+  ---
+  death_date : datetime
+  %}
+
+  classdef MouseDeath < dj.Manual
+  end
diff --git a/docs-parts/concepts/04-Integrity_lang2.rst b/docs-parts/concepts/04-Integrity_lang2.rst
@@ -0,0 +1,28 @@
+``+test/EEGRecording.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.Session
+  eeg_recording_id : int
+  ---
+  eeg_system : varchar(64)
+  num_channels : int
+  %}
+
+  classdef EEGRecording < dj.Manual
+  end
+
+``+test/ChannelData.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.EEGRecording
+  channel_idx : int
+  ---
+  channel_data : longblob
+  %}
+
+  classdef ChannelData < dj.Imported
+  end
diff --git a/docs-parts/concepts/04-Integrity_lang3.rst b/docs-parts/concepts/04-Integrity_lang3.rst
@@ -0,0 +1,37 @@
+``+test/Mouse.m``
+
+.. code-block:: matlab
+
+  %{
+  mouse_name : varchar(64)
+  ---
+  mouse_dob : datetime
+  %}
+
+  classdef Mouse < dj.Manual
+  end
+
+``+test/SubjectGroup.m``
+
+.. code-block:: matlab
+
+  %{
+  group_number : int
+  ---
+  group_name : varchar(64)
+  %}
+
+  classdef SubjectGroup < dj.Manual
+  end
+
+``+test/SubjectGroupGroupMember.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.SubjectGroup
+  -> test.Mouse
+  %}
+
+  classdef SubjectGroupGroupMember < dj.Part
+  end
diff --git a/docs-parts/concepts/04-Integrity_lang4.rst b/docs-parts/concepts/04-Integrity_lang4.rst
@@ -0,0 +1,34 @@
+``+test/RecordingModality.m``
+
+.. code-block:: matlab
+
+  %{
+  modality : varchar(64)
+  %}
+
+  classdef RecordingModality < dj.Lookup
+  end
+
+``+test/MultimodalSession.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.Session
+  modes : int
+  %}
+
+  classdef MultimodalSession < dj.Manual
+  end
+
+``+test/MultimodalSessionSessionMode.m``
+
+.. code-block:: matlab
+
+  %{
+  -> test.MultimodalSession
+  -> test.RecordingModality
+  %}
+
+  classdef MultimodalSessionSessionMode < dj.Part
+  end
diff --git a/docs-parts/definition/03-Table-Definition_lang2.rst b/docs-parts/definition/03-Table-Definition_lang2.rst
@@ -1,2 +1,2 @@
-
+Users do not need to do anything special to have the table created in the database.
 The table is created upon the first attempt to use the class for manipulating its data (e.g. inserting or fetching entities).
diff --git a/docs-parts/definition/14-Drop_lang2.rst b/docs-parts/definition/14-Drop_lang2.rst
@@ -0,0 +1,4 @@
+.. note::
+
+  This rule is currently not enforced in MATLAB, but calling ``drop`` directly on a part table will produce an error in the future.
+  See `issue #125 <https://github.com/datajoint/datajoint-matlab/issues/125>`_ on ``datajoint-matlab`` for more information.
diff --git a/docs-parts/existing/1-Virtual-Modules_lang1.rst b/docs-parts/existing/1-Virtual-Modules_lang1.rst
@@ -0,0 +1,5 @@
+A ``TableAccessor`` object is created as a property of a schema during each schema's creation.
+This property is named ``schema.v``, for *virtual class generator*.
+The ``TableAccessor`` ``v`` itself has properties that refer to the tables of the schema.
+For example, one can access the ``Session`` table using ``schema.v.Session`` with no need for any ``Session`` class to exist in MATLAB.
+Tab completion of table names is possible because the table names are added as dynamic properties of ``TableAccessor``.
diff --git a/docs-parts/existing/empty.txt b/docs-parts/existing/empty.txt
diff --git a/docs-parts/manipulation/1-Insert_lang1.rst b/docs-parts/manipulation/1-Insert_lang1.rst
@@ -21,3 +21,6 @@ Quick entry of multiple entities takes advantage of MATLAB's cell array notation
     })
 
 In this case, the values must match the order of the attributes in the table.
+
+The optional parameter ``command`` can be either ``'IGNORE'`` or ``'REPLACE'``.
+Duplicates, unmatched attributes, or missing required attributes will cause insert errors, unless ``command`` is specified.
diff --git a/docs-parts/manipulation/1-Insert_lang2.rst b/docs-parts/manipulation/1-Insert_lang2.rst
@@ -0,0 +1,9 @@
+
+.. code-block:: matlab
+
+  % Server-side inserts are faster...
+  phase_two.Protocol.insert(phase_one.Protocol)
+
+  % ...than fetching before inserting
+  protocols = phase_one.Protocol.fetch();
+  phase_two.Protocol.insert(protocols)
diff --git a/docs-parts/manipulation/2-Delete_lang3.rst b/docs-parts/manipulation/2-Delete_lang3.rst
@@ -0,0 +1,4 @@
+.. note::
+
+  This rule is currently not enforced in MATLAB, but calling ``del`` directly on a part table will produce an error in the future.
+  See `issue #193 <https://github.com/datajoint/datajoint-matlab/issues/193>`_ on ``datajoint-matlab`` for more information.
diff --git a/docs-parts/queries/06-Restriction_lang1.rst b/docs-parts/queries/06-Restriction_lang1.rst
@@ -1,8 +1,7 @@
 
 * another table
-* a query expression
 * a mapping, or ``struct``
 * an expression in a character string
 * a collection of conditions as a ``struct`` or cell array
 * a Boolean expression (``true`` or ``false``)
-
+* a query expression
diff --git a/docs-parts/queries/06-Restriction_lang2.rst b/docs-parts/queries/06-Restriction_lang2.rst
@@ -1,5 +1,4 @@
 
 .. code-block:: matlab
 
-    Session & struct('session_dat', '2018-01-01')
-
+    ephys.Session & struct('session_dat', '2018-01-01')
diff --git a/docs-parts/queries/06-Restriction_lang3.rst b/docs-parts/queries/06-Restriction_lang3.rst
@@ -1,17 +1,8 @@
 
-.. warning::
-  This section documents future intended behavior in MATLAB, which is contrary to current behavior.
-  DataJoint for MATLAB has an open `issue <https://github.com/datajoint/datajoint-matlab/issues/128>`_ tracking this change.
-
-A collection can be a cell array or structure array.
-Cell arrays can contain collections of arbitrary restriction conditions.
-Structure arrays are limited to collections of mappings, each having the same attributes.
-
 .. code-block:: matlab
 
-    % a cell aray:
-    cond_cell = {'first_name = "Aaron"', 'last_name = "Aaronson"'}
+  % All the sessions performed by Alice
+  ephys.Session & 'user = "Alice"'
 
-    % a structure array:
-    cond_struct = struct('first_name', 'Aaron', 'last_name', 'Paul')
-    cond_struct(2) = struct('first_name', 'Rosie', 'last_name', 'Aaronson')
+  % All the experiments at least one minute long
+  ephys.Experiment & 'duration >= 60'
diff --git a/docs-parts/queries/06-Restriction_lang4.rst b/docs-parts/queries/06-Restriction_lang4.rst
@@ -1,11 +1,17 @@
 
-.. code-block:: matlab
+.. warning::
+  This section documents future intended behavior in MATLAB, which is contrary to current behavior.
+  DataJoint for MATLAB has an open `issue <https://github.com/datajoint/datajoint-matlab/issues/128>`_ tracking this change.
 
-    university.Student() & {'first_name = "Aaron"', 'last_name = "Aaronson"'}
+A collection can be a cell array or structure array.
+Cell arrays can contain collections of arbitrary restriction conditions.
+Structure arrays are limited to collections of mappings, each having the same attributes.
 
-.. figure:: ../_static/img/matlab_collection.png
-    :align: center
-    :alt: restriction by collection
+.. code-block:: matlab
 
-    Restriction by a collection, returning any entities matching any condition in the collection.
+    % a cell aray:
+    cond_cell = {'first_name = "Aaron"', 'last_name = "Aaronson"'}
 
+    % a structure array:
+    cond_struct = struct('first_name', 'Aaron', 'last_name', 'Paul')
+    cond_struct(2) = struct('first_name', 'Rosie', 'last_name', 'Aaronson')
diff --git a/docs-parts/queries/06-Restriction_lang5.rst b/docs-parts/queries/06-Restriction_lang5.rst
@@ -1,4 +1,11 @@
 
-``A & true`` and ``A - false`` are equivalent to ``A``.
-``A & false`` and ``A - true`` are empty.
+.. code-block:: matlab
+
+    university.Student() & {'first_name = "Aaron"', 'last_name = "Aaronson"'}
+
+.. figure:: ../_static/img/matlab_collection.png
+    :align: center
+    :alt: restriction by collection
+
+    Restriction by a collection, returning any entities matching any condition in the collection.
 
diff --git a/docs-parts/queries/06-Restriction_lang6.rst b/docs-parts/queries/06-Restriction_lang6.rst
@@ -0,0 +1,4 @@
+
+``A & true`` and ``A - false`` are equivalent to ``A``.
+
+``A & false`` and ``A - true`` are empty.
diff --git a/docs-parts/queries/06-Restriction_lang7.rst b/docs-parts/queries/06-Restriction_lang7.rst
diff --git a/docs-parts/queries/06-Restriction_lang8.rst b/docs-parts/queries/06-Restriction_lang8.rst
@@ -0,0 +1,4 @@
+.. code-block:: matlab
+
+  query = ephys.Session & 'user = "Alice"'
+  ephys.Experiment & query
diff --git a/docs-parts/queries/09-Aggr_lang1.rst b/docs-parts/queries/09-Aggr_lang1.rst
@@ -2,7 +2,6 @@
 .. code-block:: matlab
 
   % Number of students in each course section
-  university.Section.aggr(university.Enroll, n: count())
+  university.Section.aggr(university.Enroll, 'count(*)->n')
   % Average grade in each course
-  university.Course.aggr(university.Grade * university.LetterGrade, avg_grade: avg(points))
-
+  university.Course.aggr(university.Grade * university.LetterGrade, 'avg(points)->avg_grade')
diff --git a/docs-parts/queries/11-Universal-Sets_lang1.rst b/docs-parts/queries/11-Universal-Sets_lang1.rst

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	+Behavior of the ``populate`` method depends on the number of output arguments requested in the function call.
	`2`	`+When no output arguments are requested, errors will halt population.`
	`3`	+With two output arguments (``failedKeys`` and ``errors``), ``populate`` will catch any encountered errors and return them along with the offending keys.
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1 @@`
	`1`	+A custom key source can be configured by setting the ``keySource`` property within a table's ``classdef`` block, using MATLAB's `dependent properties <https://www.mathworks.com/help/matlab/matlab_oop/access-methods-for-dependent-properties.html>`_ syntax.