adding docs and fixing bugs

Author: carsen-stringer

README.md

@@ -16,13 +16,15 @@
 Rastermap is a discovry algorithm for neural data. The algorithm was written by 
 Carsen Stringer and Marius Pachitariu. To learn about Rastermap, read the [paper]() or watch the [talk](). For support,  please open an [issue](https://github.com/MouseLand/rastermap/issues). Please see install instructions [below](README.md/#Installation).
 
-Rastermap runs in python 3.8+ and has a graphical user interface (GUI) for running it easily. Rastermap can also be run in a jupyter notebook locally or on google colab:
+Rastermap runs in python 3.8+ and has a graphical user interface (GUI) for running it easily. Rastermap can also be run in a jupyter notebook locally or on google colab, see these demos:
 * [rastermap_largescale.ipynb](notebooks/rastermap_largescale.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_largescale.ipynb) shows how to use it with large-scale data from mouse cortex (> 200 neurons) 
 * [rastermap_singleneurons.ipynb](notebooks/rastermap_singleneurons.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_singleneurons.ipynb) shows how to use it with small to medium sized data (< 200 neurons), in this case recorded from rat hippocampus 
 * [rastermap_zebrafish.ipynb](notebooks/rastermap_zebrafish.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_zebrafish.ipynb) shows how to use it with large-scale data from zebrafish 
 * [rastermap_widefield.ipynb](notebooks/rastermap_widefield.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_widefield.ipynb) shows how to use it with widefield imaging data, or other types of datasets that are too large to fit into memory 
 * [tutorial.ipynb](notebooks/tutorial.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/tutorial.ipynb) is a guided tutorial for integrating rastermap and facemap to visualize behavioral representations 
 
+**all demo data available [here](https://osf.io/xn4cm/)**
+
 Here is what the output looks like for a segment of a mesoscope recording in a mouse during spontaneous activity (3.2Hz sampling rate), compared to random neural sorting:
 
 <img src="https://www.suite2p.org/static/images/rastermap_spont.png" width="800" alt="random sorting and rastermap sorting of spontaneous activity"/>
@@ -86,7 +88,7 @@ The quickest way to start is to open the GUI from a command line terminal. You m
 python -m rastermap
 ~~~
 
-To start using the GUI, save your data into an npy file that is just a matrix that is neurons x timepoints. Then "File > Load data matrix" and choose this file. Next click "Run > Run rastermap" and click run. See the parameters section to learn about the parameters.
+To start using the GUI, save your data into an npy file that is just a matrix that is neurons x timepoints. Then "File > Load data matrix" and choose this file (or drag and drop your file). Next click "Run > Run rastermap" and click run. See the parameters section to learn about the parameters.
 
 The GUI will start with a highlighted region that you can drag to visualize the average activity of neurons in a given part of the plot. To draw more regions, you right-click to start a region, then right-click to end it. The neurons' activity traces then show up on the botton of the GUI, and if the neuron positions are loaded, you will see them colored by the region color. You can delete a region by holding CTRL and clicking on it. You can save the ROIs you've drawn with the "Save > Save processed data" button. They will save along with the embedding so you can reload the file with the "Load processed data" option.
 
@@ -230,8 +232,8 @@ Here is the list of all variables assigned from `fit`:
 
 * **embedding** : array, shape (n_samples, 1)
             embedding of each neuron / voxel
-* **isort** : sorting along first dimension of input matrix
-            use this to get neuron / voxel sorting
+* **isort** : array, shape (n_samples,)
+    sorting along first dimension of input matrix - use this to get neuron / voxel sorting
 * **igood** : array, shape (n_samples, 1)
     neurons/voxels which had non-zero activity and were used for sorting
 * **Usv** : array, shape (n_samples, n_PCs) 
@@ -253,6 +255,20 @@ Here is the list of all variables assigned from `fit`:
 * **X_embedding** : array, shape (n_samples//bin_size, n_features)
     normalized data binned across samples (if compute_X_embedding is True)
 
+The output from the GUI and the command line is a file that ends with `_embedding.npy`. This file contains:
+* **filename**: str,
+    path to file that rastermap was run on
+* **save_path**: str,
+    folder with filename
+* **embedding** : array, shape (n_samples, 1)
+            embedding of each neuron / voxel
+* **isort** : array, shape (n_samples,)
+    sorting along first dimension of input matrix - use this to get neuron / voxel sorting
+* **user_clusters**: list, 
+    list of user drawn clusters in GUI
+* **ops**: dict,
+    dictionary of options used to run rastermap
+
 
 # License
 

rastermap/__main__.py

@@ -62,14 +62,12 @@
         model.fit(data=X, Usv=Usv, Vsv=Vsv)
 
         proc = {
-            "embedding": model.embedding,
+            "filename": args.S,
+            "save_path": os.path.split(args.S)[0],
             "isort": model.isort,
-            "Usv": model.Usv,
-            "Vsv": model.Vsv,
-            "sv": model.sv,
+            "embedding": model.embedding,
+            "user_clusters": None,
             "ops": ops,
-            "filename": args.S,
-            "train_time": train_time
         }
         basename, fname = os.path.split(args.S)
         fname = os.path.splitext(fname)[0]

rastermap/gui/gui.py

@@ -229,7 +229,6 @@ def sat_changed(self):
         self.show()
 
     def reset(self):
-        self.run_embedding_button.setEnabled(False)
         self.p1.clear()
         self.p2.clear()
         self.p3.clear()
@@ -461,7 +460,7 @@ def smooth_activity(self):
             self.sp_smoothed = np.maximum(-2, np.minimum(5, self.sp_smoothed)) + 2
             self.sp_smoothed /= 7
         else:
-            self.sp_smoothed = self.sp.copy()
+            self.sp_smoothed = self.sp[self.sorting].copy()
         self.nsmooth = self.sp_smoothed.shape[0]
         yr0 = min(4, self.nsmooth // 4)
         ym = self.nsmooth // 2
@@ -475,6 +474,8 @@ def smooth_activity(self):
         self.get_behav_corr() if self.behav_data else None
         if self.neuron_pos is not None or self.behav_data is not None:
             self.update_scatter(init=True)
+        elif self.neuron_pos is None and self.scatter_comboBox.currentIndex()==0:
+            self.p5.clear()
         self.p2.show()
         self.p3.show()
 
@@ -558,7 +559,6 @@ def update_scatter(self, init=False, roi_id=None):
             self.p5.invertY(False)
         request = self.scatter_comboBox.currentIndex()
         if request > 0:
-
             self.plot_behav_corr(roi_id=roi_id, init=init)
         else:
             self.plot_neuron_pos(roi_id=roi_id, init=init)

rastermap/gui/io.py

@@ -12,7 +12,6 @@
 from ..io import _load_iscell, _load_stat, load_activity
 
 def _load_activity_gui(parent, X, Usv, Vsv, xy):
-        
     igood = None
     if X is not None:
         parent.update_status_bar(
@@ -45,8 +44,7 @@ def _load_activity_gui(parent, X, Usv, Vsv, xy):
     else:
         raise ValueError("file missing keys / data")
 
-    if xy is not None:
-        parent.neuron_pos = xy if igood is None else xy[igood]
+    parent.neuron_pos = xy if igood is None else xy[igood]
 
     parent.n_samples = (parent.sp.shape[0] if parent.sp is not None 
                         else parent.Usv.shape[0])
@@ -67,10 +65,8 @@ def load_mat(parent, name=None):
         name = QFileDialog.getOpenFileName(parent, "Open *.npy, *.npz, *.nwb or *.mat",
                                             filter="*.npy *.npz *.mat *.nwb")
         parent.fname = name[0]
-        parent.filebase = name[0]
     else:
         parent.fname = name
-        parent.filebase = name
 
     X, Usv, Vsv, xy = load_activity(parent.fname)
     _load_activity_gui(parent, X, Usv, Vsv, xy)
@@ -416,7 +412,7 @@ def save_proc(parent):  # Save embedding output
                     parent, "Choose save folder")
                 parent.save_path = folderName
             if parent.save_path:
-                filename = parent.fname.split("/")[-1]
+                filename = os.path.split(parent.fname)[-1]
                 filename, ext = os.path.splitext(filename)
                 savename = os.path.join(parent.save_path,
                                         ("%s_embedding.npy" % filename))
@@ -439,7 +435,6 @@ def save_proc(parent):  # Save embedding output
                     "save_path": parent.save_path,
                     "isort": parent.sorting,
                     "embedding": parent.embedding,
-                    "Usv": parent.U,
                     "user_clusters": user_clusters,
                     "ops": ops
                 }

rastermap/gui/run.py

@@ -82,7 +82,7 @@ def run_RMAP(self, parent):
         ops_path = os.path.join(os.getcwd(), "rmap_ops.npy")
         np.save(ops_path, self.ops)
         print("Running rastermap with command:")
-        cmd = f"-u -W ignore -m rastermap --ops {ops_path} --S {parent.filebase}"
+        cmd = f"-u -W ignore -m rastermap --ops {ops_path} --S {parent.fname}"
         if parent.file_iscell is not None:
             cmd += f"--iscell {parent.file_iscell}"
         print("python " + cmd)

rastermap/io.py

@@ -65,7 +65,7 @@ def _load_dict(dat, keys):
             raise ValueError("Vsv must have 2 dimensions")
 
     if xpos is not None and xy is None:
-        xy = np.stack((xpos, ypos), axis=1)
+        xy = np.stack((ypos, xpos), axis=1)
 
     if xy is not None:
         if xy.ndim != 2:
@@ -76,7 +76,7 @@ def _load_dict(dat, keys):
         if xy is not None:
             if X is not None and X.shape[0]!=xy.shape[0]:
                 xy = None
-            elif Usv.shape[0]!=xy.shape[0]:
+            elif Usv is not None and Usv.shape[0]!=xy.shape[0]:
                 xy = None
             if xy is None:
                 print("cannot use xy from file: x and y positions of neurons are not same size as activity")

rastermap/rastermap.py

@@ -241,7 +241,7 @@ def fit(self, data=None, Usv=None, Vsv=None, U_nodes=None, itrain=None,
 
         """
         t0 = time.time()
-
+        self.n_clusters = None if self.n_clusters==0 else self.n_clusters
 
         # normalize data
         igood = ~np.isnan(data[:,0]) if data is not None else ~np.isnan(Usv[:,0])