You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/faq.md
+20-4Lines changed: 20 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,21 +16,37 @@
16
16
17
17
**What do I do with the data SharpEyes saves out?**
18
18
19
-
It depends on how far through the workflow you go. The [Pupil Finding](workflow/pupil%20finding.md) tab saves pupil positions and timestamps as numpy arrays, which can be loaded into the [Calibration](workflow/calibration.md) tab or passed to the `SharpEyesCalibrator` class in the [Eyetracking](https://github.com/gallantlab/Eyetracking) Python repo for external calibration. That can then be brought back to SharpEyes.
19
+
It depends on how far through the workflow you go.
20
+
The [Pupil Finding](workflow/pupil%20finding.md) tab saves pupil positions and timestamps as numpy arrays,
21
+
which can be loaded into the [Calibration](workflow/calibration.md) tab or passed to the `SharpEyesCalibrator` class
22
+
in the [Eyetracking](https://github.com/gallantlab/Eyetracking) Python repo for external calibration. That can then be brought back to SharpEyes.
20
23
21
-
The [Stimulus & Gaze](workflow/stimulus%20and%20gaze.md) tab can load gaze positions in numpy, CSV, or Eyelink formats for viewing and editing. The [Recentering](workflow/recentering.md) tab exports retinotopic frames as PNG files or a numpy array. The [Motion-Energy](workflow/motion%20energy.md) tab saves a numpy array of features, a plain-text info file recording all parameters, and a CSV describing each filter in the pyramid.
24
+
The [Stimulus & Gaze](workflow/stimulus%20and%20gaze.md) tab can load gaze positions in numpy, CSV, or Eyelink formats for viewing and editing.
25
+
The [Recentering](workflow/recentering.md) tab exports retinotopic frames as PNG files or a numpy array.
26
+
The [Motion-Energy](workflow/motion%20energy.md) tab saves a numpy array of features,
27
+
a plain-text info file recording all parameters, and a CSV describing each filter in the pyramid.
22
28
23
29
**There's no option to read .edf files from Eyelink**
24
30
25
-
You need to provide your own copy of the edfapi.dll/so library from the SR Research Eyelink SDK. SharpEyes will find it automatically if it is on your system path, or you can specify its location explicitly in General Settings. There is no EDF support for macOS.
31
+
You need to provide your own copy of the edfapi.dll/so library from the SR Research Eyelink SDK.
32
+
SharpEyes will find it automatically if it is on your system path,
33
+
or you can specify its location explicitly in General Settings. There is no EDF support for macOS from SR Research.
34
+
35
+
**The videos don't play in real time**
36
+
37
+
Because SharpEyes is running image processing under the hood, either for pupil finding or recentering,
38
+
SharpEyes deals with the raw pixel values on each frame of the video as a bitmap array.
39
+
We have to manually draw this to the canvas, which takes more overhead than a normal video player.
26
40
27
41
**What do I need to build SharpEyes from source?**
28
42
29
43
SharpEyes was built with Visual Studio and JetBrains Rider.
30
44
31
45
**What about GPU support?**
32
46
33
-
Motion-energy feature extraction can run on an NVIDIA GPU via CUDA or on Apple Silicon via MPS, if PyTorch is installed and the hardware is available. The available backends are listed in Python Settings, where you can also set your preferred order. All other processing runs on CPU.
47
+
Motion-energy feature extraction can run on an NVIDIA GPU via CUDA or on Apple Silicon via MPS,
48
+
if PyTorch is installed and the hardware is available. The available backends are listed in Python Settings,
49
+
where you can also set your preferred order. All other processing runs on CPU.
Copy file name to clipboardExpand all lines: docs/index.md
+9-4Lines changed: 9 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,12 +4,17 @@ SharpEyes is a tool for eyetracking, gaze correction, and motion-energy feature
4
4
5
5
SharpEyes offers a UI for finding pupils in eyetracking videos, and the ability to edit the pupil locations manually.
6
6
7
-
It then lets you overlay the gaze on the stimulus to look at eyetraces, and then also the option to manually correct that again. It can also filter the gaze traces to smooth things out
7
+
It then lets you overlay the gaze on the stimulus to look at eyetraces, and then also the option to manually correct that again.
8
+
It can also filter the gaze traces to smooth things out.
8
9
9
10
Given traces and stimulus video, you can then recenter the video to be retinotopic, and generate motion-energy features from stimuli.
10
11
11
-
The motion-energy features are actually processed by [PyMoten](https://github.com/gallantlab/pymoten), but the user doesn't have to interact all with it. Sharpeyes will handle everything under the hood, including setup.
12
+
The motion-energy features are actually processed by [PyMoten](https://github.com/gallantlab/pymoten), but the user doesn't have to interact with it.
13
+
Sharpeyes will handle everything under the hood, including setup.
12
14
13
-
<imgsrc="assets/stimulus and gaze.png"width="800" />
15
+
<videowidth="800"controls>
16
+
<source src="assets/live%20motion-energy.mp4" type = "video/mp4">
17
+
</video>
14
18
15
-
See [Getting Started](getting%20started.md) for installation and Python setup. See the [Workflow](workflow/index.md) section for documentation on each tab. If you use SharpEyes in your work, see [Citing SharpEyes](cite.md).
19
+
See [Getting Started](getting%20started.md) for installation and Python setup. See the [Workflow](workflow/index.md) section for documentation on each tab.
20
+
If you use SharpEyes in your work, see [Citing SharpEyes](cite.md).
The Motion-Energy tab computes motion-energy features from a video using [PyMoten](https://github.com/gallantlab/pymoten). It can operate on recentered video sent from the [Recentering](recentering.md) tab, or on a raw video loaded directly. If you want to compute motion-energy for retinotopic videos, then they _have_ to be sent over from the [Recentering](./recentering.md) tab. The video preview overlays the motion-energy pyramid on the frames so you can visually verify the filter parameters before committing to a full computation.
5
+
The Motion-Energy tab computes motion-energy features from a video using [PyMoten](https://github.com/gallantlab/pymoten).
6
+
It can operate on recentered video sent from the [Recentering](recentering.md) tab, or on a raw video loaded directly.
7
+
If you want to compute motion-energy for retinotopic videos, then they _have_ to be sent over from the [Recentering](./recentering.md) tab.
8
+
The video preview overlays the motion-energy pyramid on the frames so you can visually verify the filter parameters before committing to a full computation.
9
+
If you have computed motion-energy features, you can use the "Load Features" button to select the text file that SharpEyes generated,
10
+
and read in the previously computed features and thei parameters along with the video and eyetracking information.
6
11
7
12
## Motion-energy frame parameters
8
13
@@ -24,16 +29,16 @@ Click **Restore defaults** to reset these parameters.
24
29
25
30
These control the filter bank that PyMoten uses to decompose the video into motion-energy features.
26
31
27
-
| Control | Description |
28
-
|---|---|
29
-
| Video FPS | Frame rate of the input video, used to convert temporal frequencies from cycles/second to cycles/frame. This is read from the video but can be overridden|
30
-
| Spatial frequencies | The spatial frequencies (cycles/image) to include in the pyramid. Select multiple; use + and − to add or remove values. |
31
-
| Temporal frequencies | The temporal frequencies (cycles/second) to include in the pyramid. Select multiple; use + and − to add or remove values. |
32
-
| Directions | The motion directions (degrees) to include in the pyramid. Select multiple; use + and − to add or remove values. |
33
-
| Show motion-energy pyramid | When enabled, the pyramid filters are overlaid on the video preview as circles and arrows so you can verify coverage. |
34
-
| Show dynamic responses | When enabled, the pyramid overlay is colored by the computed filter responses for the current frame. Only available after a feature extraction run completes. A warning is shown if filter parameters have changed since the last extraction, in which case the responses are stale and should be recomputed. |
35
-
| Response scaling | Controls how filter responses are normalized before being mapped to opacity. Options: **Per-filter** (each filter is normalized against its own maximum), **Global** (all filters share the same maximum), **Percentile** (uses the 95th percentile as the ceiling), **Logarithmic** (log-scaled). |
36
-
| Max opacity | Maximum opacity applied to the dynamic response overlay elements at full response strength. |
| Video FPS | Frame rate of the input video, used to convert temporal frequencies from cycles/second to cycles/frame. This is read from the video but can be overridden|
35
+
| Spatial frequencies | The spatial frequencies (cycles/image) to include in the pyramid. Select multiple; use + and − to add or remove values. |
36
+
| Temporal frequencies | The temporal frequencies (cycles/second) to include in the pyramid. Select multiple; use + and − to add or remove values. |
37
+
| Directions | The motion directions (degrees) to include in the pyramid. Select multiple; use + and − to add or remove values. |
38
+
| Show motion-energy pyramid | When enabled, the pyramid filters are overlaid on the video preview as circles and arrows so you can verify coverage. |
39
+
| Show dynamic responses | When enabled, the pyramid overlay is colored by the computed filter responses for the current frame. Only available after a feature extraction run completes or if saved features are loaded. If filter parameters have changed since the last extraction, the responses are stale and should be recomputed. |
40
+
| Response scaling | Controls how filter responses are normalized before being mapped to opacity. Options: **Per-filter** (each filter is normalized against its own maximum), **Global** (all filters share the same maximum), **Percentile** (uses the 95th percentile as the ceiling), **Logarithmic** (log-scaled). |
41
+
| Max opacity | Maximum opacity applied to the dynamic response overlay elements at full response strength. |
37
42
38
43
Click **Compute pyramid** to compute and display the pyramid for the current frame. Click **Restore defaults** to reset the pyramid parameters to their defaults.
0 commit comments