Skip to content

Commit b921ee2

Browse files
paprpapr
authored
Merge pull request #9 from romanroibu/pupil-invisible-relay
Pupil Invisible LSL Relay
2 parents c6d78d2 + 6bfa343 commit b921ee2

13 files changed

Lines changed: 694 additions & 44 deletions

File tree

‎.gitignore‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Created by https://www.gitignore.io/api/python,pycharm,visualstudiocode
2+
# Edit at https://www.gitignore.io/?templates=python,pycharm,visualstudiocode
3+
4+
### PyCharm ###
5+
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm
6+
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
7+
8+
# User-specific stuff
9+
.idea/**/workspace.xml
10+
.idea/**/tasks.xml
11+
.idea/**/usage.statistics.xml
12+
.idea/**/dictionaries
13+
.idea/**/shelf
14+
15+
# Generated files
16+
.idea/**/contentModel.xml
17+
18+
# Sensitive or high-churn files
19+
.idea/**/dataSources/
20+
.idea/**/dataSources.ids
21+
.idea/**/dataSources.local.xml
22+
.idea/**/sqlDataSources.xml
23+
.idea/**/dynamic.xml
24+
.idea/**/uiDesigner.xml
25+
.idea/**/dbnavigator.xml
26+
27+
# Gradle
28+
.idea/**/gradle.xml
29+
.idea/**/libraries
30+
31+
# Gradle and Maven with auto-import
32+
# When using Gradle or Maven with auto-import, you should exclude module files,
33+
# since they will be recreated, and may cause churn. Uncomment if using
34+
# auto-import.
35+
# .idea/modules.xml
36+
# .idea/*.iml
37+
# .idea/modules
38+
# *.iml
39+
# *.ipr
40+
41+
# CMake
42+
cmake-build-*/
43+
44+
# Mongo Explorer plugin
45+
.idea/**/mongoSettings.xml
46+
47+
# File-based project format
48+
*.iws
49+
50+
# IntelliJ
51+
out/
52+
53+
# mpeltonen/sbt-idea plugin
54+
.idea_modules/
55+
56+
# JIRA plugin
57+
atlassian-ide-plugin.xml
58+
59+
# Cursive Clojure plugin
60+
.idea/replstate.xml
61+
62+
# Crashlytics plugin (for Android Studio and IntelliJ)
63+
com_crashlytics_export_strings.xml
64+
crashlytics.properties
65+
crashlytics-build.properties
66+
fabric.properties
67+
68+
# Editor-based Rest Client
69+
.idea/httpRequests
70+
71+
# Android studio 3.1+ serialized cache file
72+
.idea/caches/build_file_checksums.ser
73+
74+
### PyCharm Patch ###
75+
# Comment Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-215987721
76+
77+
# *.iml
78+
# modules.xml
79+
# .idea/misc.xml
80+
# *.ipr
81+
82+
# Sonarlint plugin
83+
.idea/**/sonarlint/
84+
85+
# SonarQube Plugin
86+
.idea/**/sonarIssues.xml
87+
88+
# Markdown Navigator plugin
89+
.idea/**/markdown-navigator.xml
90+
.idea/**/markdown-navigator/
91+
92+
### Python ###
93+
# Byte-compiled / optimized / DLL files
94+
__pycache__/
95+
*.py[cod]
96+
*$py.class
97+
98+
# C extensions
99+
*.so
100+
101+
# Distribution / packaging
102+
.Python
103+
build/
104+
develop-eggs/
105+
dist/
106+
downloads/
107+
eggs/
108+
.eggs/
109+
lib/
110+
lib64/
111+
parts/
112+
sdist/
113+
var/
114+
wheels/
115+
pip-wheel-metadata/
116+
share/python-wheels/
117+
*.egg-info/
118+
.installed.cfg
119+
*.egg
120+
MANIFEST
121+
122+
# PyInstaller
123+
# Usually these files are written by a python script from a template
124+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
125+
*.manifest
126+
*.spec
127+
128+
# Installer logs
129+
pip-log.txt
130+
pip-delete-this-directory.txt
131+
132+
# Unit test / coverage reports
133+
htmlcov/
134+
.tox/
135+
.nox/
136+
.coverage
137+
.coverage.*
138+
.cache
139+
nosetests.xml
140+
coverage.xml
141+
*.cover
142+
.hypothesis/
143+
.pytest_cache/
144+
145+
# Translations
146+
*.mo
147+
*.pot
148+
149+
# Scrapy stuff:
150+
.scrapy
151+
152+
# Sphinx documentation
153+
docs/_build/
154+
155+
# PyBuilder
156+
target/
157+
158+
# pyenv
159+
.python-version
160+
161+
# pipenv
162+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
163+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
164+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
165+
# install all needed dependencies.
166+
#Pipfile.lock
167+
168+
# celery beat schedule file
169+
celerybeat-schedule
170+
171+
# SageMath parsed files
172+
*.sage.py
173+
174+
# Spyder project settings
175+
.spyderproject
176+
.spyproject
177+
178+
# Rope project settings
179+
.ropeproject
180+
181+
# Mr Developer
182+
.mr.developer.cfg
183+
.project
184+
.pydevproject
185+
186+
# mkdocs documentation
187+
/site
188+
189+
# mypy
190+
.mypy_cache/
191+
.dmypy.json
192+
dmypy.json
193+
194+
# Pyre type checker
195+
.pyre/
196+
197+
### VisualStudioCode ###
198+
.vscode/*
199+
!.vscode/settings.json
200+
!.vscode/tasks.json
201+
!.vscode/launch.json
202+
!.vscode/extensions.json
203+
204+
### VisualStudioCode Patch ###
205+
# Ignore all local history of files
206+
.history
207+
208+
# End of https://www.gitignore.io/api/python,pycharm,visualstudiocode

‎README.md‎

Lines changed: 12 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,16 @@
1-
# Pupil LSL Relay Plugin
1+
# App-PupilLabs
22

3-
Plugin for _[Pupil Capture](https://github.com/pupil-labs/pupil/wiki/Pupil-Capture)_ that publishes realtime gaze data using the [lab streaming layer](https://github.com/sccn/labstreaminglayer) framework.
3+
This repository contains implementations of relays that publish realtime gaze data using the [lab streaming layer](https://github.com/sccn/labstreaminglayer) framework.
44

5-
## Installation
5+
- **`pupil_capture`** contains a [Pupil Capture][pupil-capture-app] plugin that works with the [Pupil Core headset][pupil-core-headset] and the [VR/AR add-ons][vr-ar-addons].
6+
More information on how to install and use the plugin is available [here][pupil-core-lsl-readme]
7+
- **`pupil_invisible_lsl_relay`** is a command line tool for publishing data from [Pupil Invisible][pupil-invisible-headset-and-app].
8+
More information on how to install and use the tool is available [here][pupil-invisible-lsl-readme].
69

7-
[user plugin directory](https://docs.pupil-labs.com/#plugin-guide)
810

9-
1. Install `pylsl`
10-
2. Copy or symlink `pylsl` with all its content to the _plugin directory_.
11-
3. Copy [`pupil_lsl_relay.py`](pupil_lsl_relay.py) to the _plugin directory_.
12-
13-
14-
## Usage
15-
16-
1. Start _Pupil Capture_.
17-
2. [Open the _Pupil LSL Relay_ plugin](https://docs.pupil-labs.com/#open-a-plugin).
18-
3. Now the LSL outlet is ready to provide data to other inlets in the network.
19-
20-
## LSL Outlet
21-
22-
The plugin opens a single outlet named `pupil_capture` that follows the [Gaze Meta Data](https://github.com/sccn/xdf/wiki/Gaze-Meta-Data) format.
23-
24-
See our [pupil-helpers](https://github.com/pupil-labs/pupil-helpers/tree/master/LabStreamingLayer) for examples on how to record and visualize the published data.
25-
26-
The published LSL data is simply a flattened version (see `extract_*()` functions in `pupil_lsl_relay.py`) of the original Pupil gaze data stream. The stream's channels will be filled with best effort, i.e. if there is a monocular gaze datum the values for the opposite eye will be set to `NaN`. The actual pairing of pupil data to binocular gaze data happens in [Capture](https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/calibration_routines/gaze_mappers.py#L95-L140) and is not a LSL specific behaviour. Therefore, it is possible to apply the same [flattening code](https://github.com/papr/App-PupilLabs/blob/master/pupil_lsl_relay.py#L226-L287) to offline calibrated gaze data and reproduce the stream published by the LSL outlet.
27-
28-
## Data Format
29-
30-
'confidence': Normalized (0-1) confidence.
31-
32-
'norm_pos_x', 'norm_pos_y': Normalized (0-1) coordinates on the screen.
33-
34-
'gaze_point_3d_x', 'gaze_point_3d_y', 'gaze_point_3d_z': World coordinates in mm
35-
36-
'eye_centerright_3d_x' ... (for right/left eyes, for x/y/z): Position of eye center in world coordinates in mm.
37-
38-
'gaze_normalright_x' (right/left, x/y/z): End point of vector from eye center (I think).
39-
40-
'diameterright_2d' (right/left): Pupil diameter in pixels
41-
42-
'diameterright_3d' (right/left): Pupil diameter in mm
43-
44-
## LSL Clock Synchronization
45-
46-
The `Pupil LSL Relay` plugin adjusts Capture's timebase to synchronize Capture's own clock with the `pylsl.local_clock()`. This allows the recording of native Capture timestamps and removes the necessity of manually synchronize timestamps after the effect.
47-
48-
**Warning**: The time synchronization will potentially break if other time alternating actors (e.g. the `Time Sync` plugin or `hmd-eyes`) are active.
11+
[pupil-capture-app]: https://github.com/pupil-labs/pupil/releases/latest
12+
[pupil-core-headset]: https://pupil-labs.com/products/core
13+
[pupil-invisible-headset-and-app]: https://pupil-labs.com/products/invisible/
14+
[pupil-core-lsl-readme]: https://github.com/labstreaminglayer/App-PupilLabs/blob/master/pupil_capture/README.md
15+
[pupil-invisible-lsl-readme]: https://github.com/labstreaminglayer/App-PupilLabs/blob/master/pupil_invisible_lsl_relay/README.md
16+
[vr-ar-addons]: https://pupil-labs.com/products/vr-ar/

‎pupil_capture/README.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Pupil Capture LSL Relay Plugin
2+
3+
Plugin for _[Pupil Capture](https://github.com/pupil-labs/pupil/releases/latest)_ that publishes realtime gaze data using the [lab streaming layer](https://github.com/sccn/labstreaminglayer) framework.
4+
5+
## Installation
6+
7+
Please see our documentation on where to find the [user plugin directory](https://docs.pupil-labs.com/developer/core/plugin-api/#adding-a-plugin).
8+
9+
1. Install `pylsl`
10+
2. Copy or symlink `pylsl` with all its content to the _plugin directory_.
11+
3. Copy [`pupil_capture_lsl_relay.py`](pupil_capture_lsl_relay.py) to the _plugin directory_.
12+
13+
14+
## Usage
15+
16+
1. Start _Pupil Capture_.
17+
2. [Open the _Pupil Capture LSL Relay_ plugin](https://docs.pupil-labs.com/core/software/pupil-capture/#plugins).
18+
3. Now the LSL outlet is ready to provide data to other inlets in the network.
19+
20+
## LSL Outlet
21+
22+
The plugin opens a single outlet named `pupil_capture` that follows the [Gaze Meta Data](https://github.com/sccn/xdf/wiki/Gaze-Meta-Data) format.
23+
24+
See our [pupil-helpers](https://github.com/pupil-labs/pupil-helpers/tree/master/LabStreamingLayer) for examples on how to record and visualize the published data.
25+
26+
The published LSL data is simply a flattened version (see `extract_*()` functions in `pupil_capture_lsl_relay.py`) of the original Pupil gaze data stream. The stream's channels will be filled with best effort, i.e. if there is a monocular gaze datum the values for the opposite eye will be set to `NaN`. The actual pairing of pupil data to binocular gaze data happens in [Capture](https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/calibration_routines/gaze_mappers.py#L95-L140) and is not a LSL specific behaviour. Therefore, it is possible to apply the same [flattening code](https://github.com/papr/App-PupilLabs/blob/master/pupil_lsl_relay.py#L226-L287) to offline calibrated gaze data and reproduce the stream published by the LSL outlet.
27+
28+
## Data Format
29+
30+
'confidence': Normalized (0-1) confidence.
31+
32+
'norm_pos_x', 'norm_pos_y': Normalized (0-1) coordinates on the screen.
33+
34+
'gaze_point_3d_x', 'gaze_point_3d_y', 'gaze_point_3d_z': World coordinates in mm
35+
36+
'eye_centerright_3d_x' ... (for right/left eyes, for x/y/z): Position of eye center in world coordinates in mm.
37+
38+
'gaze_normalright_x' (right/left, x/y/z): End point of vector from eye center (I think).
39+
40+
'diameterright_2d' (right/left): Pupil diameter in pixels
41+
42+
'diameterright_3d' (right/left): Pupil diameter in mm
43+
44+
## LSL Clock Synchronization
45+
46+
The `Pupil LSL Relay` plugin adjusts Capture's timebase to synchronize Capture's own clock with the `pylsl.local_clock()`. This allows the recording of native Capture timestamps and removes the necessity of manually synchronize timestamps after the effect.
47+
48+
**Warning**: The time synchronization will potentially break if other time alternating actors (e.g. the `Time Sync` plugin or `hmd-eyes`) are active.

pupil_lsl_relay.py renamed to pupil_capture/pupil_capture_lsl_relay.py

File renamed without changes.

‎pupil_invisible_lsl_relay/README.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Pupil Invisible Gaze Relay
2+
3+
## Installation
4+
5+
```bash
6+
git clone https://github.com/labstreaminglayer/App-PupilLabs.git
7+
8+
cd App-PupilLabs/
9+
git checkout pupil-invisible-relay
10+
11+
# Use the Python 3 installation of your choice
12+
python -m pip install -U pip
13+
python -m pip install -r requirements.txt
14+
```
15+
16+
## Usage
17+
18+
#### Basic mode
19+
20+
The basic usage of the Pupil Invisible Gaze Relay module is to provide a device host name as an argument. The module will wait for that device to announce a gaze sensor, will connect to it and start pushing the gaze data to the LSL outlet named `pupil_invisible`.
21+
22+
```bash
23+
pupil_invisible_lsl_relay --host-name <DEVICE_NAME>
24+
```
25+
26+
#### Interactive mode
27+
28+
In interactive mode, there is no need to provide the device name beforehand. Instead, the module monitors the network and shows a list of available devices which the user can select.
29+
30+
```bash
31+
pupil_invisible_lsl_relay
32+
```

‎pupil_invisible_lsl_relay/__init__.py‎

Whitespace-only changes.

‎pupil_invisible_lsl_relay/__main__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
from .cli import main
2+
3+
if __name__ == "__main__":
4+
main()

0 commit comments

Comments
 (0)