-
Notifications
You must be signed in to change notification settings - Fork 52
Create streamlined 'golden path' installer for new users #1414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 65 commits
5bd3372
ca4e571
e810f97
580afe9
faa513d
fe1f1bf
37121e4
37dd736
f9b3bab
8f4631c
0bd6148
74b0b82
3a6782d
eb46f79
f585d37
aafae3c
64fc378
776774f
a32d6a4
36b5178
4c8e4a4
9ac86df
4e36964
b4f8434
b94899d
056325c
5d78e92
f89970c
fd5097a
3120e91
65ec4aa
20c991e
d0279d5
79f0d28
58c6dd5
5fae72c
c882f99
334e718
58c4a1e
8cb7cf3
c7bc043
b36cc7f
b7f2666
9c16b7c
25552be
14c5257
9922e44
d7595e3
4df0b3d
c393556
b4beaee
9fa8811
e1be1b7
39e5953
a4df4a0
505a13d
0edf34f
0114bdf
33718ac
c9826fe
d4fd310
00a2cbd
0db4a02
8375e22
d3d12c9
3e45d0d
24dd1a7
5f70148
b7ed983
a6e9798
0a8486a
e8f99e5
39d1b54
32ed3b2
a710e67
2fd03aa
c283631
e1a4742
bdfb0cd
3478130
e7da335
1d56f0e
df56596
a48ae80
b4aeced
aad2387
03a33c6
a941b24
3125f3a
3e1581a
d99d873
7d97d90
c8dc6ed
96d841e
285c65a
e68c610
9944aef
d135748
d3b8fdc
f85360a
f3deccb
e80b822
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| # Spyglass Quickstart (5 minutes) | ||
|
|
||
| Get from zero to analyzing neural data with Spyglass in just a few commands. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - **Python**: Version 3.9 or higher | ||
| - **Disk Space**: ~10GB for installation + data storage | ||
| - **Operating System**: macOS or Linux (Windows experimental) | ||
| - **Package Manager**: [conda](https://docs.conda.io/) (23.10.0+ recommended) or [mamba](https://mamba.readthedocs.io/) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally, I would have markdownlint break long lines for easier future edits/reviews. This opinionated formatter would also chage spacing around H3s below |
||
|
|
||
| If you don't have mamba/conda, install [miniforge](https://github.com/conda-forge/miniforge#install) first. | ||
|
|
||
| ## Installation (2 commands) | ||
|
|
||
| ### 1. Download and run quickstart | ||
| ```bash | ||
| # Clone the repository | ||
| git clone https://github.com/LorenFrankLab/spyglass.git | ||
| cd spyglass | ||
|
|
||
| # Run quickstart (minimal installation) | ||
| python scripts/quickstart.py | ||
|
||
| ``` | ||
|
|
||
| ### 2. Validate installation | ||
| ```bash | ||
| # Activate the environment | ||
| conda activate spyglass | ||
|
|
||
| # Run validation | ||
| python scripts/validate_spyglass.py -v | ||
| ``` | ||
|
|
||
| **That's it!** Total time: ~5-10 minutes | ||
|
|
||
| ## Next Steps | ||
|
|
||
| ### Run first tutorial | ||
| ```bash | ||
| cd notebooks | ||
| jupyter notebook 01_Concepts.ipynb | ||
| ``` | ||
|
|
||
| ### Configure for your data | ||
| - Place NWB files in `~/spyglass_data/raw/` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should 'where do you want your base dir?' be an additional question? This seems like it would set up a default few people would use, and then creates the issue of having the low-tech user change the config after the fact
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed, if nothing else this approach means all our lab members would need to change it after running this. Seems a reasonable option to ask for input
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It appears the code is updated to allow this but not this doc |
||
| - See [Data Import Guide](https://lorenfranklab.github.io/spyglass/latest/notebooks/01_Insert_Data/) for details | ||
|
|
||
| ### Join community | ||
| - 📖 [Documentation](https://lorenfranklab.github.io/spyglass/) | ||
| - 💬 [GitHub Discussions](https://github.com/LorenFrankLab/spyglass/discussions) | ||
| - 🐛 [Report Issues](https://github.com/LorenFrankLab/spyglass/issues) | ||
|
|
||
| --- | ||
|
|
||
| ## Installation Options | ||
|
|
||
| Need something different? The quickstart supports these options: | ||
|
|
||
| ```bash | ||
| python scripts/quickstart.py --full # All optional dependencies | ||
| python scripts/quickstart.py --pipeline=dlc # DeepLabCut pipeline | ||
| python scripts/quickstart.py --no-database # Skip database setup | ||
| python scripts/quickstart.py --help # See all options | ||
| ``` | ||
|
|
||
| ## What Gets Installed | ||
|
|
||
| The quickstart creates: | ||
| - **Conda environment** with Spyglass and core dependencies | ||
| - **MySQL database** (local Docker container) | ||
| - **Data directories** in `~/spyglass_data/` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See above comment
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doc needs updated here too |
||
| - **Jupyter environment** for running tutorials | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Installation fails? | ||
| ```bash | ||
| # Remove environment and retry | ||
| conda env remove -n spyglass | ||
| python scripts/quickstart.py | ||
| ``` | ||
|
|
||
| ### Validation fails? | ||
| 1. Check error messages for specific issues | ||
| 2. Ensure Docker is running (for database) | ||
| 3. Try: `python scripts/quickstart.py --no-database` | ||
|
|
||
| ### Need help? | ||
| - Check [Advanced Setup Guide](https://lorenfranklab.github.io/spyglass/latest/notebooks/00_Setup/) for manual installation | ||
| - Ask questions in [GitHub Discussions](https://github.com/LorenFrankLab/spyglass/discussions) | ||
|
|
||
| --- | ||
|
|
||
| **Next tutorial**: [01_Concepts.ipynb](notebooks/01_Concepts.ipynb) | ||
| **Full documentation**: [lorenfranklab.github.io/spyglass](https://lorenfranklab.github.io/spyglass/) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # Minimal Spyglass Environment - Core Dependencies Only | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I intentionally spaced the yml files so it wold be easy to diff them at a glance by flipping between and seeing what was commented out in one vs another. I find that helps maintenance This is now our 5th environment file, so I think they belong in a directory rather than at root |
||
| # 1. Install: `mamba env create -f environment-min.yml` | ||
| # 2. Activate: `conda activate spyglass` | ||
| # 3. For full features, use environment.yml instead | ||
| # | ||
| # This environment includes only the essential dependencies needed for basic | ||
| # spyglass functionality. Optional dependencies for specific pipelines | ||
| # (LFP analysis, spike sorting, etc.) are excluded. | ||
|
|
||
| name: spyglass | ||
| channels: | ||
| - conda-forge | ||
| - franklab | ||
| - edeno | ||
| dependencies: | ||
| # Core Python scientific stack | ||
| - python>=3.9,<3.13 | ||
| - pip | ||
| - numpy | ||
| - matplotlib | ||
| - bottleneck | ||
| - seaborn | ||
|
|
||
| # Core Jupyter environment for notebooks | ||
| - jupyterlab>=3.* | ||
| - ipympl | ||
|
|
||
| # Essential spyglass dependencies (installed via pip for development mode) | ||
| - pip: | ||
| - . | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| """ | ||
| Common utilities shared between Spyglass scripts. | ||
|
|
||
| This module provides shared functionality for quickstart.py and validate_spyglass.py | ||
| to improve consistency and reduce code duplication. | ||
| """ | ||
|
|
||
| from collections import namedtuple | ||
| from enum import Enum | ||
| from typing import NamedTuple | ||
|
|
||
|
|
||
| # Shared color definitions using namedtuple (immutable and functional) | ||
| Colors = namedtuple( | ||
| "Colors", | ||
| [ | ||
| "HEADER", | ||
| "OKBLUE", | ||
| "OKCYAN", | ||
| "OKGREEN", | ||
| "WARNING", | ||
| "FAIL", | ||
| "ENDC", | ||
| "BOLD", | ||
| "UNDERLINE", | ||
| ], | ||
| )( | ||
| HEADER="\033[95m", | ||
| OKBLUE="\033[94m", | ||
| OKCYAN="\033[96m", | ||
| OKGREEN="\033[92m", | ||
| WARNING="\033[93m", | ||
| FAIL="\033[91m", | ||
| ENDC="\033[0m", | ||
| BOLD="\033[1m", | ||
| UNDERLINE="\033[4m", | ||
| ) | ||
|
|
||
| # Disabled colors for no-color mode | ||
| DisabledColors = namedtuple( | ||
| "DisabledColors", | ||
| [ | ||
| "HEADER", | ||
| "OKBLUE", | ||
| "OKCYAN", | ||
| "OKGREEN", | ||
| "WARNING", | ||
| "FAIL", | ||
| "ENDC", | ||
| "BOLD", | ||
| "UNDERLINE", | ||
| ], | ||
| )(*([""] * 9)) | ||
|
|
||
|
|
||
| # Shared exception hierarchy | ||
| class SpyglassSetupError(Exception): | ||
| """Base exception for setup errors.""" | ||
|
|
||
| pass | ||
|
|
||
|
|
||
| class SystemRequirementError(SpyglassSetupError): | ||
| """System doesn't meet requirements.""" | ||
|
|
||
| pass | ||
|
|
||
|
|
||
| class EnvironmentCreationError(SpyglassSetupError): | ||
| """Environment creation failed.""" | ||
|
|
||
| pass | ||
|
|
||
|
|
||
| class DatabaseSetupError(SpyglassSetupError): | ||
| """Database setup failed.""" | ||
|
|
||
| pass | ||
|
|
||
|
|
||
| # Enums for type-safe choices | ||
| class MenuChoice(Enum): | ||
| """User menu choices for installation type.""" | ||
|
|
||
| MINIMAL = 1 | ||
| FULL = 2 | ||
| PIPELINE = 3 | ||
|
|
||
|
|
||
| class DatabaseChoice(Enum): | ||
| """Database setup choices.""" | ||
|
|
||
| DOCKER = 1 | ||
| EXISTING = 2 | ||
| SKIP = 3 | ||
|
|
||
|
|
||
| class ConfigLocationChoice(Enum): | ||
| """Configuration file location choices.""" | ||
|
|
||
| REPO_ROOT = 1 | ||
| CURRENT_DIR = 2 | ||
| CUSTOM = 3 | ||
|
|
||
|
|
||
| class PipelineChoice(Enum): | ||
| """Pipeline-specific installation choices.""" | ||
|
|
||
| DLC = 1 | ||
| MOSEQ_CPU = 2 | ||
| MOSEQ_GPU = 3 | ||
| LFP = 4 | ||
| DECODING = 5 | ||
|
|
||
|
|
||
| # Configuration constants | ||
| class Config: | ||
| """Centralized configuration constants.""" | ||
|
|
||
| DEFAULT_TIMEOUT = 1800 # 30 minutes | ||
| DEFAULT_DB_PORT = 3306 | ||
| DEFAULT_ENV_NAME = "spyglass" | ||
|
|
||
| # Timeouts for different operations | ||
| TIMEOUTS = { | ||
| "environment_create": 1800, # 30 minutes for env creation | ||
| "package_install": 600, # 10 minutes for packages | ||
| "database_check": 60, # 1 minute for DB readiness | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| """Core business logic modules for Spyglass setup. | ||
|
|
||
| This package contains pure functions and business logic extracted from | ||
| the main setup scripts, as recommended in REVIEW.md. | ||
| """ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A desire to bump to 3.10 has come up in a fer recent PR reviews for easier type hinting