-
Notifications
You must be signed in to change notification settings - Fork 52
Create streamlined 'golden path' installer for new users #1414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 79 commits
5bd3372
ca4e571
e810f97
580afe9
faa513d
fe1f1bf
37121e4
37dd736
f9b3bab
8f4631c
0bd6148
74b0b82
3a6782d
eb46f79
f585d37
aafae3c
64fc378
776774f
a32d6a4
36b5178
4c8e4a4
9ac86df
4e36964
b4f8434
b94899d
056325c
5d78e92
f89970c
fd5097a
3120e91
65ec4aa
20c991e
d0279d5
79f0d28
58c6dd5
5fae72c
c882f99
334e718
58c4a1e
8cb7cf3
c7bc043
b36cc7f
b7f2666
9c16b7c
25552be
14c5257
9922e44
d7595e3
4df0b3d
c393556
b4beaee
9fa8811
e1be1b7
39e5953
a4df4a0
505a13d
0edf34f
0114bdf
33718ac
c9826fe
d4fd310
00a2cbd
0db4a02
8375e22
d3d12c9
3e45d0d
24dd1a7
5f70148
b7ed983
a6e9798
0a8486a
e8f99e5
39d1b54
32ed3b2
a710e67
2fd03aa
c283631
e1a4742
bdfb0cd
3478130
e7da335
1d56f0e
df56596
a48ae80
b4aeced
aad2387
03a33c6
a941b24
3125f3a
3e1581a
d99d873
7d97d90
c8dc6ed
96d841e
285c65a
e68c610
9944aef
d135748
d3b8fdc
f85360a
f3deccb
e80b822
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Spyglass Docker Compose Configuration | ||
| # | ||
| # This file shows available configuration options with their defaults. | ||
| # Docker Compose works fine without a .env file - defaults work for most users. | ||
| # | ||
| # To customize (optional): | ||
| # 1. Copy this file: cp .env.example .env | ||
| # 2. Edit .env with your preferred settings | ||
| # 3. Run: docker compose up -d | ||
| # | ||
| # IMPORTANT: If you change MYSQL_PORT or MYSQL_ROOT_PASSWORD, you must also | ||
| # update your DataJoint configuration file (dj_local_conf.json) to match. | ||
| # See docs/DATABASE.md for details on DataJoint configuration. | ||
|
|
||
| # ============================================================================= | ||
| # Database Configuration (Required) | ||
| # ============================================================================= | ||
|
|
||
| # MySQL root password (default: tutorial) | ||
| # For local development, 'tutorial' is fine | ||
| # For production, use a strong password | ||
| MYSQL_ROOT_PASSWORD=tutorial | ||
|
|
||
| # MySQL port (default: 3306) | ||
| # Change this if port 3306 is already in use | ||
| MYSQL_PORT=3306 | ||
|
|
||
| # MySQL Docker image (default: datajoint/mysql:8.0) | ||
| # You can specify a different version if needed | ||
| MYSQL_IMAGE=datajoint/mysql:8.0 | ||
|
|
||
| # ============================================================================= | ||
| # Optional Configuration | ||
| # ============================================================================= | ||
|
|
||
| # Database name to create on startup (optional) | ||
| # Leave empty to skip database creation | ||
| MYSQL_DATABASE= | ||
|
|
||
| # ============================================================================= | ||
| # Notes | ||
| # ============================================================================= | ||
| # - Don't commit .env file to git (it contains passwords) | ||
| # - Port range: 1024-65535 | ||
| # - If you change MYSQL_PORT, update your DataJoint config accordingly |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| # Spyglass Quickstart (5 minutes) | ||
|
|
||
| Get from zero to analyzing neural data with Spyglass in just a few commands. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - **Python**: Version 3.9 or higher | ||
| - **Disk Space**: ~10GB for installation + data storage | ||
| - **Operating System**: macOS or Linux (Windows experimental) | ||
| - **Package Manager**: [conda](https://docs.conda.io/) (23.10.0+ recommended) or [mamba](https://mamba.readthedocs.io/) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally, I would have markdownlint break long lines for easier future edits/reviews. This opinionated formatter would also chage spacing around H3s below |
||
|
|
||
| If you don't have mamba/conda, install [miniforge](https://github.com/conda-forge/miniforge#install) first. | ||
|
|
||
| ## Installation (2 commands) | ||
|
|
||
| ### 1. Download and run quickstart | ||
| ```bash | ||
| # Clone the repository | ||
| git clone https://github.com/LorenFrankLab/spyglass.git | ||
| cd spyglass | ||
|
|
||
| # Run quickstart (minimal installation) | ||
| python scripts/quickstart.py | ||
|
||
| ``` | ||
|
|
||
| ### 2. Validate installation | ||
| ```bash | ||
| # Activate the environment | ||
| conda activate spyglass | ||
|
|
||
| # Run validation | ||
| python scripts/validate_spyglass.py -v | ||
| ``` | ||
|
|
||
| **That's it!** Total time: ~5-10 minutes | ||
|
|
||
| ## Next Steps | ||
|
|
||
| ### Run first tutorial | ||
| ```bash | ||
| cd notebooks | ||
| jupyter notebook 01_Concepts.ipynb | ||
| ``` | ||
|
|
||
| ### Configure for your data | ||
| - Place NWB files in `~/spyglass_data/raw/` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should 'where do you want your base dir?' be an additional question? This seems like it would set up a default few people would use, and then creates the issue of having the low-tech user change the config after the fact
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed, if nothing else this approach means all our lab members would need to change it after running this. Seems a reasonable option to ask for input
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It appears the code is updated to allow this but not this doc |
||
| - See [Data Import Guide](https://lorenfranklab.github.io/spyglass/latest/notebooks/01_Insert_Data/) for details | ||
|
|
||
| ### Join community | ||
| - 📖 [Documentation](https://lorenfranklab.github.io/spyglass/) | ||
| - 💬 [GitHub Discussions](https://github.com/LorenFrankLab/spyglass/discussions) | ||
| - 🐛 [Report Issues](https://github.com/LorenFrankLab/spyglass/issues) | ||
|
|
||
| --- | ||
|
|
||
| ## Installation Options | ||
|
|
||
| Need something different? The quickstart supports these options: | ||
|
|
||
| ```bash | ||
| python scripts/quickstart.py --full # All optional dependencies | ||
| python scripts/quickstart.py --pipeline=dlc # DeepLabCut pipeline | ||
| python scripts/quickstart.py --no-database # Skip database setup | ||
| python scripts/quickstart.py --help # See all options | ||
| ``` | ||
|
|
||
| ## What Gets Installed | ||
|
|
||
| The quickstart creates: | ||
| - **Conda environment** with Spyglass and core dependencies | ||
| - **MySQL database** (local Docker container) | ||
| - **Data directories** in `~/spyglass_data/` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See above comment
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doc needs updated here too |
||
| - **Jupyter environment** for running tutorials | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Installation fails? | ||
| ```bash | ||
| # Remove environment and retry | ||
| conda env remove -n spyglass | ||
| python scripts/quickstart.py | ||
| ``` | ||
|
|
||
| ### Validation fails? | ||
| 1. Check error messages for specific issues | ||
| 2. Ensure Docker is running (for database) | ||
| 3. Try: `python scripts/quickstart.py --no-database` | ||
|
|
||
| ### Need help? | ||
| - Check [Advanced Setup Guide](https://lorenfranklab.github.io/spyglass/latest/notebooks/00_Setup/) for manual installation | ||
| - Ask questions in [GitHub Discussions](https://github.com/LorenFrankLab/spyglass/discussions) | ||
|
|
||
| --- | ||
|
|
||
| **Next tutorial**: [01_Concepts.ipynb](notebooks/01_Concepts.ipynb) | ||
| **Full documentation**: [lorenfranklab.github.io/spyglass](https://lorenfranklab.github.io/spyglass/) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Spyglass Database Setup with Docker Compose | ||
| # | ||
| # Quick start (no setup needed - defaults work for most users): | ||
| # docker compose up -d | ||
| # | ||
| # This starts a MySQL database for Spyglass with: | ||
| # - Persistent data storage (survives container restart) | ||
| # - Health checks (ensures database is ready) | ||
| # - Standard configuration (matches manual Docker setup) | ||
| # | ||
| # Common tasks: | ||
| # Start: docker compose up -d | ||
| # Stop: docker compose stop | ||
| # Logs: docker compose logs mysql | ||
| # Restart: docker compose restart | ||
| # Remove: docker compose down -v # WARNING: Deletes all data! | ||
| # | ||
| # Customization (optional): | ||
| # Create .env file from .env.example to customize settings | ||
| # See .env.example for available configuration options | ||
| # | ||
| # Troubleshooting: | ||
| # Port 3306 in use: Create .env file and change MYSQL_PORT | ||
| # Services won't start: Run 'docker compose logs' to see errors | ||
| # Can't connect: Ensure Docker Desktop is running | ||
|
|
||
| services: | ||
| mysql: | ||
| image: ${MYSQL_IMAGE:-datajoint/mysql:8.0} | ||
|
|
||
| # Container name MUST be 'spyglass-db' to match existing code | ||
| container_name: spyglass-db | ||
|
|
||
| ports: | ||
| - "${MYSQL_PORT:-3306}:3306" | ||
|
|
||
| environment: | ||
| MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-tutorial} | ||
| # Optional: Create database on startup | ||
| MYSQL_DATABASE: ${MYSQL_DATABASE:-} | ||
|
|
||
| volumes: | ||
| # Named volume for persistent storage | ||
| # Data survives 'docker compose down' but is removed by 'down -v' | ||
| - spyglass-db-data:/var/lib/mysql | ||
|
|
||
| healthcheck: | ||
| # Check if MySQL is ready without exposing password in process list | ||
| test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "--silent"] | ||
| interval: 10s | ||
| timeout: 5s | ||
| retries: 5 | ||
| start_period: 30s | ||
|
|
||
| restart: unless-stopped | ||
|
|
||
| networks: | ||
| - spyglass-network | ||
|
|
||
| volumes: | ||
| spyglass-db-data: | ||
| # Explicit name for predictability | ||
| name: spyglass-db-data | ||
|
|
||
| networks: | ||
| spyglass-network: | ||
| name: spyglass-network | ||
| driver: bridge |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A desire to bump to 3.10 has come up in a fer recent PR reviews for easier type hinting