Skip to content

Commit 7f4ca37

Browse files
docs: secrets troubleshooting fixes + framework-version sweep (2.0 → 2.0+) (#190)
* docs(secrets): real diagnostic snippet + rotate-first incident order - Replace the non-existent dj.config._config_sources line with a working snippet that uses find_config_file / find_secrets_dir from datajoint.settings to show which paths were resolved. - Reorder "Accidentally Committed Secrets" so credential rotation is the first concrete step — history rewrite is cleanup that doesn't un-leak the secret. Add a warning admonition spelling that out, and expand step 1 with the specific credential classes to rotate (database, object store, third-party tokens) and the access-log audit. * docs: refer to documented framework as DataJoint 2.0+ The docs cover the entire 2.x line, not the 2.0 release specifically. Switch product-level references from "DataJoint 2.0" to "DataJoint 2.0+" in README and the how-to / reference / tutorial pages that describe the framework as it currently exists. Historical statements ("introduced in 2.0", "What's New in 2.0", paper citation "DataJoint 2.0: A Computational Substrate...", pre-2.0 vs 2.0 implementation comparisons) intentionally keep their literal version references.
1 parent aa2f5de commit 7f4ca37

8 files changed

Lines changed: 40 additions & 15 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# DataJoint Documentation
22

3-
Official documentation for [DataJoint](https://github.com/datajoint/datajoint-python) 2.0 — an open-source framework for building scientific data pipelines.
3+
Official documentation for [DataJoint](https://github.com/datajoint/datajoint-python) 2.0+ — an open-source framework for building scientific data pipelines.
44

55
**📖 Live site:** https://docs.datajoint.com
66

src/how-to/deploy-production.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Development and production environments have different requirements:
1212
| Naming | Ad-hoc schema names | Consistent project prefixes |
1313
| Configuration | Local settings | Environment-based |
1414

15-
DataJoint 2.0 provides settings to enforce production discipline.
15+
DataJoint 2.0+ provides settings to enforce production discipline.
1616

1717
## Prevent Automatic Table Creation
1818

src/how-to/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Install DataJoint Python and set up your environment.
44

5-
## Install DataJoint 2.0
5+
## Install DataJoint 2.0+
66

77
```bash
88
pip install datajoint

src/how-to/manage-secrets.md

Lines changed: 31 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -425,11 +425,21 @@ print(dj.config)
425425

426426
# Check specific setting
427427
print(dj.config['database.user'])
428+
```
429+
430+
**Find which files were picked up:**
431+
432+
```python
433+
from datajoint.settings import find_config_file, find_secrets_dir
428434

429-
# See where value came from
430-
print(dj.config._config_sources) # Not a real attribute, just conceptual
435+
config_path = find_config_file()
436+
print("config file:", config_path)
437+
print("secrets dir:", find_secrets_dir(config_path))
431438
```
432439

440+
If `find_config_file()` returns `None`, no `datajoint.json` was found via the
441+
upward search — values come from environment variables and defaults only.
442+
433443
### Permission Errors
434444

435445
```bash
@@ -455,10 +465,25 @@ conn = dj.conn(reset=True)
455465

456466
### Accidentally Committed Secrets
457467

458-
**Immediate actions:**
468+
!!! warning "Assume the secret is compromised"
469+
Once a credential reaches a remote branch, treat it as public. Anyone with
470+
repository access — and any service that mirrors or caches commits — may
471+
already have it. Rewriting git history does **not** un-leak the secret;
472+
rotation does.
473+
474+
**Step 1 — Rotate every exposed credential first.** Do this before touching
475+
git history; until rotation completes, the leaked secret is still valid.
476+
477+
- Database users (`database.user` / `database.password`): change the password
478+
on the server, then update your local `.secrets/datajoint.json`.
479+
- Object-store credentials (`stores.<name>.access_key` / `secret_key`, or the
480+
equivalent in your cloud provider): issue new keys and revoke the old ones.
481+
- Any third-party tokens that appeared in the same file.
482+
- Audit recent access logs (DB connection log, S3 access log) for unfamiliar
483+
IPs or unexpected activity during the exposure window.
459484

460-
1. **Rotate credentials immediately**
461-
2. Remove from git history:
485+
**Step 2 — Remove from git history.** After rotation, scrub the file so old
486+
clones don't leak it further:
462487

463488
```bash
464489
# Remove file from history
@@ -470,7 +495,7 @@ git filter-branch --force --index-filter \
470495
git push origin --force --all
471496
```
472497

473-
3. **Verify removal:**
498+
**Step 3 — Verify removal:**
474499

475500
```bash
476501
git log --all --full-history -- .secrets/datajoint.json

src/how-to/migrate-to-v20.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Migrate to DataJoint 2.0
22

3-
Upgrade existing pipelines from legacy DataJoint (pre-2.0) to DataJoint 2.0.
3+
Upgrade existing pipelines from legacy DataJoint (pre-2.0) to DataJoint 2.0+.
44

55
> **This guide is optimized for AI coding assistants.** Point your AI agent at this
66
> document and it will execute the migration with your oversight.
@@ -13,14 +13,14 @@ Upgrade existing pipelines from legacy DataJoint (pre-2.0) to DataJoint 2.0.
1313

1414
### System Requirements
1515

16-
| Component | Legacy (pre-2.0) | DataJoint 2.0 |
16+
| Component | Legacy (pre-2.0) | DataJoint 2.0+ |
1717
|-----------|-----------------|---------------|
1818
| **Python** | 3.8+ | **3.10+** |
1919
| **MySQL** | 5.7+ | **8.0+** |
2020
| **Character encoding** | (varies) | **UTF-8 (utf8mb4)** |
2121
| **Collation** | (varies) | **utf8mb4_bin** |
2222

23-
**Action required:** Upgrade your Python environment and MySQL server before installing DataJoint 2.0.
23+
**Action required:** Upgrade your Python environment and MySQL server before installing DataJoint 2.0+.
2424

2525
**Character encoding and collation:** DataJoint 2.0 standardizes on UTF-8 encoding with binary collation (case-sensitive comparisons). This is configured **server-wide** and is assumed by DataJoint:
2626

src/reference/specs/fetch-api.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# DataJoint 2.0 Fetch API Specification
1+
# DataJoint 2.0+ Fetch API Specification
22

33
## Overview
44

src/reference/specs/object-store-configuration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ DataJoint's Object-Augmented Schema (OAS) integrates relational tables with obje
88

99
### Storage Models
1010

11-
DataJoint 2.0 supports three storage models, all sharing the same store configuration:
11+
DataJoint 2.0+ supports three storage models, all sharing the same store configuration:
1212

1313
| Model | Data Types | Path Structure | Integration | Use Case |
1414
|-------|------------|----------------|-------------|----------|

src/tutorials/basics/04-queries.ipynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4292,7 +4292,7 @@
42924292
"source": [
42934293
"## Fetching Data\n",
42944294
"\n",
4295-
"DataJoint 2.0 provides explicit methods for different output formats.\n",
4295+
"DataJoint 2.0+ provides explicit methods for different output formats.\n",
42964296
"\n",
42974297
"### `to_dicts()` — List of Dictionaries"
42984298
]

0 commit comments

Comments
 (0)