Skip to content

Commit e31e350

Browse files
committed
templates: add Obsidian /note skill + restic backup script
Two pieces previously living only in the production workspace: skills/note/SKILL.md Obsidian-formatted note creation skill. Two modes (verbatim fix, summarize/concept), YAML frontmatter requirement (3-5 tags + YYYY-MMM-DD created), full Obsidian callout vocabulary ([!bug], [!success], [!tip], [!warning], [!info], [!note], [!example], [!question]) with usage rules. Sync target is left configurable (rclone-to-gdrive, local vault, bisync timer). scripts/backup-restic.sh Twice-daily restic backup to two destinations: Google Drive via rclone and a local NAS mount. Encrypted, deduplicated, retention 7d/4w/3m. Conservative rclone throttling to survive Drive quota pressure. Skips NAS cleanly if not mounted. Sanitized paths and repo names. Plus two memory cards that explain when/why to use each: memory/cards/obsidian-notes.md Verbatim-vs-concept mode selection, callout vocabulary table, sync target shapes, when NOT to /note (durable agent knowledge belongs in cards, not the vault), relationship to the memory system. memory/cards/backup-restic.md Two-destination rationale (workstation-fail vs NAS-fail vs account-lock vs ransomware matrix), cadence (twice-daily 03/15), default paths and excludes, retention math, rclone throttle reasoning, NAS lockfile handling, recovery drill, monitoring hook, anti-patterns. Wired into the workspace profile manifest with the script marked executable. Tests cover both new files and the executable bit.
1 parent a5a2c53 commit e31e350

10 files changed

Lines changed: 1085 additions & 0 deletions

File tree

memory/cards/backup-restic.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
---
2+
topic: backup-restic
3+
category: infrastructure
4+
tags: [backup, restic, rclone, gdrive, nas, retention, recovery]
5+
---
6+
7+
# Workspace Backup (Restic + rclone + NAS)
8+
9+
Twice-daily restic backups to two destinations: Google Drive (via rclone) and a local NAS mount. Encrypted, deduplicated, snapshot-pruned. The reference script ships at `scripts/backup-restic.sh`; this card explains *why* the shape is what it is.
10+
11+
## Why both destinations
12+
13+
| Failure mode | Local NAS only | gdrive only | Both |
14+
|--------------|----------------|-------------|------|
15+
| Workstation disk dies | Recover from NAS, fast | Recover from gdrive, slow | Either |
16+
| NAS hardware failure | Lose everything | Recover from gdrive | gdrive saves you |
17+
| Google account locked / quota | Lose everything that ran since last NAS run | Lose everything | NAS saves you |
18+
| Ransomware hits workstation | Possibly hits NAS too | Off-site immutable copy | gdrive saves you |
19+
20+
Two destinations covers the "one of them is broken" case without raising the recovery time for the common case (NAS is faster).
21+
22+
## Cadence
23+
24+
```cron
25+
0 3,15 * * * /path/to/backup-restic.sh
26+
```
27+
28+
03:00 + 15:00. Twice a day means worst-case data loss window is ~12 hours. Adjust if your write velocity is higher.
29+
30+
## What gets backed up
31+
32+
Default paths:
33+
34+
- The agent workspace (`~/.solo-mise` or your equivalent)
35+
- All repos under `~/repos`
36+
- Local scripts and bin (`~/bin`)
37+
- Dotfiles: `.bashrc`, `.profile`, `.gitconfig`, `.ssh`, `.claude`, `.codex`, `.npmrc`
38+
- Notes (`~/notes`)
39+
- Obsidian vault (`~/Obsidian`)
40+
41+
Excluded by default: `node_modules`, `.git/objects`, `__pycache__`, `*.pyc`, `.venv`, `dist`, `build`, `.next`, `.astro`, `coverage`, `.turbo`, `*.jsonl`, `.pm2/logs`, `.pm2/pids`, `.ollama`, `.obsidian/workspace*.json`, `.obsidian/cache`, `.trash`.
42+
43+
Edit the `BACKUP_PATHS` and `EXCLUDES` arrays in the script for your stack.
44+
45+
## Retention
46+
47+
```text
48+
--keep-daily 7
49+
--keep-weekly 4
50+
--keep-monthly 3
51+
```
52+
53+
About 14 snapshots overlapping over three months. Plenty for human-paced workflows.
54+
55+
## Why rclone is throttled hard
56+
57+
Google Drive can reject bursty restic-over-rclone writes when other rclone jobs (Obsidian bisync, cookbook sync, etc.) are running. The script sets:
58+
59+
```bash
60+
RCLONE_TRANSFERS=1
61+
RCLONE_CHECKERS=2
62+
RCLONE_TPSLIMIT=4
63+
RCLONE_TPSLIMIT_BURST=4
64+
RCLONE_DRIVE_PACER_MIN_SLEEP=500ms
65+
RCLONE_DRIVE_PACER_BURST=10
66+
RCLONE_RETRIES=8
67+
RCLONE_LOW_LEVEL_RETRIES=20
68+
```
69+
70+
Conservative on purpose. A backup that takes 40 minutes and finishes beats one that races, hits Drive quota, and dies in a retry loop.
71+
72+
## NAS shape
73+
74+
The NAS mount is typically an SMB/NFS share at `/mnt/nas/backups`. The script:
75+
76+
1. Skips cleanly if the mount is not present.
77+
2. Uses a separate restic repo path on the NAS (independent encryption + deduplication state).
78+
3. Tags NAS snapshots distinctly (`scheduled-nas`) so summaries are easy to read.
79+
80+
NAS-side considerations:
81+
82+
- **Permissions:** the NAS share must allow write from the workstation user.
83+
- **Lock files:** restic uses lockfiles inside the repo. A killed process can leave stale locks; the script runs `restic unlock --remove-all` after each successful backup.
84+
- **Read-only by default:** if the NAS holds irreplaceable family photos or other "do not touch" data, keep that data on a separate path and treat the rest of the NAS as read-only for the agent. See `SAFETY_RULES.md`.
85+
86+
## Password
87+
88+
```bash
89+
echo "<strong-password>" > ~/.solo-mise/.restic-password
90+
chmod 600 ~/.solo-mise/.restic-password
91+
```
92+
93+
Never commit this file. Never paste the password in a chat. If the password is lost, the encrypted snapshots are unrecoverable.
94+
95+
## Recovery
96+
97+
```bash
98+
# list snapshots
99+
restic snapshots
100+
101+
# restore a specific snapshot to /tmp/restore/
102+
restic restore <id> --target /tmp/restore
103+
104+
# restore just one path
105+
restic restore <id> --target /tmp/restore --include "$HOME/repos/<project>"
106+
```
107+
108+
Practice this. A backup you have never restored is a hope, not a backup.
109+
110+
## Monitoring
111+
112+
Log file path:
113+
114+
```text
115+
~/.solo-mise/logs/backup-YYYYMMDD.log
116+
```
117+
118+
Worth wiring into the morning report (`memory/cards/pipeline-standups.md`): grep recent logs for `ERROR` and surface in the briefing. A silent backup that has been failing for three weeks is the second-worst kind of bug.
119+
120+
## Anti-patterns
121+
122+
- **One destination only.** Single point of failure.
123+
- **No retention policy.** Old snapshots accumulate, gdrive quota fills, new backups fail.
124+
- **Backing up `node_modules`.** Wastes deduplication windows. Use the excludes.
125+
- **Backing up secrets unencrypted.** `.env` files get backed up too; that is intentional because restic encrypts everything at rest. The password file is the keystone; protect it.
126+
- **Skipping verification.** Run `restic check` periodically. Snapshots that exist but are corrupted are not snapshots.

memory/cards/obsidian-notes.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
topic: obsidian-notes
3+
category: foundation
4+
tags: [obsidian, notes, callouts, vault, sync, knowledge-capture]
5+
---
6+
7+
# Obsidian Notes (Verbatim + Concept)
8+
9+
The `/note` skill captures a session's durable knowledge as an Obsidian-formatted markdown file in `~/notes/`, then syncs it to the user's configured Obsidian vault inbox. It is the user-facing complement to the [memory-scanner](memory-scanner.md): cards are for the agent, Obsidian notes are for the human.
10+
11+
Full skill spec lives in `skills/note/SKILL.md`. This card explains *when* to use it and the two modes you must distinguish before writing.
12+
13+
## Two modes
14+
15+
| Mode | Use when | Output style |
16+
|------|----------|--------------|
17+
| **Verbatim fix** | Troubleshooting, root-causing, post-incident. Future-you needs the exact commands to reproduce the fix. | `[!bug]` -> `[!success]` -> "How it works" -> `[!warning]` gotchas. Heavy on exact strings and runnable code. |
18+
| **Summarize / concept** | Learning, capturing a workflow, documenting a system. The reader needs to understand *why* before they trust the *how*. | Overview -> How it works -> Example -> Tips and gotchas. More prose than callouts. |
19+
20+
Ask the user which mode applies if it is not obvious. Default to verbatim for bugs and to concept for everything else.
21+
22+
## Callout vocabulary
23+
24+
Obsidian callouts make the critical parts stick out without breaking prose flow. Use them sparingly. A note should be mostly regular markdown with callouts highlighting the parts that matter.
25+
26+
| Callout | Use for |
27+
|---------|---------|
28+
| `[!bug]` | Problems, errors, symptoms |
29+
| `[!success]` | Solutions, fixes, what worked |
30+
| `[!tip]` | Helpful hints, shortcuts |
31+
| `[!warning]` | Dangers, things that can break |
32+
| `[!info]` | Background context |
33+
| `[!note]` | General annotations |
34+
| `[!example]` | Practical examples, runnable commands |
35+
| `[!question]` | Open questions, things to investigate |
36+
37+
Syntax:
38+
39+
```markdown
40+
> [!success] Optional title
41+
> Content.
42+
> Multi-line is fine.
43+
```
44+
45+
## YAML frontmatter (required)
46+
47+
```yaml
48+
---
49+
tags:
50+
- tag1
51+
- tag2
52+
created: YYYY-MMM-DD
53+
---
54+
```
55+
56+
- 3-5 lowercase, hyphenated tags.
57+
- Date format `YYYY-MMM-DD` (e.g. `2026-Jan-24`). Not ISO. The vault sorts on this.
58+
59+
## Sync target
60+
61+
Three common shapes; the user picks one and documents it in `TOOLS.md`:
62+
63+
1. **Google Drive + rclone bisync.** `~/notes/` writes propagate to the vault inbox on the next bisync timer fire. Most common for cross-machine vaults.
64+
2. **Local Obsidian vault on disk.** `cp ~/notes/<slug>.md ~/Obsidian/<Vault>/<Inbox>/`. Same machine only.
65+
3. **Direct rclone copy.** `rclone copy ~/notes/<slug>.md "gdrive:My Drive/<VaultPath>/<Inbox>/"`. One-shot, no bisync.
66+
67+
If no sync target is configured, leave the file in `~/notes/` and surface the path. Do not invent a sync path.
68+
69+
## When NOT to use /note
70+
71+
- **Durable agent-facing knowledge** -> memory card via the handoff flow, not a vault note. Cards are searched semantically by the agent every session; vault notes are read by humans.
72+
- **Operational runbook detail** -> `TOOLS.md` or a `rules/*.md` file. The publish gate and memory ingester treat those as canonical.
73+
- **Sensitive personal context** -> not in `~/notes/` if the vault syncs to a cloud provider. Use the local-only sync path or skip the note entirely.
74+
75+
## Relationship to the memory system
76+
77+
Vault notes are human-readable references; memory cards are agent-readable durable knowledge. The two have different shapes and different audiences:
78+
79+
- A `/note` is written for future-you reading on a phone at a coffee shop.
80+
- A memory card is written for the agent to retrieve via `memory_search` mid-session.
81+
82+
When both apply, write the card first (through the handoff flow) and then optionally write a `/note` if the user will want to reference it outside the workspace. Do not duplicate content.

scripts/backup-restic.sh

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
#!/usr/bin/env bash
2+
# ============================================================================
3+
# Workspace backup via restic -> Google Drive (rclone) + local NAS mount
4+
# Run twice daily via cron. Tunable. Sanitize before commit.
5+
# ============================================================================
6+
#
7+
# Setup once:
8+
# 1. apt install restic rclone
9+
# 2. rclone config # set up `gdrive` remote
10+
# 3. echo "<strong-password>" > ~/.solo-mise/.restic-password
11+
# chmod 600 ~/.solo-mise/.restic-password
12+
# 4. (optional) mount your NAS at $NAS_MOUNT
13+
# 5. crontab -e:
14+
# 0 3,15 * * * /path/to/backup-restic.sh
15+
#
16+
set -euo pipefail
17+
18+
# ---- Paths ----
19+
WORKSPACE_ROOT="${WORKSPACE_ROOT:-${HOME}}"
20+
LOG_FILE="${WORKSPACE_ROOT}/.solo-mise/logs/backup-$(date +%Y%m%d).log"
21+
mkdir -p "$(dirname "$LOG_FILE")"
22+
23+
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"; }
24+
25+
# ---- Config (edit for your stack) ----
26+
RESTIC_REPO_GDRIVE="rclone:gdrive:Backup/<your-repo-name>"
27+
RESTIC_PASSWORD_FILE="${HOME}/.solo-mise/.restic-password"
28+
NAS_MOUNT="${NAS_MOUNT:-/mnt/nas/backups}"
29+
RESTIC_REPO_NAS="${NAS_MOUNT}/<your-repo-name>"
30+
31+
# Retention policy. Tune to taste; the defaults below cover ~3 months of
32+
# overlapping daily/weekly/monthly snapshots.
33+
KEEP_DAILY="${KEEP_DAILY:-7}"
34+
KEEP_WEEKLY="${KEEP_WEEKLY:-4}"
35+
KEEP_MONTHLY="${KEEP_MONTHLY:-3}"
36+
37+
# Paths to back up. Replace with your actual roots.
38+
BACKUP_PATHS=(
39+
"${HOME}/.solo-mise"
40+
"${HOME}/repos"
41+
"${HOME}/bin"
42+
"${HOME}/.bashrc"
43+
"${HOME}/.profile"
44+
"${HOME}/.gitconfig"
45+
"${HOME}/.ssh"
46+
"${HOME}/.claude"
47+
"${HOME}/.codex"
48+
"${HOME}/notes"
49+
"${HOME}/Obsidian"
50+
)
51+
52+
# Exclude rules shared between gdrive + NAS runs.
53+
EXCLUDES=(
54+
--exclude='node_modules'
55+
--exclude='.git/objects'
56+
--exclude='__pycache__'
57+
--exclude='*.pyc'
58+
--exclude='.venv'
59+
--exclude='venv'
60+
--exclude='dist'
61+
--exclude='build'
62+
--exclude='.next'
63+
--exclude='.astro'
64+
--exclude='coverage'
65+
--exclude='.turbo'
66+
--exclude='*.jsonl'
67+
--exclude='.pm2/logs'
68+
--exclude='.pm2/pids'
69+
--exclude='.ollama'
70+
--exclude='.obsidian/workspace*.json'
71+
--exclude='.obsidian/cache'
72+
--exclude='.trash'
73+
)
74+
75+
export RESTIC_PASSWORD_FILE
76+
77+
# ---- rclone tuning ----
78+
# Google Drive can reject bursty restic-over-rclone writes when other rclone
79+
# jobs are running. Keep the backend intentionally conservative so scheduled
80+
# backups finish reliably instead of spinning in quota retry loops.
81+
export RCLONE_TRANSFERS="${RCLONE_TRANSFERS:-1}"
82+
export RCLONE_CHECKERS="${RCLONE_CHECKERS:-2}"
83+
export RCLONE_TPSLIMIT="${RCLONE_TPSLIMIT:-4}"
84+
export RCLONE_TPSLIMIT_BURST="${RCLONE_TPSLIMIT_BURST:-4}"
85+
export RCLONE_DRIVE_PACER_MIN_SLEEP="${RCLONE_DRIVE_PACER_MIN_SLEEP:-500ms}"
86+
export RCLONE_DRIVE_PACER_BURST="${RCLONE_DRIVE_PACER_BURST:-10}"
87+
export RCLONE_RETRIES="${RCLONE_RETRIES:-8}"
88+
export RCLONE_LOW_LEVEL_RETRIES="${RCLONE_LOW_LEVEL_RETRIES:-20}"
89+
90+
# ---- Pre-flight ----
91+
command -v restic >/dev/null || { log "ERROR: restic not installed."; exit 1; }
92+
command -v rclone >/dev/null || { log "ERROR: rclone not installed."; exit 1; }
93+
[ -f "$RESTIC_PASSWORD_FILE" ] || { log "ERROR: password file not found: $RESTIC_PASSWORD_FILE"; exit 1; }
94+
95+
# ---- Helper: ensure a restic repo is usable ----
96+
ensure_repo() {
97+
local repo="$1"
98+
export RESTIC_REPOSITORY="$repo"
99+
if restic snapshots --json >/dev/null 2>&1; then
100+
log "Repo exists at $repo, proceeding."
101+
return 0
102+
fi
103+
log "Initializing repo: $repo"
104+
if restic init 2>&1 | tee -a "$LOG_FILE"; then
105+
return 0
106+
fi
107+
# init may fail because the repo already exists; retry the check.
108+
if restic snapshots --json >/dev/null 2>&1; then
109+
log "Repo already exists (init not needed)."
110+
return 0
111+
fi
112+
log "ERROR: cannot access or initialize repo: $repo"
113+
return 1
114+
}
115+
116+
# ---- Helper: backup + forget for a configured repo ----
117+
run_backup() {
118+
local repo="$1"
119+
local tag="$2"
120+
export RESTIC_REPOSITORY="$repo"
121+
log "Backing up to $repo (tag=$tag)..."
122+
restic backup --verbose --tag "$tag" "${EXCLUDES[@]}" "${BACKUP_PATHS[@]}" 2>&1 | tee -a "$LOG_FILE"
123+
log "Pruning old snapshots..."
124+
restic forget \
125+
--keep-daily "$KEEP_DAILY" \
126+
--keep-weekly "$KEEP_WEEKLY" \
127+
--keep-monthly "$KEEP_MONTHLY" \
128+
--prune 2>&1 | tee -a "$LOG_FILE"
129+
}
130+
131+
# ---- Google Drive backup ----
132+
if ensure_repo "$RESTIC_REPO_GDRIVE"; then
133+
run_backup "$RESTIC_REPO_GDRIVE" "scheduled"
134+
log "Clearing any stale gdrive locks..."
135+
restic unlock --remove-all 2>&1 | tee -a "$LOG_FILE" || true
136+
else
137+
log "Skipping gdrive backup; repo unavailable."
138+
fi
139+
140+
# ---- NAS backup (skip cleanly if not mounted) ----
141+
if mountpoint -q "${NAS_MOUNT}" 2>/dev/null || [ -d "${NAS_MOUNT}" ]; then
142+
if ensure_repo "$RESTIC_REPO_NAS"; then
143+
run_backup "$RESTIC_REPO_NAS" "scheduled-nas"
144+
restic unlock --remove-all 2>&1 | tee -a "$LOG_FILE" || true
145+
else
146+
log "Skipping NAS backup; repo unavailable."
147+
fi
148+
else
149+
log "NAS not mounted at ${NAS_MOUNT}, skipping NAS backup."
150+
fi
151+
152+
# ---- Summary ----
153+
export RESTIC_REPOSITORY="$RESTIC_REPO_GDRIVE"
154+
SNAPSHOT_COUNT=$(restic snapshots --json 2>/dev/null | python3 -c "import sys,json; print(len(json.load(sys.stdin)))" 2>/dev/null || echo "?")
155+
log "Done. Total snapshots (gdrive): $SNAPSHOT_COUNT"
156+
log "Log: $LOG_FILE"

0 commit comments

Comments
 (0)