A Linux desktop app that syncs Proton Drive into a local folder. Read SPEC.md first — it is the source of truth for features, architecture, and milestones. macOS and Windows support was dropped in v0.2; do not reintroduce platform-conditional code without asking.
- Host UI shell: MyGo (Go) —
mygo-shell/is the production Linux dashboard (Overview/Activity/Conflicts/Settings, tray, autostart, single instance) - Sync engine: Go —
engine-go/(pds-engine-go) runs the sync engine (SQLite state, watcher, queues, FUSE VFS) and exposes a JSON-line Unix socket so the MyGo shell can control it - Drive SDK sidecar: Node.js + official
ProtonDriveApps/sdk(TypeScript client module), spawned as a child process, JSON-RPC over stdio - VFS: FUSE via
go-fuse(engine-go/internal/vfs/) - State: SQLite
The Rust workspace (crates/ + Cargo.toml) was removed after the Go port reached parity (Phase 2, done 2026-11-08) — nothing in the tree references it anymore. Its concepts survive only as history in the milestone log below.
- Conservative deletes. Remote deletes are always Trash (never permanent), only for nodes previously confirmed synced in SQLite, and subject to the mass-deletion guard (SPEC.md F3). An empty local folder must never wipe the Drive.
- Proton compliance. Event-based sync only (no polling, no recursive tree scans). Set an honest
x-pm-appversionheader in theexternal-drive-{name}@{semver}-{channel}format. No Proton branding. Login UI must show: "This is a third-party application not officially supported by Proton." - All Drive API traffic goes through the Node sidecar. Never call Proton endpoints from Go directly — the sidecar isolates SDK churn and the upcoming 2026/27 crypto migration.
- Keep-both on conflicts — never silently overwrite in either direction.
- Never evict pinned, dirty (un-uploaded), or open files from the local cache.
- Product specification written (
SPEC.md, v0.2 Linux-only) - M0 — Spike: PASSED 2026-10-23 against a real account — fork auth, key unlock, SDK bootstrap, list, upload, verified download, trash, event cursor (FastForward). Go decision confirmed.
- M1 — Headless sync engine: PASSED live E2E 2026-10-24 (scratch account) — F1–F4, F6, F7 all verified against real Proton Drive. 54/54 Rust tests, sidecar smoke + event-loop tests green. E2E bugs found & fixed: telemetry-on-stdout, F7 stale-base race (server 409 → parked → event-path keep-both), guard cascade, CLI per-root resolution, event-stream polling (sidecar now polls /events/{cursor} every ~30s).
- M2 — GPUI shell: done 2026-10-24 (tray app). Rewritten 2026-10-29 as a single-window Linux desktop app (sidebar + Overview/Activity/Conflicts/Settings, login screen, sign-out modal) when macOS support was dropped;
gpui-trayremoved. - M3.1 — FUSE placeholders: done 2026-10-29 —
pds mountwith on-demand hydration; engine placeholder mode (placeholders: true→ VFS store, backing dir.pds-vfs-backing/, no materialization); real-kernel E2E in the VM (3 MB file read through the mount). - M3.2 — FUSE write-through: done 2026-10-29 —
create→upload on close,mkdir/unlink/rmdirmirror remotely; store write-queues drained by the engine main loop (reuses F7/retry/pause machinery); write overlay +reconcile_overlaykeep pre-upload files visible without double-listing; snapshot-backed files stay RO (revision writes = M3.3). E2E covers create→upload→read-back→mkdir→nested upload→unlink→trash. Pins + LRU eviction next (same milestone). - M3.2 — FUSE write-through + pins + LRU eviction: done 2026-10-29 — write-through (see previous entry);
pds pin/unpin <folder> <path>via the control socket (folders recursive, immediate hydration); LRU eviction (EngineConfig::eviction_cap, default 10 GiB inpds mount) on the engine idle path — skips pinned/dirty/open/just-hydrated files, publishes the snapshot so placeholders reappear. Tests: store LRU/open-handle unit tests + two engine eviction tests + full FUSE E2E. Revision writes = M3.3. - M3.3 — FUSE revision writes: done 2026-10-29 — snapshot-backed files open RW (open waits bounded for hydration, fd on the backing file);
release/fsyncupload the revision preservingnode_uid(F7check_remote_headstill guards stale heads);O_TRUNC=setattr(size, fh)(fuser never negotiatesATOMIC_O_TRUNC) → marks the handle dirty, upload deferred toreleasesoopen(O_TRUNC)→write→close is ONE upload, no empty intermediate; baretruncate(2)queues viarecord_write. Prerequisite bug fixed: the watcher/initial scan did not exclude.pds-vfs-backing/, so hydration/eviction/revision events re-entered the pipeline as phantom uploads/deletes —is_internal_path()now guards all three filter sites (+ regression tests). Revision-handle bookkeeping: open-handle tracked (evict-protected),write/fsync/releaseuse the handle's authoritativerel. E2E covers revision write (head bytes replaced, tail preserved, same node, one upload),fs::writeO_TRUNC rewrite, and full M3.2 flow. - M3.4 — Desktop placeholder integration: done 2026-10-30 — the app defaults NEW sync roots to placeholder mode and mounts the FUSE view at the sync root (
~/ProtonDriveis the placeholder view). Key decisions: (1) all engine state (SQLite DB, VFS backing cache,rootmarker) lives under$XDG_DATA_HOME/proton-drive-sync/roots/<hash>/— OUTSIDE the root — because mounting over the root shadows everything under it and running the DB through FUSE would risk corruption;crates/core/src/mode.rsowns these paths (engine_state_dir,root_key). (2) The app probes FUSE with a throwaway temp-dir mount BEFORE building the engine — the engine consumes the sidecar's single notification stream, so a post-hoc mirror fallback would be deaf to remote events; probe failure downgrades the session to mirror (MountUnavailablenotice in Settings). (3) The real mount goes up only afterwait_for_watcher— the engine's first-run local scan and the layout conversion must see the real folder (the mount serves an empty tree until the first snapshot publish, which would hide every pre-existing local file). (4) Mode is a persisted per-root setting (meta.sync_mode,$PDS_SYNC_MODEoverride); Settings toggle → confirm modal → persist → restart the engine thread. Legacy in-root state DBs are adopted into the external dir (copy, original removed after success) and default to mirror so upgrades change nothing. (5)convert_layout(engine, runs inEngine::newwhenbacking_diris external): mirror→placeholders moves tracked files into the backing cache (hydrated, no re-download) AND adopts untracked leftovers as dirty pending-uploads (nothing stranded under the mount); placeholders→mirror moves hydrated content back; idempotent via a DIRECTION-AWARE.layout-convertedmarker (records which layout it produced — a ran-once marker would strand content on the second toggle), never deletes. (6) Placeholder-mode bug fixed while tracing content paths: F7 conflict renames and remote-name-collision renames wrote into the VISIBLE root instead of the backing dir — all content paths now flow throughcontent_dir(). (7) Self-review hardening:$PDS_SYNC_MODEis session-only (never rewrites the persistedsync_mode); legacy in-root DBs are removed only after the adopted copy verifies as a readable SQLite DB; the app's mount gets an EMPTY.pdsproxy dir (the fs's state-dir proxy would otherwise serve the live DB + backing cache inside the user-facing mount); the engine never unlinks visible paths in placeholder mode when the root is the mount (an unlink would round-trip through FUSE into a spurious store delete — placeholders vanish via snapshot refresh instead); e2e regression proves mounting over a non-empty root shadows leftovers and preserves every byte. Tests: mode unit tests (incl. env-session-only),convert_layoutround-trips,Engine::newmode-resolution, external-backing adoption integration test, non-empty-root FUSE e2e; fulldevenv testgreen (Rust + sidecar) plus headless GUI smoke. - M3.5 — Configurable cache cap: done 2026-10-30 — the placeholder backing-cache cap (until now hardwired to
EVICTION_CAP_DEFAULT, 10 GiB) is user-configurable on three surfaces with precedencepds mount --cache-cap <SIZE>>$PDS_CACHE_CAP(session-only, never rewrites the persisted value — same rule asPDS_SYNC_MODE) > the persisted per-root Settings choice > 10 GiB default.unlimited(no eviction) is expressible on all three. Sizes parse as bytes with an optional binaryK/M/G/Tsuffix (parse_cache_cap;0/none/off/unlimited→ unlimited; invalid values are ignored with a warning, never silently disabling eviction). Persistence: meta keycache_cap_bytes(Unlimited stored as the literalunlimited; unset key = never configured → default). Settings applies live:Command::SetEvictionCap(Option<u64>)mutates the running engine's cap and runs an immediatemaybe_evictpass when the new cap is already exceeded (no restart, unlike the mode toggle);EngineStatus.cache_capfeeds the Settings chips (2/5/10/25 GB + Unlimited on the Sync mode card, with a$PDS_CACHE_CAP-override notice). App engine build resolves env > persisted > default;pds sync/pds mountresolve the same way (mount additionally honoring the flag). Tests: parse/precedence/persistence unit tests, liveset_cache_capintegration test (immediate eviction on shrink, no eviction after re-widening to unlimited); fulldevenv testgreen incl. real-kernel FUSE e2e. - M4 — Hardening: done 2026-10-30 — (1) crash recovery: rows stuck
Uploadingfrom a previous run reset to Dirty at startup (re-uploaded through the full F7/retry machinery; vanished files tombstone pre-approved so the guard never trips on recovery); stale*.pds-parttemps swept from both content roots. (2) Rate limits:transfer.errornow carriesdata.retryAfter(sidecar + Rust decode) and the engine honors the server hint over the computed backoff;retry_backoffunit-tested. (3) Soak battery (tests/soak.rs): repeated simultaneous edits, remote-delete-vs-edit, remote burst convergence, conflict-copy churn — found a real F7 hole:apply_remote_deletetrustedsync_statealone, so a local write inside the watcher debounce window was DESTROYED by a concurrent remote delete; now hashes on-disk content first (mirrorsapply_node_upsert). (4) Test-flake root fix:EngineHandle::wait_for_watcher— fixtures synchronized on initial-sync completion, which can precede watcher attach, losing inotify events for immediate writes (the "rotating flake" across engine tests). (5) SDK rehearsal documented (docs/sdk-upgrade-rehearsal.md); current stack verified green (SDK 0.23.0, crypto 2.2.0 + openpgp patch, Node 22). - M5 — Background service, login start, tray: done 2026-10-31
- MyGo migration Option B: done 2026-11-05 —
crates/engine-headless/is promoted to the production background sync service;mygo-shell/is a full MyGo-native UI replacement that replicates the GPUI Overview/Activity/Conflicts/Settings surface, tray, XDG autostart, single-instance lock, and--minimizedstart. The engine process loops across sign-out and sync-mode switches instead of exiting. The existing GPUI app, CLI, FUSE, and sidecar are untouched. Seedocs/research/mygo-migration-poc.md. - MyGo migration Phase 2 (engine → Go): done 2026-11-08 — plan in
docs/research/go-engine-migration.md; FUSE-on-Go validated with a real kernel mount (go-fuse v2.12.1);engine-go/speaks the socket protocol (pds-engine-gobinary) and the Go engine has reached parity with the Rust one. The Rust workspace (crates/,Cargo.toml,Cargo.lock,rust-toolchain.toml) has been removed: the tree is the Go engine + MyGo shell + Node sidecar, anddevenv test(Go engine + sidecar) is green with zero cargo references left (the handful of remaining mentions are historical milestone notes below).
The host Mac is not a build target anymore — the MyGo shell and Go engine target Linux only. Build and test inside the Ubuntu 24.04 arm64 VM:
ssh pds-linux@orb
cd /Users/christopher/Developer/proton-drive-sync # repo is shared into the VM
devenv shell # same env as CI (devenv.nix/lock)
(cd engine-go && go vet ./... && go test ./... -short)
(cd mygo-shell && go vet ./... && go test ./...)Note: the MyGo shell module needs Go ≥ 1.27.3 (its go.mod declares it, via the mygo dependency), while devenv ships Go 1.26.10 — so go build/test in mygo-shell/ currently pulls the 1.27.3 toolchain via GOTOOLCHAIN (needs network on first run). engine-go declares go 1.26.0 and builds with the local toolchain (GOTOOLCHAIN=local). Bump devenv.nix to go_1_27 (1.27.3 is in the pinned nixpkgs) to drop the download entirely.
The MyGo shell dlopens libglib-2.0.so.0 directly, so glib must be listed explicitly in the Nix package's runtimeLibs and in the development environments. GTK's transitive dependency on GLib does not put it on LD_LIBRARY_PATH; omitting it makes the installed app exit before creating a window. Tray creation additionally dlopens libayatana-appindicator3.so.1; keep libayatana-appindicator in those same runtime library sets or mygo.NewTray returns ErrUnsupported and the app runs without an icon.
devenv test runs the Go engine tests + sidecar typecheck/tests in the pinned environment; install Nix/devenv in the VM once, then it matches
CI exactly. (MyGo shell tests are run separately, as above — the shell is not part of devenv test.)
Notes:
- The repo sits on a virtiofs share in the VM: inotify events are unreliable there, so the fsnotify-based engine tests can flake under load (
go test ./...inengine-go/). They pass on VM-native disk (copy the tree to~/pds-src) and on macOS. - GUI smoke test without a desktop: run the shell under a headless Wayland compositor and screenshot it with grim:
Rendering works with Mesa lavapipe (
mkdir -p /tmp/xdg && chmod 700 /tmp/xdg WLR_BACKENDS=headless WLR_LIBINPUT_NO_DEVICES=1 WLR_RENDERER=pixman \ XDG_RUNTIME_DIR=/tmp/xdg sway -c /dev/null & (cd mygo-shell && go build -o proton-drive-sync .) # Go >= 1.27.3 toolchain via GOTOOLCHAIN (cd engine-go && go build -o pds-engine-go ./cmd/pds-engine-go) WAYLAND_DISPLAY=wayland-1 PDS_SYNC_ROOT=/tmp/root mygo-shell/proton-drive-sync & sleep 15 && WAYLAND_DISPLAY=wayland-1 grim shot.png
VK_ICD_FILENAMESunset — pointing it at a wrong filename disables software Vulkan).
- Tests must synchronize on the watcher, not initial sync. A fixture that returns when initial-sync materializes files can beat the watcher attach (step 3 of
run()); any file written immediately after is invisible to inotify. That was the "rotating flake" hitting a different engine test each run under VM load. Fix:EngineHandle::wait_for_watcher(Arc flipped right afterstart_watcher). apply_remote_deletecannot trustsync_state— a local write inside the 500 ms debounce window leaves the rowSyncedwhile the disk content is already new. Hash before deleting (same discipline asapply_node_upsert). Found by the soak battery.- Attempt counting races failure bookkeeping:
upload_attemptsincrements when the transfer STARTS; the backoff is scheduled when thetransfer.erroris processed. Tests that act between the two (e.g. resume clearing backoffs) must wait for the row to come back Dirty first. - Mass-delete guard tests must let the deletes land first. The watcher's ~500 ms debounce can split a burst of removals into several batches under CI load; a guard-paused engine keeps recording tombstones but drains nothing, so a resume issued after the first batch approves only part of the set and the late ones legitimately re-trip the guard. Synchronize on "every delete accounted for" (
wait_for_deletes_accounted: pending tombstones + trashed == total) before resuming — CI flake, 2026-10-31.
- A
Filesystemmethod that never replies wedges the kernel caller (not just that request):read_dirhangs with no error, no panic. Every path must consumereplyexactly once. - A reply with the wrong data can fail silently at the mount level. The M3.1 bug:
Tree::node("")returnedNonefor the root (""is not inby_path), so rootgetattranswered ENOENT; the kernel then failed allread_diron the mountpoint with ENOENT whilemount | grepstill showed the fs present andinit/getattrlogs looked healthy. Test the root-inode path explicitly. - fuser 0.18 API: trait methods take
&self(not&mut self);readdiroffset isu64(1 = "after entry 0");ReplyDirectory::add(ino, offset, kind, name)returnstruewhen the buffer is full;ReplyStatfs::statfstakes 8 args (blocks, bfree, bavail, files, ffree, bsize, namelen, frsize); errors arefuser::Errno::ENOENT-style constants;Configis#[non_exhaustive]— mutatecfg.mount_optionsafterConfig::default(), no struct literal. editions = 2024:genis a reserved keyword — don't name struct fields or localsgen.- Panic in a fuser session thread is swallowed (session thread dies, callers get ENOENT/EIO). Wrap suspect methods with
std::panic::catch_unwind+eprintlnwhen debugging; a custom panic hook also works.
- Inodes the kernel holds must keep resolving. After
createreplies with an overlay inode, the kernel caches that dentry (attr TTL). If the engine's snapshot then covers the path and the overlay drops its inode mapping, a lateropen(overlay_ino)fails ENOENT. Fix: overlay keeps aino → relalias (retire, notremove) so cached dentries resolve; only unlink drops the alias. writetrait signature (fuser 0.18) takes 10 params:_req, ino, fh, offset, data, WriteFlags, OpenFlags, Option<LockOwner>, reply— easy to under-declare; the compiler error names the trait signature.- RW mounts need the flag dropped at the kernel level.
MountOption::ROblocksopen(O_CREAT)before the fs code runs (fs-level EROFS is never asked). Mount RW and enforce per-file RO inopen/setattrinstead. - The test's tracing filter ignores RUST_LOG when
with_env_filter("literal")is hardcoded — readRUST_LOGwith a fallback or VFS debug logs never show up incargo test -- --nocapture. - No
FUSE_ATOMIC_O_TRUNC(M3.3). fuser'sINIT_FLAGSis justASYNC_READ | BIG_WRITES, soopen(O_TRUNC)never truncates atomically — the kernel follows up with a separatesetattr(size, fh)afteropen. Truncate handling must therefore coversetattrwith a passed fh, and defer the upload toreleaseoropen(O_TRUNC)→write→close double-uploads (an empty intermediate revision first). - Mock default tree (M3.3 E2E).
MockSidecar::new()already seedshello.txt;remote_create_file("root", "hello.txt", …)in a test creates a DUPLICATE node, andfind_uidmay resolve to the one the engine never touched. Checkchild_namesfor surprises before asserting onfile_content.
listxattrmust choose the reply form fromsize.ReplyXattr::sizeis only valid for the size query (size == 0). Answering a request that supplied a buffer with the size form makes the kernel fail the caller with EIO — symptom:ls -lerrors on every entry while plainls(names only) works.- Create the sync root before canonicalizing it.
prepare_layoutcanonicalized~/ProtonDrivefirst, so first launch on a fresh machine failed before the engine started.create_dir_allprecedes thecanonicalizenow.
(The Session::new-without-spawn() freeze is covered by the upstream
fix(app): serve the placeholder FUSE session commit and its probe test.)
- Every PR must include screenshot/video proof. Before opening or marking a PR ready, attach visual proof of the change: screenshots for UI changes, a screen recording for flows/interactions, or terminal output for CLI/engine work. If the change isn't visually verifiable, say so explicitly in the PR description.
- Auto-approve subagent permission prompts (user's directive, 2026-10-23): when Herdr subagents block on permission dialogs, approve them without asking, as long as the request is research/build-scoped and non-destructive.
- Research reports live in
docs/research/— consult them before touching auth or SDK code.
@protontech/cryptoships raw TypeScript → always run viatsx, tsconfig uses Bundler resolution +allowImportingTsExtensions.Uint8Arraybase64/hex polyfill must be the first import (src/polyfill.ts); types live insrc/types/uint8array-encoding.d.ts.openpgp/lightweighthas no Node export → patched viapatch-package(seesidecar/patches/), applied bypostinstall.- SDK deep imports need the
dist/prefix:@protontech/drive-sdk/dist/cache. - Never run the spike repeatedly with wrong credentials —
/auth/*is the most rate-limited/CAPTCHA-prone path in the API. - Password-SRP
POST /auth/v4is rejected forexternal-drive-*clients (422 code 2028 "This version of the app is no longer supported", observed 2026-10-23). The browser session-fork flow (src/fork.ts, same as Proton's own CLI and pdcli) is the primary auth path: no password handling, no CAPTCHA risk, key password arrives in the fork payload. SRP code insrc/auth.tsis kept for reference behindPDS_AUTH=srp. - Fork sign-in URL format:
https://account.proton.me/desktop/login?app=drive&pv=3#payload=0:{UserCode}:{base64-AES-key}:external-drive; pollGET /auth/v4/sessions/forks/{selector}(422 = not ready).
proton-drive-sync/
├── SPEC.md # Product spec — source of truth
├── AGENT.md # This file
├── mygo-shell/ # MyGo-native production UI shell (Go 1.27+ toolchain, see above)
├── engine-go/ # Go sync engine (pds-engine-go; SQLite, FUSE VFS via go-fuse)
├── sidecar/ # Node.js + official Proton Drive SDK
└── packaging/linux/ # .desktop entry, icon, install script
(The pre-Go-port Rust workspace crates/ was removed in Phase 2, 2026-11-08.)
- Dev environment:
devenv.nix(pinned bydevenv.lock) is the source of truth for toolchains and native deps; CI runsdevenv test. Add new libraries there, not as ad-hoc apt/brew installs. - Go:
go vet ./...clean,gofmt-clean (bothengine-go/andmygo-shell/). - Sidecar: TypeScript, strict mode, ESM. SDK version pinned exactly in
package.json. - Commit messages: conventional commits (
feat:,fix:,chore:…) — they drive the changelog (cliff.toml), so keep scopes meaningful. - Releases: single product version (
versioninsidecar/package.json);scripts/release.shmirrors it to the sidecarAPP_VERSIONand the Nix derivations. Cut releases only viascripts/release.sh <major|minor|patch>— it bumps every version site, regeneratesCHANGELOG.md(git-cliff), and opens arelease/vX.Y.ZPR.mainis protected: merge it by PR, never push the bump directly. After the merge,.github/workflows/tag-release.ymlreads the new version, tagsvX.Y.Zand dispatches.github/workflows/release.yml(tarball + deb + rpm); tags pushed by CI do not firepush: tags, hence the explicit dispatch.scripts/check-versions.shruns in CI to catch drift.actionlint+shellcheckare in the devenv — run them after touching workflows or release scripts. Release smoke checks must test each binary with its owntest -x; passing both paths to onetest -xis invalid. - When a milestone is completed, check it off above and note any spec deviations in
SPEC.md§9.