From 94f3aea206bd71943e7702e789549d1fd73f22fa Mon Sep 17 00:00:00 2001 From: Boris Ilijic Date: Wed, 1 Jul 2026 14:01:38 +0200 Subject: [PATCH 1/2] Add ai's agent context files --- CLAUDE.md | 92 +++++++++++++++++++++++++++++++++++++++++++ x/CLAUDE.md | 111 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 203 insertions(+) create mode 100644 CLAUDE.md create mode 100644 x/CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..3d309d241 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,92 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Repository overview + +Percona Backup for MongoDB (PBM): a distributed backup/restore tool for MongoDB sharded clusters and replica sets. +It uses Go, vendored dependencies and shell for tests and build. + +Two production binaries: +- `pbm` — CLI that issues commands (`cmd/pbm`) +- `pbm-agent` — daemon that runs alongside every `mongod` and executes operations (`cmd/pbm-agent`) + +Plus `pbm-speed-test` and `pbm-agent-entrypoint` (used in containers). + +There is **no Go API**: external callers must use the `pbm` CLI. Exported Go types/funcs are not a stable surface — see `README.md` "API" section. + +## Common commands + +All builds use `-mod=vendor -tags gssapi` (Kerberos auth requires `krb5-devel` / `libkrb5-dev` at build time). + +```sh +make build # build pbm, pbm-agent, pbm-speed-test → ./bin/ +make build-pbm # just the CLI +make build-agent # just the agent +make install # go install equivalents +make build-race # with -race +make build-static # CGO_ENABLED=0, statically linked +make build-cover # with -cover (used by CI) +``` + +Tests: + +```sh +make test # runs e2e-tests/run-all (heavy; spins MongoDB) +go test ./... # unit tests +go test ./pbm/oplog/... -run TestX -v # single package / single test +MONGODB_VERSION=8.0 ./e2e-tests/run-sharded # quicker e2e subset (sharded only) +MONGODB_VERSION=8.0 ./e2e-tests/run-rs # quicker e2e subset (replset only) +MONGODB_VERSION=8.0 ./e2e-tests/start-cluster # bring up a docker-compose dev cluster +``` + +`MONGODB_VERSION` selects PSMDB version (default `8.0`). The e2e harness lives in `e2e-tests/docker/` and `e2e-tests/pkg/`. + +Lint: `golangci-lint run` — config in `.golangci.yml` (v2 schema, custom enabled set including `lll`, `nilnil`, `errname`, `gochecknoinits`, `nonamedreturns`). + +CI is GitHub Actions (`.github/workflows/ci.yml`) but the actual functional tests live in the external `Percona-QA/psmdb-testing` repo and run via pytest against a docker-compose stack — `make test` is the local equivalent. + +## Architecture + +### Coordination model + +PBM has no central coordinator process. Coordination is done **through MongoDB itself**: the agents on each node read/write "PBM Control collections" to elect leaders, distribute work, and report status. Locking, command queue, and op log capture all flow through these collections. When tracing a flow, expect to read code that polls or writes BSON documents rather than RPC. + +### Layered packages under `pbm/` + +`pbm/` holds the domain logic, organized by concern: + +- `ctrl/` — the command protocol (CmdBackup, CmdRestore, CmdPITR, …) and send/recv helpers. Start here when tracing how a CLI invocation reaches an agent. +- `connect/` — MongoDB client wrapper used everywhere +- `lock/` — distributed locks via the control collection +- `topo/` — cluster topology discovery (shards, replset members, primary detection) +- `config/` — PBM configuration (storage, PITR, etc.) persisted in MongoDB +- `backup/`, `restore/`, `oplog/`, `slicer/`, `resync/` — the operations themselves +- `storage/` — pluggable backup destinations: `s3/`, `gcs/`, `azure/`, `oci/` (Oracle Cloud), `oss/` (Alibaba), `fs/`, `mio/` (MinIO), `blackhole/` (testing). All implement the `storage.Storage` interface in `storage.go`. +- `compress/`, `archive/`, `snapshot/` — data pipeline for streaming backups +- `prio/`, `defs/`, `errors/`, `log/`, `version/`, `util/` — cross-cutting + +The `pbm-agent` binary wires these together; the `pbm` CLI uses `sdk/` (a thin in-process wrapper around the same packages) to inject commands. + +### Errors + +Use the project's `pbm/errors` package (it wraps `pkg/errors`); it also defines sentinel errors like `ErrNotFound` that callers compare against. Don't switch to stdlib `errors` ad hoc. + +## The `x/` directory — pbm-x (experimental) + +`x/` is a **separate Go module** (`github.com/percona/percona-backup-mongodb/x`) and is a proof-of-concept for a future major version of PBM. + +- Self-contained — **no code is shared between `pbm` and `pbm-x`** in either direction. Don't add cross-imports. +- **Not vendored** — `x/` builds in normal module mode (no `vendor/`, no `-mod=vendor`, no `gssapi` tag, no Makefile target). Build/test from inside `x/`: `cd x && go build ./...` (binary lands at `x/cmd/pbmx/pbmx`, which is gitignored) and `go test ./...`. Run `go mod tidy` after touching `x/go.mod`. + +**`x/` has its own `x/CLAUDE.md`** — read it before doing pbm-x work. It carries the prototype's design notes plus working preferences that override defaults here: follow the Google + Uber Go style guides, never auto-create commits/PRs, and never modify anything outside `x/` without asking first. + +If a task is scoped to "pbm-x", work only inside `x/`. If it's scoped to PBM proper, ignore `x/`. + +## Conventions worth knowing + +- Vendored deps under `vendor/` — run `go mod vendor` after touching `go.mod`. Builds will fail otherwise (the `-mod=vendor` flag is hardcoded in the Makefile). +- The `gssapi` build tag is mandatory for the production binaries; bare `go build ./...` works for unit tests but won't reproduce production behavior for auth code. +- Indent: tabs in Go (per `.editorconfig`), 2-space in YAML/JSON, 4-space elsewhere. +- Branch naming: include the Jira ticket, e.g. `PBM-1750-pbm-x-base-setup`. Commits and PR titles start with `PBM-XXXX. …`. +- Bug tracker: Percona Jira project `PBM` (https://jira.percona.com/projects/PBM), not GitHub Issues. diff --git a/x/CLAUDE.md b/x/CLAUDE.md new file mode 100644 index 000000000..0993682a1 --- /dev/null +++ b/x/CLAUDE.md @@ -0,0 +1,111 @@ +# CLAUDE.md — pbm-x + +Context for working inside `x/` (the experimental next-major-version prototype). The root `CLAUDE.md` covers the legacy `pbm/` codebase; this file is just for `x/`. + +**Hard rule:** `x/` is a separate Go module. Do not import from `github.com/percona/percona-backup-mongodb/...` (the legacy module) or vice versa. Reuse ideas, not code. + +--- + +## Vision + + + +_TBD_ + +## Scope of the current prototype + + + +- In scope: _TBD_ +- Out of scope (for now): _TBD_ +- Out of scope (forever / explicit non-goals): _TBD_ + +## Design principles + + + +_TBD_ + +## Architecture sketch + +Same overall shape as legacy PBM — an agent runs alongside each `mongod` — but agents now come in **two roles**: + +- **ctrl agent** — owns PBM's control-collection state (the leader-election / command-queue / status that legacy PBM kept inside MongoDB control collections). In pbm-x this state lives in an **etcd** database that the ctrl agent runs/manages. +- **worker agent** — does **not** run etcd. Its core responsibility is executing the actual work: backup and restore. + +``` + +-------------------+ control-collection state + | ctrl agent | runs (etcd) + +-------------------+ ^ + | etcd client API + | (local or remote) + +-------------------+ | + | worker agent | --etcd client--+ also does backup / restore + +-------------------+ +``` + +**Coordination:** worker agents connect to the ctrl agent's **etcd as clients** to read/write control state — they speak the etcd client API directly, not a separate protocol. A worker may be co-located with the ctrl agent or remote, so etcd's client endpoint must be externally reachable (we bind `0.0.0.0`). The peer API is for etcd↔etcd traffic between ctrl agents. + +Open: how many ctrl agents run (single vs. quorum) and etcd deployment topology — see Open questions. + +### Code layout + +A single `pbmx` binary covers both the CLI and the agent (selected at runtime by `--ctrl-agent` / `--worker-agent`). Two layers, with a one-way dependency: + +``` +cmd/pbmx/ → pbm/ + CLI only all runtime logic +``` + +- **`cmd/pbmx/`** — CLI wiring *only*: cobra command definitions, flags, env-var bindings, and config-file loading. It parses input and dispatches; it holds no operational logic. The root command's `RunE` reads the role flag and calls into `pbm/`. +- **`pbm/`** (`github.com/percona/percona-backup-mongodb/x/pbm`) — all other logic. Today: `RunCtrlAgent` + embedded-etcd startup/graceful-shutdown (`etcd.go`). Worker-agent, backup/restore, and coordination logic land here too. + +Rule: dependency flows one way, `cmd/pbmx/` → `pbm/`. `pbm/` must not import `cmd/pbmx/`. Anything beyond CLI parsing/dispatch belongs in `pbm/`. + +## What's intentionally different from legacy PBM + + + +- **Coordination / state store:** legacy PBM keeps control-collection state (locks, command queue, status) inside MongoDB itself. pbm-x moves that state into an **etcd** database, owned by the **ctrl agent**. +- **Agent roles split in two:** legacy PBM runs one uniform agent per node. pbm-x splits responsibilities into **ctrl agents** (manage control state in etcd) and **worker agents** (run backup/restore, no etcd). + +## Open questions / decisions to make + + + +- **etcd is currently plaintext + unauthenticated.** The client and peer APIs are served over `http` with no auth, and the client API is bound on `0.0.0.0` (remote workers legitimately need it — see Coordination). This cannot be closed by binding localhost; it's an **auth/TLS** task to tackle later (etcd RBAC and/or client/peer TLS), not a bind-address one. +- How many ctrl agents run — single node vs. a 3/5-member quorum — and the etcd deployment topology that follows from it. + +## Glossary + + + +- **ctrl agent** — agent role that owns and manages PBM's control-collection state, backed by an etcd database it runs. +- **worker agent** — agent role that performs backup and restore. Does not run etcd. +- **control-collection state** — in legacy PBM this lived in MongoDB collections; in pbm-x it lives in etcd (managed by the ctrl agent). + +## Working preferences for this prototype +- when generating golang code follow following guidelines: + - https://google.github.io/styleguide/go/ + - https://google.github.io/styleguide/go/guide + - https://google.github.io/styleguide/go/decisions + - https://google.github.io/styleguide/go/best-practices +- additionally you can expand with uber's style guide: + - https://github.com/uber-go/guide/blob/master/style.md +- git + - never create git commits or pr requests automatically. +- never modify anything out of x directory. If it's necessary ask for confirmation. + + From 5866e2bb6049dbe16757fda690a6ba3909833618 Mon Sep 17 00:00:00 2001 From: Boris Ilijic Date: Wed, 1 Jul 2026 14:02:36 +0200 Subject: [PATCH 2/2] Add link for agent context for AGENTS.md --- AGENTS.md | 1 + x/AGENTS.md | 1 + 2 files changed, 2 insertions(+) create mode 120000 AGENTS.md create mode 120000 x/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 000000000..681311eb9 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +CLAUDE.md \ No newline at end of file diff --git a/x/AGENTS.md b/x/AGENTS.md new file mode 120000 index 000000000..681311eb9 --- /dev/null +++ b/x/AGENTS.md @@ -0,0 +1 @@ +CLAUDE.md \ No newline at end of file