Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ postgresql-partition-manager run all

- **Automatic provisioning** — create upcoming partitions ahead of time
- **Cleanup management** — delete or detach outdated partitions with configurable retention
- **Lifecycle hooks** — execute shell commands or SQL at each stage of partition cleanup
- **Configuration checking** — verify partitions match expected configuration
- **Multiple intervals** — daily, weekly, monthly, quarterly, and yearly partitioning
- **Flexible partition keys** — `date`, `timestamp`, `timestamptz`, and `uuid` (UUIDv7) columns
Expand Down
10 changes: 8 additions & 2 deletions cmd/run/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,10 @@ const (
InvalidDateExitCode = 7
)

var ErrUnsupportedPostgreSQLVersion = errors.New("unsupported PostgreSQL version")
var (
ErrUnsupportedPostgreSQLVersion = errors.New("unsupported PostgreSQL version")
dryRun bool
)

func RunCmd() *cobra.Command {
runCmd := &cobra.Command{
Expand All @@ -38,6 +41,9 @@ func RunCmd() *cobra.Command {
},
}

AllCmd.Flags().BoolVar(&dryRun, "dry-run", false, "Preview which hooks would be executed without actually running them")
CleanupCmd.Flags().BoolVar(&dryRun, "dry-run", false, "Preview which hooks would be executed without actually running them")

runCmd.AddCommand(AllCmd)
runCmd.AddCommand(CheckCmd)
runCmd.AddCommand(ProvisioningCmd)
Expand Down Expand Up @@ -135,7 +141,7 @@ func initCmd() *ppm.PPM {

log.Info("Work date", "work-date", workDate)

client := ppm.New(context.TODO(), *log, db, config.Partitions, workDate)
client := ppm.New(context.TODO(), *log, db, config.Partitions, workDate, config.ConnectionURL, config.Hooks, dryRun)

if err = client.CheckServerRequirements(); err != nil {
log.Error("Server is incompatible", "error", err)
Expand Down
16 changes: 14 additions & 2 deletions docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,15 @@ Perform partitions provisioning, cleanup, and check
**Usage:**

```
postgresql-partition-manager run all
postgresql-partition-manager run all [flags]
```

**Flags:**

| Flag | Shorthand | Default | Description |
|------|-----------|---------|-------------|
| --dry-run | | false | Preview which hooks would be executed without actually running them |

**Inherited Flags:**

| Flag | Shorthand | Default | Description |
Expand Down Expand Up @@ -93,9 +99,15 @@ Remove outdated partitions
**Usage:**

```
postgresql-partition-manager run cleanup
postgresql-partition-manager run cleanup [flags]
```

**Flags:**

| Flag | Shorthand | Default | Description |
|------|-----------|---------|-------------|
| --dry-run | | false | Preview which hooks would be executed without actually running them |

**Inherited Flags:**

| Flag | Shorthand | Default | Description |
Expand Down
235 changes: 235 additions & 0 deletions docs/development/hook-types.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
# Developing a Hook Type

This page is for contributors who want to add a new **hook type** (a "runner") to
PostgreSQL Partition Manager — for example an `s3` type that archives a partition
to object storage.

If you only want to *use* the existing hook types, see [Shell](../hooks/shell.md) and
[PostgreSQL](../hooks/postgresql.md) instead.

The hook engine lives in the `internal/infra/hook` package.

## Architecture

A hook type is described by three pieces of behavior, bundled in a `typeHandler`
(in `registry.go`):

| Field | Responsibility |
| ----------- | ----------------------------------------------------------- |
| `validate` | Check the raw `config` map at configuration-load time. |
| `resolve` | Render template variables and build a typed config. |
| `newRunner` | Build the runner that executes the hook. |

These are wired together by the `registry` map. Everything else is
type-agnostic and does **not** change when you add a type:

- **Orchestrator** — resolves, logs, executes, applies the `on_failure` policy, records metrics.
- **Executor** — adds timeout and retry/backoff around any runner.
- **RegistryRunner** — dispatches to the right runner based on the hook type.

The lifecycle of a single hook:

```text
HookEntry (raw config map)
│ validate ← at config load
│ resolve ← templates applied → typed config
ResolvedHook { Type, Config, ConnectionURL, PartitionContext, ... }
│ Executor (timeout + retry)
│ RegistryRunner → runner for Type
Runner.Run(ctx, hook) ← your runner does the work
```

## Interfaces you implement

```go
// RenderedConfig is your config after template variables are substituted.
// It only has to describe itself for structured logging.
type RenderedConfig interface {
LogAttrs() []any // key/value pairs, e.g. []any{"bucket", c.Bucket}
}

// Runner executes the hook. The context carries the timeout deadline.
type Runner interface {
Run(ctx context.Context, hook *ResolvedHook) error
}
```

At execution time your runner receives a `*ResolvedHook`. Type-assert
`hook.Config` to your concrete config type. `hook.ConnectionURL` carries the
database connection details if you need them.

!!! warning
`LogAttrs` output is emitted at debug level and in `--dry-run`. Only return
fields that are safe to log — never secrets.

## Step by step: an `s3` runner

Everything below goes in a single new file `s3_runner.go`, plus **one line** in
the `registry` map and **one** type constant.

### 1. Add the type constant

In `config.go`:

```go
const (
ShellType HookType = "shell"
PostgreSQLType HookType = "postgresql"
S3Type HookType = "s3" // new
)
```

The type-validity check in `HookEntry.Validate` is registry-driven, so
registering the handler (step 5) is what makes `type: s3` accepted — there is no
switch to update.

### 2. Define the rendered config and `LogAttrs`

```go
type S3Config struct {
Bucket string `mapstructure:"bucket"`
Key string `mapstructure:"key"`
}

var _ RenderedConfig = (*S3Config)(nil)

func (c *S3Config) LogAttrs() []any {
return []any{"bucket", c.Bucket, "key", c.Key}
}
```

### 3. Validate the raw config

Runs at config-load time, before any partition work. Use static, wrapped error
variables (mirror the existing `Err*` vars in `config.go`):

```go
var (
ErrS3ConfigRequired = errors.New("config section is required for s3 hooks")
ErrS3BucketRequired = errors.New("'bucket' is required in config for s3 hooks")
)

func validateS3Config(config map[string]interface{}) error {
if config == nil {
return ErrS3ConfigRequired
}
if _, ok := config["bucket"]; !ok {
return ErrS3BucketRequired
}
return nil
}
```

### 4. Resolve templates

Render every user-supplied string field with `Render(value, partition)` so
[template variables](../hooks/index.md#template-variables) such as `{{.Schema}}` and
`{{.Table}}` work:

```go
func resolveS3Config(config map[string]interface{}, partition PartitionContext) (RenderedConfig, error) {
cfg := &S3Config{}

if v, ok := config["bucket"]; ok {
rendered, err := Render(fmt.Sprintf("%v", v), partition)
if err != nil {
return nil, fmt.Errorf("rendering bucket: %w", err)
}
cfg.Bucket = rendered
}

if v, ok := config["key"]; ok {
rendered, err := Render(fmt.Sprintf("%v", v), partition)
if err != nil {
return nil, fmt.Errorf("rendering key: %w", err)
}
cfg.Key = rendered
}

return cfg, nil
}
```

### 5. Implement the runner

```go
var _ Runner = (*S3Runner)(nil)

type S3Runner struct {
logger slog.Logger
}

func NewS3Runner(logger slog.Logger) *S3Runner {
return &S3Runner{logger: logger}
}

func (r *S3Runner) Run(ctx context.Context, hook *ResolvedHook) error {
cfg, ok := hook.Config.(*S3Config)
if !ok {
return fmt.Errorf("s3 configuration is nil for hook %q", hook.Name)
}

r.logger.Debug("Executing s3 hook", "hook", hook.Name, "bucket", cfg.Bucket, "key", cfg.Key)

// ... perform the upload, honoring ctx for the timeout deadline ...

return nil
}
```

!!! tip
Honor `ctx` — it carries the per-hook timeout. Retry and backoff are handled
for you by the executor based on the hook's `retry` config, so do not add
your own retry loop.

### 6. Register the handler

This is the only edit outside your new file, in `registry.go`:

```go
var registry = map[HookType]typeHandler{
ShellType: { /* ... */ },
PostgreSQLType: { /* ... */ },
S3Type: {
validate: validateS3Config,
resolve: resolveS3Config,
newRunner: func(logger slog.Logger) Runner { return NewS3Runner(logger) },
},
}
```

That is all. The orchestrator, executor, dispatcher, logging, dry-run, metrics,
and config validation now support `type: s3`.

## Checklist

- [ ] `HookType` constant added in `config.go`
- [ ] Config struct implements `RenderedConfig` (`LogAttrs` + compile-time `var _`)
- [ ] `validateXxxConfig` with static, wrapped error variables
- [ ] `resolveXxxConfig` renders every templated field via `Render`
- [ ] Runner implements `Runner`, type-asserts `hook.Config`, and honors `ctx`
- [ ] Handler registered in the `registry` map
- [ ] Unit tests for the runner, resolve, and validate functions
- [ ] User documentation added under `docs/hooks/` and linked in `mkdocs.yml`
- [ ] `LogAttrs` exposes no secrets

## Testing conventions

Mirror the existing tests in the `internal/infra/hook` package:

- **Runner** — table-driven success / failure / nil-config cases (see
`shell_runner_test.go`). For runners with external dependencies, inject a seam
(like `PostgreSQLRunner`'s `ConnectorFunc`) so tests don't hit the network.
- **Validation** — assert the specific `Err*` sentinel with `errors.Is` (see
`config_test.go`).
- **Resolution** — assert template variables are substituted, and that an
unknown variable surfaces an error (templates use `missingkey=error`).

Run the suite before opening a pull request:

```bash
make test
make lint
```
49 changes: 49 additions & 0 deletions docs/development/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Development

Guides for contributors working on PostgreSQL Partition Manager itself, rather
than operating it.

If you want to *use* PPM, start with [Getting Started](../getting-started.md)
and [Configuration](../configuration.md).

## Getting set up

The repository [`CONTRIBUTING.md`](https://github.com/qonto/postgresql-partition-manager/blob/main/CONTRIBUTING.md)
covers the full local development workflow: building, running the test suites
(unit, Bats, Helm), linting, and the PostgreSQL/Kubernetes dev environments.

Quick reference:

```bash
make build # Build the PPM binary
make test # Run unit tests with coverage
make lint # Run golangci-lint
```

## Topics

| Guide | Description |
|-------|-------------|
| [Developing a Hook Type](hook-types.md) | Add a new hook runner (e.g. `s3`) to the hook engine |

## Documentation

This site is built with [MkDocs](https://www.mkdocs.org/) and the Material
theme. To preview changes locally:

```bash
pip install -r requirements-docs.txt
mkdocs serve
```

Build and validate (treats warnings as errors, matching CI):

```bash
mkdocs build --strict
```

!!! note
The [CLI Reference](../cli-reference.md) page is auto-generated from the
Cobra command tree with `make docs-generate`. Never edit it by hand, and
regenerate it in the same commit as any change to commands or flags in
`cmd/`.
Loading
Loading