Feature Request: Support meta-dir for directory-based database configurations

[RichardAtRiva]

Feature Request: Support meta-dir for directory-based database configurations

When using directory-based database discovery (dir: + pattern: + recursive: true), there is no way to store Litestream's metadata/state files on a separate volume from the application data.

Today, meta-path exists for individual path:-based database entries, but it doesn't apply to directory configurations. Litestream always creates -litestream directories alongside the discovered database files.

Use case

We run Litestream as a standalone ECS service that monitors SQLite databases on a shared EBS volume. The same volume is used by our application for reads and writes. Litestream's local state files (L0/L1 LTX files, WAL position tracking) are written to the same volume.

We experienced a bug in a prior Litestream version (v0.5.5) where phantom L0 files accumulated at ~209 MB/min, filling the shared disk and taking down both Litestream and our application. While this specific bug is fixed in v0.5.9, the underlying risk remains: any future bug or misconfiguration in Litestream's local state management could exhaust disk space on the application's data volume.

Since we use dir: with recursive: true to dynamically discover databases, we can't use meta-path (which requires knowing each database path ahead of time).

Proposed solution

Add a meta-dir option at the directory-database level that redirects all metadata to a parallel directory tree on a separate path:

dbs:
  - dir: /mnt/ebs
    pattern: "*.metadata"
    recursive: true
    meta-dir: /litestream-state
    replica:
      url: s3://my-bucket/backups

For a database discovered at /mnt/ebs/tenant-a/data.metadata, Litestream would store its metadata at /litestream-state/tenant-a/data.metadata-litestream/ instead of /mnt/ebs/tenant-a/data.metadata-litestream/.

The relative path from dir would be preserved under meta-dir, maintaining the same parallel tree structure that already works for replica paths.

Benefits

• Isolates Litestream state from application data, preventing disk-filling bugs from affecting the application
• Allows using ephemeral storage (container filesystem) for state, since it's reconstructable from S3
• No impact on existing behavior — meta-dir would be optional, defaulting to current behavior when omitted
• Consistent with how replica paths already mirror the directory structure

Environment

• Litestream v0.5.9
• ECS on EC2, standalone task with shared EBS volume
• Using dir: + pattern: "*.metadata" + recursive: true