Skip to content

feat(vfs): thread-safe config registry for per-database VFS configuration#1157

Open
corylanou wants to merge 2 commits intomainfrom
issue-1150-feat-vfs-support-thread-safe-configuration-via-uri-parameters-and-pragmas-no-setenv-required
Open

feat(vfs): thread-safe config registry for per-database VFS configuration#1157
corylanou wants to merge 2 commits intomainfrom
issue-1150-feat-vfs-support-thread-safe-configuration-via-uri-parameters-and-pragmas-no-setenv-required

Conversation

@corylanou
Copy link
Collaborator

@corylanou corylanou commented Feb 23, 2026

Description

Add a thread-safe Go config registry for the VFS extension, eliminating the need for setenv() which is not thread-safe on Linux. Library users who don't know VFS config values at process startup can now safely configure each database connection at runtime via the registry API without calling os.Setenv.

Key changes:

  • Config registry (vfs_config.go): SetVFSConfig/GetVFSConfig/DeleteVFSConfig keyed by database name, protected by sync.RWMutex with defensive copies on set/get to prevent concurrent mutation
  • Per-connection clients: openMainDB() checks registry and creates a per-connection ReplicaClient when config specifies replica_url; cleaned up on VFSFile.Close() and on Open() failure
  • Optional env vars: LitestreamVFSRegister() no longer requires LITESTREAM_REPLICA_URL, enabling per-database config via registry
  • C export: GoLitestreamConfigure() for Python/C callers to set per-database config before opening VFS connections
  • Nil client guard: openMainDB() returns a clear error if no replica client is available from either the VFS default or per-database config

Motivation and Context

setenv() is not thread-safe on Linux. Library users who don't know VFS config values at process startup cannot safely call os.Setenv later. The current VFS extension requires all config via env vars set before registration. URI query parameters are not viable without upstream changes to psanford/sqlite3vfs (follow-up issue).

Fixes #1150

How Has This Been Tested?

  • go test -tags vfs -race — 7 new tests pass with race detector:
    • TestVFSConfig_SetGet — basic set/get/delete
    • TestVFSConfig_ConcurrentAccess — 100 goroutines, race detector
    • TestVFSConfig_OverridesDefaults — config values override VFS defaults
    • TestVFSConfig_NilOptionalFields — nil field handling
    • TestVFSConfig_CopyOnSetAndGet — defensive copies prevent shared mutation
    • TestVFSConfig_PerConnectionOverrides — per-db config applied in openMainDB()
    • TestVFS_NilClientReturnsError — clear error when no client configured
  • go build -tags "SQLITE3VFS_LOADABLE_EXT vfs" ./cmd/litestream-vfs — loadable extension builds
  • All existing VFS tests continue to pass (only pre-existing TestVFSFile_PendingIndexRace race on main)

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist

  • My code follows the code style of this project (go fmt, go vet)
  • I have tested my changes (go test ./...)
  • I have updated the documentation accordingly (if needed)

Copy link
Owner

@benbjohnson benbjohnson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm okay with setting parameters in the URL but I think the PRAGMA part is overkill. These should all be set at initialization time and PRAGMAs can be used anytime.

@corylanou corylanou changed the title feat(vfs): thread-safe config registry + extended PRAGMAs feat(vfs): thread-safe config registry for per-database VFS configuration Feb 25, 2026
@corylanou corylanou force-pushed the issue-1150-feat-vfs-support-thread-safe-configuration-via-uri-parameters-and-pragmas-no-setenv-required branch from 661885e to 6929314 Compare March 3, 2026 20:18
@github-actions
Copy link

github-actions bot commented Mar 3, 2026

PR Build Metrics

All clear — no issues detected

Check Status Summary
Binary size 35.90 MB (0.0 KB / 0.00%)
Dependencies No changes
Vulnerabilities None detected
Go toolchain 1.25.8 (latest)
Module graph 1204 edges (0)

Binary Size

Size Change
Base (bd2c6e6) 35.90 MB
PR (85f55ab) 35.90 MB 0.0 KB (0.00%)

Dependency Changes

No dependency changes.

govulncheck Output

No vulnerabilities found.

Build Info

Metric Value
Build time 42s
Go version go1.25.8
Commit 85f55ab

History (1 previous)

Commit Updated Status Summary
e9ad8b7 2026-03-11 01:02 UTC 35.82 MB (0.0 KB / 0.00%)

🤖 Updated on each push.

Replace the requirement for setenv() (not thread-safe on Linux) with a
Go config registry that allows per-database VFS configuration at
runtime. Library users who don't know config values at process startup
can now safely configure each database connection without calling
os.Setenv.

Key changes:
- Add VFSConfig registry (SetVFSConfig/GetVFSConfig/DeleteVFSConfig)
  keyed by database name with RWMutex protection
- openMainDB() checks registry and creates per-connection ReplicaClient
  when config specifies a replica_url
- Per-connection clients are cleaned up on VFSFile.Close()
- LitestreamVFSRegister() no longer requires LITESTREAM_REPLICA_URL env
  var, enabling per-database config via registry
- Add GoLitestreamConfigure C export for Python/C callers
- Add PRAGMAs: litestream_poll_interval (R/W), litestream_cache_size
  (R/W), litestream_hydration_enabled (R/O), litestream_log_level (R/W),
  litestream_replica_url (R/O)
- Protect PollInterval/CacheSize PRAGMA access with mutex

Fixes #1150
Address Ben's review feedback: PRAGMAs are overkill for config that
should be set at initialization time. Also fix issues found during
code review.

Changes:
- Remove all new PRAGMAs (poll_interval, cache_size, hydration_enabled,
  log_level, replica_url)
- Return defensive copies from SetVFSConfig/GetVFSConfig to prevent
  concurrent mutation of shared config
- Add nil client check in openMainDB to fail early with clear error
- Close per-connection client on f.Open() failure to prevent leaks
- Revert unnecessary mutex addition in monitorReplicaClient
- Add TestVFSConfig_CopyOnSetAndGet and TestVFS_NilClientReturnsError
@corylanou corylanou force-pushed the issue-1150-feat-vfs-support-thread-safe-configuration-via-uri-parameters-and-pragmas-no-setenv-required branch from 6929314 to 5f029df Compare March 11, 2026 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(vfs): support thread-safe configuration via URI parameters and PRAGMAs (no setenv required)

2 participants