Skip to content

Commit 4f40e6c

Browse files
busserclaude
andauthored
docs: comprehensive documentation improvements (#493)
- Add Go library usage section with examples and use cases - Add error handling and troubleshooting guide - Enhance godoc comments for public API (Provider, ResolveAll) - Add GCP Secret Manager examples and authentication guidance - Standardize provider documentation format across all providers - Fix outdated path reference in CLAUDE.md (internal -> pkg) - Fix typo in terraform/README.md (Infrastructure) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7f386b4 commit 4f40e6c

File tree

5 files changed

+246
-13
lines changed

5 files changed

+246
-13
lines changed

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
1313

1414
### Testing
1515
- Run specific test: `go test ./path/to/package -run TestName`
16-
- Run e2e tests for specific provider: `go test -tags=e2e ./internal/murmur/providers/awssm`
16+
- Run e2e tests for specific provider: `go test -tags=e2e ./pkg/murmur/providers/awssm`
1717

1818
## Architecture
1919

README.md

Lines changed: 188 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,15 @@ issue so that we can track demand for it._
3333
- [Adding Murmur to a container image](#adding-murmur-to-a-container-image)
3434
- [Adding Murmur to a Kubernetes pod](#adding-murmur-to-a-kubernetes-pod)
3535
- [Parsing JSON secrets](#parsing-json-secrets)
36+
- [Go library usage](#go-library-usage)
3637
- [Providers and filters](#providers-and-filters)
3738
- [`scwsm` provider: Scaleway Secret Manager](#scwsm-provider-scaleway-secret-manager)
3839
- [`awssm` provider: AWS Secrets Manager](#awssm-provider-aws-secrets-manager)
3940
- [`azkv` provider: Azure Key Vault](#azkv-provider-azure-key-vault)
4041
- [`gcpsm` provider: GCP Secret Manager](#gcpsm-provider-gcp-secret-manager)
4142
- [`passthrough` provider: no-op](#passthrough-provider-no-op)
4243
- [`jsonpath` filter: JSON parsing and templating](#jsonpath-filter-json-parsing-and-templating)
44+
- [Error handling and troubleshooting](#error-handling-and-troubleshooting)
4345
- [Changes from v0.4 to v0.5](#changes-from-v04-to-v05)
4446

4547
## Fetching a database password
@@ -159,6 +161,99 @@ Murmur uses the Kubernetes JSONPath syntax for the `jsonpath` filter. See the
159161
[Kubernetes documentation](https://kubernetes.io/docs/reference/kubectl/jsonpath/)
160162
for a full list of capabilities.
161163

164+
## Go library usage
165+
166+
As of v0.7.0, Murmur's internal components are available as a public Go library.
167+
You can import and use Murmur's provider system and secret resolution pipeline
168+
directly in your Go applications.
169+
170+
### Basic secret resolution
171+
172+
```go
173+
package main
174+
175+
import (
176+
"fmt"
177+
"log"
178+
179+
"github.com/busser/murmur/pkg/murmur"
180+
)
181+
182+
func main() {
183+
// Define secrets using the same syntax as environment variables
184+
secrets := map[string]string{
185+
"DB_PASSWORD": "awssm:my-database-secret",
186+
"API_KEY": "gcpsm:my-project/api-credentials|jsonpath:{.key}",
187+
"DEBUG_MODE": "passthrough:true", // Non-secret values pass through
188+
}
189+
190+
// Resolve all secrets
191+
resolved, err := murmur.ResolveAll(secrets)
192+
if err != nil {
193+
log.Fatal(err)
194+
}
195+
196+
fmt.Printf("DB Password: %s\n", resolved["DB_PASSWORD"])
197+
fmt.Printf("API Key: %s\n", resolved["API_KEY"])
198+
fmt.Printf("Debug Mode: %s\n", resolved["DEBUG_MODE"])
199+
}
200+
```
201+
202+
### Using providers directly
203+
204+
```go
205+
package main
206+
207+
import (
208+
"context"
209+
"fmt"
210+
"log"
211+
212+
"github.com/busser/murmur/pkg/murmur"
213+
)
214+
215+
func main() {
216+
// Get a specific provider
217+
providerFactory, exists := murmur.ProviderFactories["awssm"]
218+
if !exists {
219+
log.Fatal("AWS Secrets Manager provider not available")
220+
}
221+
222+
provider, err := providerFactory()
223+
if err != nil {
224+
log.Fatal(err)
225+
}
226+
defer provider.Close()
227+
228+
// Fetch a secret directly
229+
secret, err := provider.Resolve(context.Background(), "my-secret")
230+
if err != nil {
231+
log.Fatal(err)
232+
}
233+
234+
fmt.Printf("Secret value: %s\n", secret)
235+
}
236+
```
237+
238+
### Available providers
239+
240+
All built-in providers are available through `murmur.ProviderFactories`:
241+
242+
- `"awssm"` - AWS Secrets Manager
243+
- `"azkv"` - Azure Key Vault
244+
- `"gcpsm"` - GCP Secret Manager
245+
- `"scwsm"` - Scaleway Secret Manager
246+
- `"passthrough"` - Testing/no-op provider
247+
248+
### Use cases
249+
250+
The Go library enables several powerful patterns:
251+
252+
- **Configuration-based secret resolution** instead of environment variables
253+
- **Custom secret injection workflows** in Go applications
254+
- **Testing with mock providers** for unit tests
255+
- **Building custom secret management tools** on top of Murmur's provider system
256+
162257
## Providers and filters
163258

164259
Murmur's architecture is built around providers and filters. Providers fetch
@@ -283,8 +378,8 @@ azkv:example.vault.azure.net/my-secret#5ddc29704c1c4429a4c53605b7949100
283378
```
284379

285380
Murmur uses the environment's default credentials to authenticate to Azure. You
286-
can set these credentials with the [environment variables listed here](https://github.com/Azure/azure-sdk-for-go/wiki/Set-up-Your-Environment-for-Authentication#configure-defaultazurecredential),
287-
or with workload identity.
381+
can set these credentials with the [environment variables listed here](https://github.com/Azure/azure-sdk-for-go/wiki/Set-up-Your-Environment-for-Authentication#configure-defaultazurecredential)
382+
(such as `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`), or with workload identity.
288383

289384
### `gcpsm` provider: GCP Secret Manager
290385

@@ -302,6 +397,20 @@ The `name` is the name of the secret.
302397
The `version` must be a valid version number. If `version` is not specified,
303398
Murmur defaults to the latest version of the secret.
304399

400+
Examples:
401+
402+
```plaintext
403+
gcpsm:my-project-123/my-secret
404+
gcpsm:my-project-123/my-secret#1
405+
gcpsm:my-project-123/my-secret#latest
406+
407+
gcpsm:123456789012/database-credentials
408+
gcpsm:123456789012/database-credentials#5
409+
```
410+
411+
Murmur uses the environment's default credentials to authenticate to GCP.
412+
You can configure Murmur the same way you can [configure the `gcloud` CLI](https://cloud.google.com/docs/authentication/provide-credentials-adc).
413+
305414
### `passthrough` provider: no-op
306415

307416
This provider is meant for demo and testing purposes. It does not fetch any
@@ -343,6 +452,83 @@ scwsm:my-secret|jsonpath:postgres://{.username}:{.password}@{.hostname}:{.port}/
343452
scwsm:my-secret|jsonpath:the secret is {@}
344453
```
345454

455+
## Error handling and troubleshooting
456+
457+
### Common errors
458+
459+
**Invalid reference format**
460+
```
461+
Error: invalid reference: missing colon
462+
```
463+
This means your secret reference doesn't follow the correct format. Ensure you use:
464+
```
465+
provider_id:secret_ref[|filter_id:filter_rule]
466+
```
467+
468+
**Authentication failures**
469+
```
470+
Error: failed to load AWS config: NoCredentialProviders
471+
```
472+
This indicates missing or invalid cloud provider credentials. Check:
473+
- AWS: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, or AWS profile configuration
474+
- Azure: `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`
475+
- GCP: `GOOGLE_APPLICATION_CREDENTIALS` or `gcloud auth application-default login`
476+
- Scaleway: `SCW_ACCESS_KEY`, `SCW_SECRET_KEY`
477+
478+
**Secret not found**
479+
```
480+
Error: failed to access secret "my-secret": secret not found
481+
```
482+
Verify:
483+
- Secret name/ID is correct
484+
- Secret exists in the specified region/project
485+
- You have the necessary permissions to read the secret
486+
- For versioned secrets, the version exists
487+
488+
**JSONPath filter errors**
489+
```
490+
Error: failed to apply filter: invalid JSONPath expression
491+
```
492+
Check your JSONPath syntax against the [Kubernetes JSONPath documentation](https://kubernetes.io/docs/reference/kubectl/jsonpath/).
493+
494+
### Debugging tips
495+
496+
**Enable verbose logging** (when using as CLI):
497+
```bash
498+
export MURMUR_LOG_LEVEL=debug
499+
murmur run -- your-command
500+
```
501+
502+
**Test secret resolution** (when using as library):
503+
```go
504+
resolved, err := murmur.ResolveAll(map[string]string{
505+
"TEST": "passthrough:hello-world",
506+
})
507+
if err != nil {
508+
log.Printf("Resolution failed: %v", err)
509+
}
510+
```
511+
512+
**Validate individual providers**:
513+
```go
514+
provider, err := murmur.ProviderFactories["awssm"]()
515+
if err != nil {
516+
log.Printf("Provider initialization failed: %v", err)
517+
}
518+
defer provider.Close()
519+
520+
secret, err := provider.Resolve(context.Background(), "test-secret")
521+
if err != nil {
522+
log.Printf("Secret resolution failed: %v", err)
523+
}
524+
```
525+
526+
### Performance considerations
527+
528+
- **Concurrent resolution**: Murmur fetches secrets concurrently per provider, but sequentially within each provider
529+
- **Caching**: Duplicate secret references are cached within a single resolution call
530+
- **Resource cleanup**: Always call `Close()` on providers when using the library directly
531+
346532
## Changes from v0.4 to v0.5
347533

348534
Following community feedback, we have made two significant changes in v0.5:

pkg/murmur/provider.go

Lines changed: 36 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -10,23 +10,52 @@ import (
1010
"github.com/busser/murmur/pkg/murmur/providers/scwsm"
1111
)
1212

13-
// A Provider fetches values from a secret store.
13+
// Provider fetches values from a secret store (e.g., AWS Secrets Manager, Azure Key Vault).
14+
// Each provider implements a specific cloud secret management service.
15+
//
16+
// Providers are designed to be:
17+
// - Thread-safe for concurrent Resolve calls
18+
// - Stateful (may hold connections, credentials)
19+
// - Resource-managed (must call Close when done)
20+
//
21+
// Example usage:
22+
//
23+
// provider, err := awssm.New()
24+
// if err != nil {
25+
// return err
26+
// }
27+
// defer provider.Close()
28+
//
29+
// secret, err := provider.Resolve(ctx, "my-secret#AWSCURRENT")
30+
// if err != nil {
31+
// return err
32+
// }
1433
type Provider interface {
15-
// Resolve returns the value of the secret with the given ref. Resolve never
16-
// gets called after Close.
34+
// Resolve returns the value of the secret with the given ref.
35+
// The ref format is provider-specific (e.g., "secret-name#version").
36+
// Resolve should be thread-safe and may be called concurrently.
37+
// Resolve will never be called after Close.
1738
Resolve(ctx context.Context, ref string) (string, error)
1839

1940
// Close signals to the provider that it can release any resources it has
2041
// allocated, like network connections. Close should return once those
21-
// resources are released.
42+
// resources are released. After Close is called, Resolve should not be called.
2243
Close() error
2344
}
2445

25-
// A ProviderFactory returns a new Provider.
46+
// ProviderFactory creates a new Provider instance.
47+
// Each call should return a fresh provider with its own resources.
2648
type ProviderFactory func() (Provider, error)
2749

28-
// ProviderFactories contains a ProviderFactory for each prefix known to
29-
// murmur.
50+
// ProviderFactories contains a ProviderFactory for each provider prefix known to murmur.
51+
// This map is used by the resolution pipeline to create providers on-demand.
52+
//
53+
// Available providers:
54+
// - "awssm": AWS Secrets Manager
55+
// - "azkv": Azure Key Vault
56+
// - "gcpsm": GCP Secret Manager
57+
// - "scwsm": Scaleway Secret Manager
58+
// - "passthrough": Testing/no-op provider
3059
var ProviderFactories = map[string]ProviderFactory{
3160
// Passthrough
3261
"passthrough": func() (Provider, error) { return passthrough.New() },

pkg/murmur/resolve.go

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,26 @@ type variable struct {
2525
err error
2626
}
2727

28-
// ResolveAll returns a map with the same keys as vars, where all values with
29-
// known prefixes have been replaced with their values.
28+
// ResolveAll resolves all secret references in the input map and returns the resolved values.
29+
//
30+
// Input map keys are preserved. Values that contain valid murmur queries (format: "provider:ref|filter:rule")
31+
// are resolved by fetching from the appropriate secret store. Values without valid queries pass through unchanged.
32+
//
33+
// The resolution process:
34+
// 1. Parse each value to identify secret queries
35+
// 2. Group queries by provider for concurrent resolution
36+
// 3. Apply filters (like JSONPath) to transform resolved secrets
37+
// 4. Return final environment-ready values
38+
//
39+
// Example:
40+
// input := map[string]string{
41+
// "DB_PASSWORD": "awssm:prod/database#AWSCURRENT",
42+
// "API_KEY": "gcpsm:my-project/api-key|jsonpath:{.token}",
43+
// "DEBUG": "true", // passes through unchanged
44+
// }
45+
// resolved, err := ResolveAll(input)
46+
//
47+
// Returns an error if any secret resolution fails. Partial results are not returned on error.
3048
func ResolveAll(vars map[string]string) (map[string]string, error) {
3149
var (
3250
rawVars = make(chan variable, len(vars))

terraform/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Cloud Infrastucture <!-- omit in toc -->
1+
# Cloud Infrastructure <!-- omit in toc -->
22

33
This directory contains all Terraform code required to provision the cloud
44
resources used to test murmur functionality.

0 commit comments

Comments
 (0)