Skip to content

Commit d8a482d

Browse files
Merge pull request #28 from did-method-plc/DavidBuchanan314/plc-replica-2
PLC Replica (Take 2)
2 parents 7727ee7 + ad68733 commit d8a482d

34 files changed

Lines changed: 4339 additions & 89 deletions

.github/workflows/golang.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ jobs:
2424
run: make build
2525
- name: Test
2626
run: make test
27+
- name: Test (race detector)
28+
run: make test-race
2729
lint:
2830
runs-on: ubuntu-latest
2931
steps:

Makefile

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,20 @@ help: ## Print info about all commands
1111
.PHONY: build
1212
build: ## Build all executables
1313
go build ./cmd/plcli
14+
go build -o plc-replica ./cmd/replica
1415

1516
.PHONY: all
1617
all: build
1718

1819
.PHONY: test
1920
test: ## Run tests
20-
go test -short ./...
21+
go test -v -short ./...
22+
./extra/pg/with-test-db.sh go test -v -short -run TestGormOpStore ./replica/...
23+
24+
.PHONY: test-race
25+
test-race: ## Run tests with race detector
26+
go test -v -short -race ./...
27+
./extra/pg/with-test-db.sh go test -v -short -race -run TestGormOpStore ./replica/...
2128

2229
.PHONY: coverage-html
2330
coverage-html: ## Generate test coverage report and open in browser

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
`go-didplc`: Go implementation of DID PLC method
33
================================================
44

5-
**NOTE:** this codebase is work-in-progress: has not been reviewed and will have rapid API breaking changes.
5+
**NOTE:** This codebase is pre-v1.0, there may be breaking API changes.
66

77
DID PLC is a self-authenticating [DID](https://www.w3.org/TR/did-core/) which is strongly-consistent, recoverable, and allows for key rotation. See <https://web.plc.directory> for details.
88

cmd/plcli/main.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import (
1010

1111
"github.com/bluesky-social/indigo/atproto/atcrypto"
1212
"github.com/bluesky-social/indigo/atproto/syntax"
13-
"github.com/did-method-plc/go-didplc"
13+
"github.com/did-method-plc/go-didplc/didplc"
1414

1515
"github.com/urfave/cli/v3"
1616
)

cmd/replica/Dockerfile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM golang:1.25 AS builder
2+
WORKDIR /src
3+
COPY go.mod go.sum ./
4+
RUN go mod download
5+
COPY . .
6+
RUN go build -o /plc-replica ./cmd/replica
7+
8+
FROM debian:bookworm-slim
9+
RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*
10+
COPY --from=builder /plc-replica /usr/local/bin/plc-replica
11+
WORKDIR /data
12+
ENTRYPOINT ["plc-replica"]

cmd/replica/README.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# PLC Replica Service
2+
3+
The `replica` command implements a `did:plc` read-replica service that syncs operations from an upstream PLC directory service, and exposes the standard HTTP APIs for resolving and auditing DID documents.
4+
5+
It performs full cryptographic validation of all inbound PLC operations, including enforcing constraints around operation nullification.
6+
7+
```
8+
NAME:
9+
plc-replica - PLC directory replica server
10+
11+
USAGE:
12+
plc-replica [global options]
13+
14+
GLOBAL OPTIONS:
15+
--db-url string Database URL (e.g. sqlite://replica.db?_journal_mode=WAL, postgres://user:pass@host/db) (default: "sqlite://replica.db?mode=rwc&cache=shared&_journal_mode=WAL") [$DATABASE_URL]
16+
--bind string HTTP server listen address (default: ":6780") [$REPLICA_BIND]
17+
--metrics-addr string Metrics HTTP server listen address (default: ":9464") [$METRICS_ADDR]
18+
--no-ingest Disable ingestion from upstream directory [$NO_INGEST]
19+
--upstream-directory-url string Upstream PLC directory base URL (default: "https://plc.directory") [$UPSTREAM_DIRECTORY_URL]
20+
--cursor-override int Initial cursor value used to sync from the upstream host. May be useful when switching the upstream host (default: -1) [$CURSOR_OVERRIDE]
21+
--num-workers int Number of validation worker threads (0 = auto) (default: 0) [$NUM_WORKERS]
22+
--log-level string Log level (debug, info, warn, error) (default: "info") [$LOG_LEVEL]
23+
--log-json Output logs in JSON format [$LOG_JSON]
24+
--help, -h show help
25+
```
26+
27+
## HTTP API
28+
29+
It exposes the following endpoints, as described in the `did:plc` [spec](https://web.plc.directory/spec/v0.1/did-plc)
30+
31+
- `GET /{did}` (see Format Differences below)
32+
- `GET /{did}/data`
33+
- `GET /{did}/log`
34+
- `GET /{did}/log/audit`
35+
- `GET /{did}/log/last`
36+
37+
Actually, some of these aren't mentioned in the spec, but they are in the [API docs](https://web.plc.directory/api/redoc) and implemented by the [reference implementation](https://github.com/did-method-plc/did-method-plc/tree/main/packages/server).
38+
39+
It does not support POSTing DID updates to `/{did}` - it only discovers new operations by importing from the upstream instance.
40+
41+
It does not currently implement the `/export` and `/export/stream` endpoints, although it may in the future.
42+
43+
### DID Document Format Differences
44+
45+
The reference implementation returns DID documents in `application/did+ld+json` format, whereas this replica returns them in `application/did+json` format. Both are described in the [DID specification](https://www.w3.org/TR/did-1.0/), but in practical terms the difference is that the `@context` field is missing.
46+
47+
Secondarily, service identifiers include the DID ([relevant issue](https://github.com/did-method-plc/did-method-plc/issues/90))
48+
49+
Although these differences are spec-compliant, some PLC client libraries may have trouble with these differences.
50+
51+
52+
## Databases
53+
54+
The service supports either PostgreSQL or SQLite. Postgres has more horizontal scaling headroom on the read path, but SQLite performs better when backfilling.
55+
56+
When using PostgresSQL, you may wish to set `synchronous_commit` to `off`. This can improve ingest performance, at the cost potentially losing some recently-committed data after e.g. a power failure. Since this is a replica service, it should be able to quickly re-sync from the upstream host if that happens, so no data is truly lost.
57+
58+
## Backfilling
59+
60+
When the service is started for the first time, it has to "backfill" the entire PLC operation history from the upstream instance. Until it "catches up", it will not provide up-to-date responses to queries. Depending on your hardware, it should take less than 24h to complete a backfill (at time of writing). Backfilling tends to be bottlenecked by database throughput.
61+
62+
## Metrics and Tracing
63+
64+
In addition to the `--metrics-addr` CLI flag, the [`OTEL_EXPORTER_OTLP_ENDPOINT`](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_endpoint) env var may be set to configure trace reporting.

cmd/replica/main.go

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"fmt"
6+
"log/slog"
7+
"net/http"
8+
"os"
9+
"runtime"
10+
11+
"github.com/did-method-plc/go-didplc/replica"
12+
"github.com/prometheus/client_golang/prometheus/promhttp"
13+
"github.com/urfave/cli/v3"
14+
"golang.org/x/sync/errgroup"
15+
)
16+
17+
func main() {
18+
cmd := &cli.Command{
19+
Name: "plc-replica",
20+
Usage: "PLC directory replica server",
21+
Flags: []cli.Flag{
22+
&cli.StringFlag{
23+
Name: "db-url",
24+
Usage: "Database URL (e.g. sqlite://replica.db?_journal_mode=WAL, postgres://user:pass@host/db)",
25+
Value: "sqlite://replica.db?mode=rwc&cache=shared&_journal_mode=WAL",
26+
Sources: cli.EnvVars("DATABASE_URL"),
27+
},
28+
&cli.StringFlag{
29+
Name: "bind",
30+
Usage: "HTTP server listen address",
31+
Value: ":6780",
32+
Sources: cli.EnvVars("REPLICA_BIND"),
33+
},
34+
&cli.StringFlag{
35+
Name: "metrics-addr",
36+
Usage: "Metrics HTTP server listen address",
37+
Value: ":9464",
38+
Sources: cli.EnvVars("METRICS_ADDR"),
39+
},
40+
&cli.BoolFlag{
41+
Name: "no-ingest",
42+
Usage: "Disable ingestion from upstream directory",
43+
Sources: cli.EnvVars("NO_INGEST"),
44+
},
45+
&cli.StringFlag{
46+
Name: "upstream-directory-url",
47+
Usage: "Upstream PLC directory base URL",
48+
Value: "https://plc.directory",
49+
Sources: cli.EnvVars("UPSTREAM_DIRECTORY_URL"),
50+
},
51+
&cli.Int64Flag{
52+
Name: "cursor-override",
53+
Usage: "Initial cursor value used to sync from the upstream host. May be useful when switching the upstream host",
54+
Value: -1,
55+
Sources: cli.EnvVars("CURSOR_OVERRIDE"),
56+
},
57+
&cli.IntFlag{
58+
Name: "num-workers",
59+
Usage: "Number of validation worker threads (0 = auto)",
60+
Value: 0,
61+
Sources: cli.EnvVars("NUM_WORKERS"),
62+
},
63+
&cli.StringFlag{
64+
Name: "log-level",
65+
Usage: "Log level (debug, info, warn, error)",
66+
Value: "info",
67+
Sources: cli.EnvVars("LOG_LEVEL"),
68+
},
69+
&cli.BoolFlag{
70+
Name: "log-json",
71+
Usage: "Output logs in JSON format",
72+
Sources: cli.EnvVars("LOG_JSON"),
73+
},
74+
},
75+
Action: run,
76+
}
77+
78+
if err := cmd.Run(context.Background(), os.Args); err != nil {
79+
slog.Error("fatal error", "error", err)
80+
os.Exit(1)
81+
}
82+
}
83+
84+
func run(ctx context.Context, cmd *cli.Command) error {
85+
// Parse configuration
86+
dbURL := cmd.String("db-url")
87+
httpAddr := cmd.String("bind")
88+
metricsAddr := cmd.String("metrics-addr")
89+
noIngest := cmd.Bool("no-ingest")
90+
directoryURL := cmd.String("upstream-directory-url")
91+
cursorOverride := cmd.Int64("cursor-override")
92+
numWorkers := cmd.Int("num-workers")
93+
logLevel := cmd.String("log-level")
94+
logJSON := cmd.Bool("log-json")
95+
96+
// Initialize logger
97+
var level slog.Level
98+
switch logLevel {
99+
case "debug":
100+
level = slog.LevelDebug
101+
case "info":
102+
level = slog.LevelInfo
103+
case "warn":
104+
level = slog.LevelWarn
105+
case "error":
106+
level = slog.LevelError
107+
default:
108+
level = slog.LevelInfo
109+
}
110+
111+
var handler slog.Handler
112+
opts := &slog.HandlerOptions{Level: level}
113+
if logJSON {
114+
handler = slog.NewJSONHandler(os.Stdout, opts)
115+
} else {
116+
handler = slog.NewTextHandler(os.Stdout, opts)
117+
}
118+
logger := slog.New(handler)
119+
slog.SetDefault(logger)
120+
121+
if numWorkers <= 0 {
122+
numWorkers = runtime.NumCPU()
123+
}
124+
125+
otelShutdown, err := setupOTel(ctx)
126+
if err != nil {
127+
return fmt.Errorf("otel setup: %w", err)
128+
}
129+
defer otelShutdown(context.Background())
130+
131+
store, err := replica.NewGormOpStore(dbURL, logger)
132+
if err != nil {
133+
return fmt.Errorf("failed to create store: %w", err)
134+
}
135+
136+
state := replica.NewReplicaState()
137+
server := replica.NewServer(store, state, httpAddr, logger)
138+
g, gctx := errgroup.WithContext(ctx)
139+
140+
g.Go(server.Run)
141+
142+
g.Go(func() error {
143+
mux := http.NewServeMux()
144+
mux.Handle("/metrics", promhttp.Handler())
145+
slog.Info("metrics server listening", "addr", metricsAddr)
146+
return http.ListenAndServe(metricsAddr, mux)
147+
})
148+
149+
if !noIngest {
150+
ingestor, err := replica.NewIngestor(store, state, directoryURL, cursorOverride, numWorkers, logger)
151+
if err != nil {
152+
return err
153+
}
154+
g.Go(func() error {
155+
return ingestor.Run(gctx)
156+
})
157+
}
158+
159+
return g.Wait()
160+
}

cmd/replica/otel.go

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"errors"
6+
"os"
7+
8+
"go.opentelemetry.io/otel"
9+
"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
10+
"go.opentelemetry.io/otel/exporters/prometheus"
11+
"go.opentelemetry.io/otel/propagation"
12+
sdkmetric "go.opentelemetry.io/otel/sdk/metric"
13+
"go.opentelemetry.io/otel/sdk/resource"
14+
sdktrace "go.opentelemetry.io/otel/sdk/trace"
15+
semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
16+
)
17+
18+
func setupOTel(ctx context.Context) (shutdown func(context.Context) error, err error) {
19+
res, err := resource.New(ctx,
20+
resource.WithAttributes(
21+
semconv.ServiceName("plc-replica"),
22+
),
23+
)
24+
if err != nil {
25+
return nil, err
26+
}
27+
28+
var shutdowns []func(context.Context) error
29+
30+
// Traces: OTLP HTTP exporter, only enabled if an endpoint is configured.
31+
if os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT") != "" || os.Getenv("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT") != "" {
32+
traceExporter, err := otlptracehttp.New(ctx)
33+
if err != nil {
34+
return nil, err
35+
}
36+
37+
tp := sdktrace.NewTracerProvider(
38+
sdktrace.WithBatcher(traceExporter),
39+
sdktrace.WithResource(res),
40+
)
41+
otel.SetTracerProvider(tp)
42+
otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
43+
propagation.TraceContext{},
44+
propagation.Baggage{},
45+
))
46+
shutdowns = append(shutdowns, tp.Shutdown)
47+
}
48+
49+
// Metrics: Prometheus exporter, served via /metrics HTTP endpoint.
50+
promExporter, err := prometheus.New()
51+
if err != nil {
52+
return nil, err
53+
}
54+
55+
mp := sdkmetric.NewMeterProvider(
56+
sdkmetric.WithReader(promExporter),
57+
sdkmetric.WithResource(res),
58+
)
59+
otel.SetMeterProvider(mp)
60+
shutdowns = append(shutdowns, mp.Shutdown)
61+
62+
shutdown = func(ctx context.Context) error {
63+
var errs []error
64+
for _, fn := range shutdowns {
65+
errs = append(errs, fn(ctx))
66+
}
67+
return errors.Join(errs...)
68+
}
69+
return shutdown, nil
70+
}
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)