Skip to content

README.md

Latest commit

 

History

History
138 lines (104 loc) · 6.19 KB

README.md

File metadata and controls

138 lines (104 loc) · 6.19 KB
Raw

redb.Tsak — deployment & observability assets

Ready-to-use manifests and configs for running and monitoring the Tsak Worker.

deploy/
├── k8s/                         Kubernetes manifests
│   ├── deployment.yaml          Deployment (correct /api/health/* probes, metrics port, OTLP env)
│   ├── service.yaml             Service (REST + /metrics on 9090)
│   └── servicemonitor.yaml      Prometheus Operator ServiceMonitor (alternative to scrape annotations)
├── grafana/
│   └── redb-tsak-dashboard.json Importable dashboard (routes + .NET runtime)
└── observability/               Local Prometheus + Grafana + Jaeger stack
    ├── docker-compose.yml
    ├── prometheus.yml
    └── grafana/provisioning/…   Auto-provisioned datasource + dashboard

Clustering — one node out of the box

The Tsak cluster is redb-backed: leader election, heartbeats and round-robin route distribution live in the redb store (this is not Quartz clustering, which stays off on SQLite). The shipped appsettings.json is already wired for it, so there is nothing to turn on:

  • Tsak:Redb:UsePro: true — Pro, including tsak.cluster, is free and unrestricted on the whole 3.x line, so there is no license key and no node cap;
  • Tsak:Cluster:Enabled: true, on an embedded SQLite store (zero external dependencies).

So a single Worker process is already a working one-node cluster — start it and it elects itself leader and begins distributing routes. Add as many nodes as you like — they point at the same redb database, share ClusterName/GroupName, each with an empty NodeId (self-assigned), and join automatically.

"Tsak": {
  "Redb":    { "Provider": "sqlite", "UsePro": true, "License": [ ] },
  "Cluster": { "Enabled": true, "ClusterName": "default", "GroupName": "default", "Strategy": "round-robin" }
}

Run standalone (no cluster) by flipping one flag:

"Tsak": { "Cluster": { "Enabled": false } }

License stays empty — the whole 3.x line runs Pro free, in production, with no node cap (licensing re-enables only at major 4.0). For a multi-node cluster the one thing you do change is the store: point every node at a shared Postgres/MSSql redb database instead of embedded SQLite.

What Tsak exposes

Signal How Toggle Port
Metrics Prometheus /metrics on the facade Tsak:Metrics:Prometheus:Enabled 9090
Traces OTel → OTLP exporter (Jaeger/collector) Tsak:Tracing:Otlp:Enabled →4317
Health K8s probes (auth-exempt) always on 9090

Both observability exporters are off by default. Enable via config or env-var (Tsak__Metrics__Prometheus__Enabled=true, Tsak__Tracing__Otlp__Enabled=true).

Metrics are served through the facade at http://<host>:9090/metrics (auth-exempt) — the same Kestrel port as the API. The OTel scrape listener binds loopback (localhost:9464) internally and is proxied out by the /metrics route, so there is no separate exposed port and no Windows URL ACL to set up. Tsak:Metrics:Prometheus:Port only changes the internal loopback port.

Health probe paths (auth-exempt)

  • GET /api/health/startup — startup probe
  • GET /api/health/live — liveness probe
  • GET /api/health/ready — readiness probe

These live under /api/health/* (the Tsak:Api:AuthExempt prefix). The rich aggregate GET /api/system/health is a separate, auth-gated endpoint for CLI/Web — do not point K8s probes at it.

Prometheus metric names

The OTel Prometheus exporter sanitises names and appends unit + _total. The exact series exported by redb.Route (verified against OpenTelemetry.Exporter.Prometheus.HttpListener 1.15):

Instrument (OTel) Type Prometheus series
redb.route.exchanges.processed counter redb_route_exchanges_processed_exchanges_total
redb.route.exchanges.failed counter redb_route_exchanges_failed_exchanges_total
redb.route.exchange.duration (ms) histogram redb_route_exchange_duration_milliseconds_{bucket,sum,count}
redb.route.exchanges.inflight gauge redb_route_exchanges_inflight_exchanges

Labels: redb_route_id, redb_route_endpoint, redb_route_scheme, plus otel_scope_name. Runtime/process series use the dotnet_* and process_* prefixes.

Local stack (Docker)

cd deploy/observability
docker compose up -d

Then run the Worker on the host with both exporters on:

Tsak__Metrics__Prometheus__Enabled=true Tsak__Tracing__Otlp__Enabled=true dotnet run --project src/redb.Tsak.Worker

Prometheus scrapes the facade's :9090/metrics; the Worker pushes traces to Jaeger's OTLP :4317.

Kubernetes

kubectl apply -n redb -f deploy/k8s/deployment.yaml -f deploy/k8s/service.yaml
# With the Prometheus Operator (kube-prometheus-stack) instead of scrape annotations:
kubectl apply -n redb -f deploy/k8s/servicemonitor.yaml

Edit the image, terminationGracePeriodSeconds / Tsak__Shutdown__TimeoutSeconds, the Redb connection (use a Secret), and the OTLP Endpoint (point at your in-cluster Jaeger collector) before applying.

Jaeger / OTLP notes

Jaeger ingests OTLP natively — no Jaeger-specific exporter needed. Set:

"Tsak": { "Tracing": { "Otlp": {
  "Enabled": true,
  "Endpoint": "http://localhost:4317",   // gRPC; use :4318 with "Protocol": "http/protobuf"
  "Protocol": "grpc"
} } }

Module-level ActivitySources (e.g. RedbIdentity) are picked up when listed under Tsak:Metrics:Prometheus:AdditionalSources — they then flow to the same OTLP exporter.