Skip to content

Commit 8ffffeb

Browse files
authored
Merge pull request #787 from dahlia/feat/bench/benchmark-mode
Add cooperative benchmark mode for federation targets
2 parents 3737de7 + de996c3 commit 8ffffeb

15 files changed

Lines changed: 1794 additions & 19 deletions

File tree

CHANGES.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,18 @@ To be released.
126126
held activities call the outbox permanent failure handler with
127127
`reason: "circuit-breaker-ttl"`. [[#620], [#778]]
128128

129+
- Added `benchmarkMode` to `createFederation()` and
130+
`FederationBuilder.build()` for cooperative federation benchmarking.
131+
When enabled, Fedify exposes `GET /.well-known/fedify/bench/stats`
132+
for in-process OpenTelemetry metric snapshots and
133+
`POST /.well-known/fedify/bench/trigger` for driving `sendActivity()`
134+
to server-configured benchmark sink recipients. Benchmark mode also
135+
defaults `allowPrivateAddress` to `true` when built-in loaders are used,
136+
defaults `signatureTimeWindow` to `false`, reports queue depth through
137+
the new `fedify.queue.depth` gauge, and adds explicit low-latency
138+
buckets to the signature verification duration histogram.
139+
[[#744], [#782], [#787]]
140+
129141
- Added OpenTelemetry metrics for ActivityPub fanout and activity
130142
lifecycle events, complementing the per-recipient
131143
`activitypub.delivery.*` counters and the per-task
@@ -248,6 +260,7 @@ To be released.
248260
[#740]: https://github.com/fedify-dev/fedify/issues/740
249261
[#741]: https://github.com/fedify-dev/fedify/issues/741
250262
[#742]: https://github.com/fedify-dev/fedify/issues/742
263+
[#744]: https://github.com/fedify-dev/fedify/issues/744
251264
[#748]: https://github.com/fedify-dev/fedify/pull/748
252265
[#752]: https://github.com/fedify-dev/fedify/issues/752
253266
[#753]: https://github.com/fedify-dev/fedify/pull/753
@@ -261,6 +274,8 @@ To be released.
261274
[#772]: https://github.com/fedify-dev/fedify/pull/772
262275
[#777]: https://github.com/fedify-dev/fedify/pull/777
263276
[#778]: https://github.com/fedify-dev/fedify/pull/778
277+
[#782]: https://github.com/fedify-dev/fedify/issues/782
278+
[#787]: https://github.com/fedify-dev/fedify/pull/787
264279

265280
### @fedify/fixture
266281

docs/.vitepress/config.mts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,7 @@ const MANUAL = {
154154
{ text: "Linting", link: "/manual/lint.md" },
155155
{ text: "Logging", link: "/manual/log.md" },
156156
{ text: "OpenTelemetry", link: "/manual/opentelemetry.md" },
157+
{ text: "Benchmarking", link: "/manual/benchmarking.md" },
157158
{ text: "Deployment", link: "/manual/deploy.md" },
158159
],
159160
activeMatch: "/manual",

docs/manual/benchmarking.md

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
---
2+
description: >-
3+
Fedify can expose cooperative benchmark endpoints for measuring federation
4+
workloads without requiring an external metrics backend.
5+
---
6+
7+
Benchmarking
8+
============
9+
10+
*This API is available since Fedify 2.3.0.*
11+
12+
Fedify can run as a cooperative benchmark target by enabling
13+
`~FederationOptions.benchmarkMode`. This mode exposes local benchmark
14+
endpoints under `/.well-known/fedify/bench/` and configures an in-process
15+
OpenTelemetry metrics reader so benchmark clients can collect server-side
16+
measurements without a separate metrics backend.
17+
18+
> [!WARNING]
19+
> Do not enable `benchmarkMode` in production. It is intended for benchmark
20+
> targets that you control.
21+
22+
23+
Enabling benchmark mode
24+
-----------------------
25+
26+
Enable `benchmarkMode` when creating the `Federation` object. If you use the
27+
benchmark trigger endpoint, configure the sink inboxes on the server:
28+
29+
~~~~ typescript twoslash
30+
import type { KvStore } from "@fedify/fedify";
31+
// ---cut-before---
32+
import { createFederation } from "@fedify/fedify";
33+
34+
const federation = createFederation<void>({
35+
// ---cut-start---
36+
kv: null as unknown as KvStore,
37+
// ---cut-end---
38+
benchmarkMode: {
39+
triggerSinks: ["https://sink.example/inbox"],
40+
},
41+
});
42+
~~~~
43+
44+
When enabled, Fedify changes only benchmark-target defaults:
45+
46+
- `~FederationOptions.allowPrivateAddress` defaults to `true`, unless a
47+
custom document loader factory is configured.
48+
- `~FederationOptions.signatureTimeWindow` defaults to `false`.
49+
- Explicit `allowPrivateAddress` and `signatureTimeWindow` values still win.
50+
- Inbox idempotency is unchanged. Benchmark clients that need repeated
51+
deliveries should mint unique activity IDs.
52+
53+
If you provide `meterProvider` together with `benchmarkMode`, Fedify throws a
54+
`TypeError`. OpenTelemetry metric readers have to be attached when a
55+
`MeterProvider` is constructed, so benchmark mode owns its in-process provider.
56+
57+
If the same application code sometimes runs with benchmark mode and sometimes
58+
runs with your normal OpenTelemetry pipeline, pass your application
59+
`meterProvider` only when benchmark mode is off:
60+
61+
~~~~ typescript twoslash
62+
import type { KvStore } from "@fedify/fedify";
63+
import type { MeterProvider } from "@opentelemetry/api";
64+
// ---cut-start---
65+
declare const process: { env: Record<string, string | undefined> };
66+
const kv = null as unknown as KvStore;
67+
const meterProvider = null as unknown as MeterProvider;
68+
// ---cut-end---
69+
import { createFederation } from "@fedify/fedify";
70+
71+
const benchmarkEnabled = process.env.FEDIFY_BENCHMARK === "1";
72+
73+
const federation = createFederation<void>({
74+
kv,
75+
benchmarkMode: benchmarkEnabled
76+
? { triggerSinks: ["https://sink.example/inbox"] }
77+
: false,
78+
meterProvider: benchmarkEnabled ? undefined : meterProvider,
79+
});
80+
~~~~
81+
82+
83+
Benchmark stats endpoint
84+
------------------------
85+
86+
`GET /.well-known/fedify/bench/stats` returns a versioned JSON snapshot of the
87+
server-side metrics collected by the benchmark mode reader:
88+
89+
~~~~ json
90+
{
91+
"version": 1,
92+
"source": "server",
93+
"generatedAt": "2026-06-02T00:00:00.000Z",
94+
"scopeMetrics": [],
95+
"errors": []
96+
}
97+
~~~~
98+
99+
The `scopeMetrics` field contains serialized OpenTelemetry scope metrics.
100+
Observable queue depth is included when configured queues implement
101+
`MessageQueue.getDepth()`.
102+
103+
104+
Benchmark trigger endpoint
105+
--------------------------
106+
107+
`POST /.well-known/fedify/bench/trigger` asks the target application to call
108+
`Context.sendActivity()` with an explicit sender, recipients, and activity.
109+
This exercises the target's normal outbox and queue path.
110+
111+
The request body has this shape:
112+
113+
~~~~ json
114+
{
115+
"sender": { "identifier": "alice" },
116+
"recipients": [
117+
{
118+
"@context": "https://www.w3.org/ns/activitystreams",
119+
"type": "Service",
120+
"id": "https://sink.example/actors/bob",
121+
"inbox": "https://sink.example/inbox"
122+
}
123+
],
124+
"activity": {
125+
"@context": "https://www.w3.org/ns/activitystreams",
126+
"type": "Create",
127+
"id": "https://example.com/activities/bench-1",
128+
"actor": "https://example.com/users/alice",
129+
"object": {
130+
"type": "Note",
131+
"id": "https://example.com/notes/bench-1",
132+
"content": "benchmark"
133+
}
134+
}
135+
}
136+
~~~~
137+
138+
The `sender` must be either `{ "identifier": string }` or
139+
`{ "username": string }`. Recipients are parsed as ActivityPub actors and must
140+
have `id` and `inbox` properties. The activity is parsed as an ActivityPub
141+
`Activity`.
142+
143+
By default, every recipient inbox must appear in the server-configured
144+
`~FederationBenchmarkOptions.triggerSinks` list. This keeps benchmark traffic
145+
pointed at benchmark sink inboxes and prevents callers from choosing their own
146+
allowlist. To bypass this guard for a controlled run, set
147+
`~FederationBenchmarkOptions.allowUnsafeTriggerRecipients` to `true` in the
148+
application configuration.
149+
150+
A successful trigger returns `202 Accepted`:
151+
152+
~~~~ json
153+
{
154+
"version": 1,
155+
"activityId": "https://example.com/activities/bench-1",
156+
"queueCorrelationId": "https://example.com/activities/bench-1",
157+
"recipientCount": 1,
158+
"inboxCount": 1
159+
}
160+
~~~~
161+
162+
The `queueCorrelationId` is the activity ID preserved on the queued fanout or
163+
outbox work.
164+
165+
166+
Metrics
167+
-------
168+
169+
Benchmark mode uses the same Fedify metrics documented in
170+
[*OpenTelemetry*](./opentelemetry.md), including queue task metrics, queue
171+
depth, HTTP server metrics, and signature verification histograms. The
172+
benchmark endpoints themselves are classified as `fedify.endpoint=benchmark`
173+
in `fedify.http.server.request.*` metrics.

docs/manual/federation.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,27 @@ Turned off by default.
275275
276276
[SSRF]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
277277
278+
### `benchmarkMode`
279+
280+
*This API is available since Fedify 2.3.0.*
281+
282+
Whether to enable cooperative benchmark mode. When enabled, Fedify exposes
283+
benchmark endpoints under `/.well-known/fedify/bench/` and configures an
284+
in-process metrics reader for benchmark clients.
285+
286+
This mode changes only benchmark-target defaults:
287+
288+
- `allowPrivateAddress` defaults to `true`, unless a custom document loader
289+
factory is configured.
290+
- `signatureTimeWindow` defaults to `false`.
291+
- Explicit option values still win.
292+
293+
> [!WARNING]
294+
> Do not enable `benchmarkMode` in production.
295+
296+
See the [*Benchmarking* section](./benchmarking.md) for endpoint details and
297+
safety rules.
298+
278299
### `userAgent`
279300
280301
*This API is available since Fedify 1.3.0.*

docs/manual/opentelemetry.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,6 +370,7 @@ Fedify records the following OpenTelemetry metrics:
370370
| `fedify.queue.task.failed` | Counter | `{task}` | Counts queue tasks Fedify abandoned because processing threw. |
371371
| `fedify.queue.task.duration` | Histogram | `ms` | Measures queue task processing duration in Fedify workers. |
372372
| `fedify.queue.task.in_flight` | UpDownCounter | `{task}` | Tracks queue tasks currently in flight in this Fedify process. |
373+
| `fedify.queue.depth` | Gauge | `{message}` | Reports queued, ready, and delayed queue depth when the queue backend supports it. |
373374

374375
### Metric attributes
375376

@@ -841,6 +842,17 @@ Fedify records the following OpenTelemetry metrics:
841842
Fedify process*, not cross-process totals. Aggregate it across
842843
replicas in your metrics backend.
843844

845+
`fedify.queue.depth`
846+
: `fedify.queue.depth.state` is always present and is one of `queued`,
847+
`ready`, or `delayed`. `fedify.queue.role` is `inbox`, `outbox`,
848+
`fanout`, or `shared`; `shared` means the same queue instance backs more
849+
than one Fedify queue role, and `fedify.queue.roles` lists those roles as a
850+
comma-separated string. `fedify.queue.backend` and
851+
`fedify.queue.native_retrial` follow the same rules as the queue task
852+
metrics. `fedify.federation.instance_id` is an opaque per-Federation
853+
instance identifier that keeps queue depth series distinct when multiple
854+
Federation instances share one [`MeterProvider`].
855+
844856
The `fedify.queue.task.*` metrics describe what Fedify's workers do with
845857
queued messages. They complement the backend-side
846858
[`MessageQueue.getDepth()` API](./mq.md#queue-depth-reporting), which
@@ -849,6 +861,10 @@ Reading both signals together (task throughput plus backlog depth)
849861
makes it possible to distinguish a small, slow queue from a large, fast
850862
one and to set alerting thresholds for delivery latency under load.
851863

864+
When [`benchmarkMode`](./benchmarking.md) is enabled, Fedify serves a
865+
versioned snapshot of these in-process metrics from
866+
`/.well-known/fedify/bench/stats`.
867+
852868
The `activitypub.inbox.activity`, `activitypub.outbox.activity`, and
853869
`activitypub.fanout.recipients` metrics describe what is happening at
854870
the *activity* level, complementing the per-recipient
@@ -951,16 +967,19 @@ for ActivityPub:
951967
| `docloader.document_url` | string | The final URL of the fetched document (after following redirects). | `"https://example.com/object/1"` |
952968
| `fedify.actor.identifier` | string | The identifier of the actor. | `"1"` |
953969
| `fedify.endpoint` | string | The bounded endpoint category that classified an inbound HTTP request handled by `Federation.fetch()`. | `"actor"` |
970+
| `fedify.federation.instance_id` | string | Opaque per-Federation instance identifier used to distinguish queue depth series on a shared `MeterProvider`. | `"fedify-1"` |
954971
| `fedify.route.template` | string | The matched URI Template, with parameter names (not values). | `"/users/{identifier}"` |
955972
| `fedify.inbox.recipient` | string | The identifier of the inbox recipient. | `"1"` |
956973
| `fedify.object.type` | string | The URI of the object type. | `"https://www.w3.org/ns/activitystreams#Note"` |
957974
| `fedify.object.values.{parameter}` | string[] | The argument values of the object dispatcher. | `["1", "2"]` |
958975
| `fedify.collection.dispatcher` | string | The collection dispatcher family: `built_in` or `custom`. | `"built_in"` |
959976
| `fedify.collection.cursor` | string | The cursor of the collection. | `"eyJpZCI6IjEiLCJ0eXBlIjoiT3JkZXJlZENvbGxlY3Rpb24ifQ=="` |
960977
| `fedify.collection.items` | number | The number of materialized items in the collection response or page. It can be less than the total items. | `10` |
961-
| `fedify.queue.role` | string | The Fedify queue role for the task: `inbox`, `outbox`, or `fanout`. | `"outbox"` |
978+
| `fedify.queue.role` | string | The Fedify queue role: `inbox`, `outbox`, `fanout`, or `shared` for queue depth rows where one queue backs multiple roles. | `"outbox"` |
962979
| `fedify.queue.backend` | string | The queue implementation's constructor name (best-effort backend identifier). | `"RedisMessageQueue"` |
963980
| `fedify.queue.native_retrial` | boolean | Whether the queue backend declares `nativeRetrial`, meaning Fedify defers retry handling to the backend. | `true` |
981+
| `fedify.queue.depth.state` | string | Queue depth count kind: `queued`, `ready`, or `delayed`. | `"queued"` |
982+
| `fedify.queue.roles` | string | Comma-separated queue roles when one queue instance backs multiple roles. | `"fanout,inbox,outbox"` |
964983
| `fedify.queue.task.attempt` | int | The zero-based attempt number recorded on `fedify.queue.task.enqueued`; non-zero for retry re-enqueues. | `1` |
965984
| `fedify.queue.task.result` | string | The terminal outcome of queue task processing: `completed`, `failed`, or `aborted`. | `"failed"` |
966985
| `http.redirect.url` | string | The redirect URL when a document fetch results in a redirect. | `"https://example.com/new-location"` |

packages/fedify/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,7 @@
147147
"@logtape/logtape": "catalog:",
148148
"@opentelemetry/api": "catalog:",
149149
"@opentelemetry/core": "catalog:",
150+
"@opentelemetry/sdk-metrics": "catalog:",
150151
"@opentelemetry/sdk-trace-base": "catalog:",
151152
"@opentelemetry/semantic-conventions": "catalog:",
152153
"byte-encodings": "catalog:",
@@ -159,7 +160,6 @@
159160
"devDependencies": {
160161
"@fedify/fixture": "workspace:*",
161162
"@fedify/vocab-tools": "workspace:^",
162-
"@opentelemetry/sdk-metrics": "catalog:",
163163
"@std/assert": "jsr:^0.226.0",
164164
"@std/path": "catalog:",
165165
"@types/node": "^24.2.1",

0 commit comments

Comments
 (0)