feat: expose assault metrics via Micrometer/Prometheus (#46)

README.md

@@ -22,6 +22,7 @@ Quarkus has excellent resilience primitives (MicroProfile Fault Tolerance, Mutin
 - **Response body injection** -- Truncate or inflate the response entity (`TRUNCATE` keeps the first N%, `INFLATE` pads it) to break strict JSON clients and length-validating consumers
 - **Response header injection** -- Set or remove headers on emitted responses (`SET` forces the value, replacing an existing header or adding it when absent; `REMOVE` deletes it when present)
 - **Client-side assaults** -- Inject latency and exceptions into outgoing MicroProfile / Quarkus REST Client calls (`quarkus-rest-client`) and Vert.x `WebClient` calls (`GoblinWebClient.enable(...)`, opt-in at client creation)
+- **Metrics** -- Optional `quarkus-goblin-metrics` module exposing assault activity as Micrometer / Prometheus metrics (`goblin_assaults_total`, `goblin_latency_injected_seconds`, `goblin_active`, see the [Metrics guide](docs/modules/ROOT/pages/metrics.adoc))
 - **Multiple types simultaneously** -- Enable latency + exception together for slow failure simulation
 - **Targeting** -- By package, by annotation, by percentage of requests
 - **Dev UI** -- Toggle assaults, edit config, view history -- all in real time
@@ -98,6 +99,25 @@ quarkus.goblin.target.level=100
 # quarkus.goblin.target.exclude-annotations=org.eclipse.microprofile.faulttolerance.Timeout
 ```
 
+## Metrics (optional)
+
+Add the `quarkus-goblin-metrics` module to expose the assault activity through Micrometer, so it shows up in your
+Prometheus / Grafana dashboards:
+
+```xml
+<dependency>
+    <groupId>io.quarkiverse.goblin</groupId>
+    <artifactId>quarkus-goblin-metrics</artifactId>
+    <version>${goblin.version}</version>
+</dependency>
+```
+
+Metrics are scraped at the standard Prometheus endpoint `/q/metrics`:
+
+- `goblin_assaults_total` -- counter of every fired assault, tagged by `type` and `source` (`server`, `rest-client`, `webclient`)
+- `goblin_latency_injected_seconds` -- timer of the delays actually injected, tagged by `source` (sum/count/max; see the [guide](docs/modules/ROOT/pages/metrics.adoc) for histogram tuning)
+- `goblin_active` -- gauge, `1` while the engine is active, `0` otherwise
+
 ## Dev UI
 
 The Chaos Dashboard provides:
@@ -124,6 +144,7 @@ All changes apply instantly with WARN logs in the console and are persisted to `
 Each module carries its own README for contributors:
 
 - [runtime](runtime/README.md) -- the assault abstraction and how to add a new assault (the extension SPI)
+- [metrics](metrics/README.md) -- optional Micrometer / Prometheus metrics for assault activity
 - [runtime-dev](runtime-dev/README.md) -- the Dev UI JSON-RPC backend (dev mode only)
 - [deployment](deployment/README.md) -- build steps, bean registration, and Dev UI wiring
 - [integration-tests](integration-tests/README.md) -- the `@QuarkusTest` suite and how to extend it

ROADMAP.md

@@ -51,8 +51,8 @@ Current status: **preview** (v0.2.0)
 
 ## v0.3.0 -- Dev UI & Observability
 
-- [ ] **Micrometer/Prometheus metrics**
-  Expose assault counters and latency histograms via Micrometer so they appear in existing Prometheus/Grafana dashboards. Metrics: `goblin_assaults_total` (tagged by type), `goblin_latency_injected_seconds` (histogram), `goblin_active` (gauge).
+- [x] **Micrometer/Prometheus metrics**
+  Expose assault counters and latency histograms via Micrometer so they appear in existing Prometheus/Grafana dashboards. Metrics: `goblin_assaults_total` (tagged by type), `goblin_latency_injected_seconds` (histogram), `goblin_active` (gauge). Delivered as the optional `quarkus-goblin-metrics` module (#46).
 
 - [ ] **OpenTelemetry tracing integration**
   Create an OTel span for each injected assault, with attributes for assault type, target method, and injected value. Link the assault span to the parent request span for end-to-end trace correlation.

docs/modules/ROOT/pages/compatibility.adoc

@@ -24,4 +24,10 @@ Client-side assaults are implemented as a standard JAX-RS `ClientRequestFilter`
 
 == Vert.x WebClient
 
-The same client-side assaults apply to outgoing `io.vertx.ext.web.client.WebClient` calls, but Vert.x 4.x exposes no public interceptor hook on `WebClient`. Goblin therefore attaches its interceptor explicitly, at the point where the application creates its client, via `GoblinWebClient.enable(webClient)`. The interceptor relies on Vert.x's internal `WebClientInternal.addInterceptor` mechanism (the same one used by Vert.x's own `OAuth2WebClient`, `CachingWebClient` and `WebClientSession` decorators), so it works with any client backed by Vert.x's default `WebClientImpl`; a custom decorator that does not extend the base client fails fast with an `IllegalArgumentException`. A plain `WebClient` is protected by Goblin's opt-in default: without `enable(...)` it is never touched.
+The same client-side assaults apply to outgoing `io.vertx.ext.web.client.WebClient` calls, but Vert.x 4.x exposes no public interceptor hook on `WebClient`. Goblin therefore attaches its interceptor explicitly, at the point where the application creates its client, via `GoblinWebClient.enable(webClient)`. The interceptor relies on Vert.x's internal `WebClientInternal.addInterceptor` mechanism (the same one used by Vert.x's own `OAuth2WebClient`, `CachingWebClient` and `WebClientSession` decorators), so it works with any client backed by Vert.x's default `WebClientImpl`; a custom decorator that does not extend the base client fails fast with an `IllegalArgumentException`. A plain `WebClient` is protected by Goblin's opt-in default: without `enable(...)` it is never touched.
+
+== Micrometer / Prometheus
+
+Observability is opt-in through the `quarkus-goblin-metrics` module (see xref:metrics.adoc[]). It depends on
+`quarkus-micrometer-registry-prometheus` and exposes the assault metrics on `/q/metrics`; it co-exists with any other
+Micrometer-based monitoring and is inert when absent from the classpath.

docs/modules/ROOT/pages/how-it-works.adoc

@@ -48,6 +48,17 @@ TIP: Add `.goblin-state.json` to your `.gitignore` to avoid committing local cha
 
 If the state file is corrupted or unreadable, Goblin logs a warning and falls back to the configuration from `application.properties`.
 
+== Observability: the AssaultObserver SPI
+
+Every recorded assault and every active-state change is broadcast to the registered `AssaultObserver` beans
+(`io.quarkiverse.goblin.AssaultObserver`, default no-op methods). Observers run synchronously on the request path:
+the engine calls each bean and skips/logs any failing observer so observability can never break an assault. The engine
+injects `Instance<AssaultObserver>`, so any `@ApplicationScoped` implementation is picked up automatically; outside
+the CDI container (plain unit tests) the notification simply does nothing.
+
+The optional `quarkus-goblin-metrics` module is the first consumer (see xref:metrics.adoc[]); OpenTelemetry tracing
+and post-assault assertions will plug into the same hook.
+
 == Adding your own assault type
 
 Goblin ships with four built-in assault types and lets you register your own. Implement the `io.quarkiverse.goblin.assault.Assault` interface, annotate the class `@ApplicationScoped`, and it is discovered automatically at build time -- no other wiring required:

docs/modules/ROOT/pages/index.adoc

@@ -71,6 +71,7 @@ You should see a response time well above your normal baseline.
 * xref:configuration-reference.adoc[] -- every configuration key and the startup/Dev UI validation rules.
 * xref:dev-ui.adoc[] -- the Chaos Dashboard, the Assault History panel, and the Markdown report export.
 * xref:how-it-works.adoc[] -- JAX-RS filter mechanics, runtime config modification, state persistence, and adding your own assault type.
+* xref:metrics.adoc[] -- optional Micrometer / Prometheus metrics for assault activity (`quarkus-goblin-metrics`).
 * xref:compatibility.adoc[] -- coexistence with MicroProfile Fault Tolerance and both JAX-RS runtimes.
 * xref:troubleshooting.adoc[] -- common issues and their fixes.
 * xref:end-to-end-example.adoc[] -- prove that your `@Timeout` and `@Fallback` actually work.

docs/modules/ROOT/pages/metrics.adoc

@@ -0,0 +1,83 @@
+= Metrics (Micrometer / Prometheus)
+
+:page-aliases: metrics
+
+:toc: left
+:toclevels: 3
+
+include::../partials/attributes.adoc[]
+
+Goblin can surface its assault activity as Micrometer metrics, so the chaos you inject shows up in your existing Prometheus / Grafana dashboards alongside your application's regular metrics.
+
+The integration is **optional** and lives in the `quarkus-goblin-metrics` module. Adding it to the classpath is enough: it observes the assaults through the engine's observer SPI and never alters the assault behavior.
+
+== Adding the dependency
+
+[source,xml]
+----
+<dependency>
+    <groupId>io.quarkiverse.goblin</groupId>
+    <artifactId>quarkus-goblin-metrics</artifactId>
+    <version>${goblin.version}</version>
+</dependency>
+----
+
+The module brings in `quarkus-micrometer-registry-prometheus`, so `/q/metrics` is available immediately. Any other Micrometer backend (e.g. a custom `MeterRegistry`) picks up the same meters: the metrics are registered against the application's `MeterRegistry` bean.
+
+== Exposed metrics
+
+[cols="1,2,3"]
+|===
+| Metric
+| Description
+| Tags
+
+| `goblin_assaults_total`
+| Counter of every fired assault, incremented once per assault (including client-side ones).
+| `type` (e.g. `latency`, `exception`, `http-status`, `response-body-truncate`) and `source` (`server`, `rest-client`, `webclient`)
+
+| `goblin_latency_injected_seconds`
+| Timer of the delays actually injected (only incremented when latency was really applied). Exposed as a histogram: `_sum` / `_count` plus `_bucket` distribution series. Custom bucket boundaries can be set with `quarkus.micrometer.export.prometheus.prometheus.bucket-boundaries`.
+| `source`
+
+| `goblin_active`
+| Gauge reading `1` while the engine is active (the master toggle), `0` otherwise.
+|
+|===
+
+NOTE: Meter names are declared with Micrometer's dotted convention (e.g. `goblin.assaults.total`), but the Prometheus export sanitizes them to underscores -- `goblin_assaults_total` in `/q/metrics`. Other backends (JVM, statsd, ...) keep the dotted names, so write PromQL against the underscored form and refer to the dotted names when consuming the tags/names elsewhere.
+
+`source` is derived from the recorded history identifier: server-side assaults (`SampleResource.hello`) are tagged `server`, outgoing REST Client calls tag `rest-client`, and outgoing Vert.x WebClient calls tag `webclient`.
+
+TIP: Reduction: summarize by `type` only with `sum(goblin_assaults_total) by (type)`; watch a specific source with `sum(rate(goblin_latency_injected_seconds_count[5m])) by (source)`.
+
+== Reading the latency histogram
+
+Micrometer's Prometheus export keeps `_sum` / `_count` **cumulative**, but `_max` (and any quantile you derive from the `_bucket` series) only reflects the **current scrape interval**. When no latency is injected in a given interval, `_max` therefore drops to `0` -- this is the standard Prometheus/Micrometer behavior, identical to Quarkus's own `http_server_requests_seconds_max`, not a broken probe. There is no `_min` line: the histogram replaces it, and a running minimum over the buckets requires PromQL.
+
+Percentile and min/max over a rolling window, e.g. per source:
+
+[source,promql]
+----
+histogram_quantile(0.99, sum(rate(goblin_latency_injected_seconds_bucket[5m])) by (le, source))  # p99 of injected latency
+min_over_time(max_over_time(goblin_latency_injected_seconds_max[10m])[10m:])                   # persistent max if the interval gaps bother you
+----
+
+Because the histogram is enabled by default, on every scrape you get the `_bucket` series (`goblin_latency_injected_seconds_bucket{source="..."}`, with boundaries from ~1 ms up to 30 s), which is also what Grafana heatmaps expect.
+
+== Assault signals for dashboards
+
+Typical alerts:
+
+* `goblin_assaults_total > 0` -- chaos was injected (expect it while testing).
+* `goblin_active == 0` -- the engine is off, none of the other signals are changing.
+* Sudden spikes on `goblin_latency_injected_seconds` -- latency assault is degrading the injected delays.
+
+These metrics are also the foundation for the planned *post-assault assertions* (issue #50): declaring "inject 500 ms latency, expect a fallback to fire" becomes verifiable from the fault-tolerance signals combined with these metrics.
+
+== Internals
+
+The `quarkus-goblin-metrics` module is a plain jar containing one `@ApplicationScoped` bean
+(`io.quarkiverse.goblin.metrics.GoblinMetricsObserver`) that implements the engine's `AssaultObserver` SPI
+(`io.quarkiverse.goblin.AssaultObserver`, default no-op methods). The observer is fed by the engine on every recorded
+assault and on every active-state change, so it has zero coupling to the JAX-RS filters or the client interceptor.

docs/modules/ROOT/pages/release-notes.adoc

@@ -2,6 +2,21 @@
 
 include::../partials/attributes.adoc[]
 
+[[version-0-3]]
+== 0.3.x
+
+[cols="1,1"]
+|===
+|Version |Date
+
+|0.3.0
+|In development
+|===
+
+=== New Features
+
+* Assault metrics and latency histograms via Micrometer / Prometheus (https://github.com/quarkiverse/quarkus-goblin/pull/51[#51]).
+
 [[version-0-2]]
 == 0.2.x
 

integration-tests/README.md

@@ -18,6 +18,7 @@ REST Client calls with latency and exceptions, and `GoblinWebClientIntegrationTe
 | `GoblinIntegrationTest` | Endpoint basics, each assault type (latency, exception, HTTP status, dependency degradation, response body truncate/inflate, response header set/remove), target-level percentage behavior |
 | `GoblinClientAssaultIntegrationTest` | Client-side latency and exception on outgoing REST Client calls (incl. interplay with the target level and isolation from incoming-request assaults) |
 | `GoblinWebClientIntegrationTest` | Client-side latency and exception on outgoing Vert.x WebClient calls armed with `GoblinWebClient.enable(...)`, against the same 8081 test-port endpoint |
+| `GoblinMetricsIntegrationTest` | The optional `quarkus-goblin-metrics` module end-to-end: meters registered with the expected names/tags on the live Prometheus registry (server, REST Client and WebClient sources), settled-step counter/timer values, and the `goblin.active` gauge |
 | `GoblinJsonRPCServiceTest` | The Dev UI JSON-RPC contract (status, toggles, editors, history, Markdown report, response body and response header config self-service) |
 | `AbstractPackageTargetingTest` + `ExcludePackageTargetingTest`, `IncludeNonMatchingPackageTargetingTest`, `IncludeMatchingPackageTargetingTest`, `ExcludeOverridesIncludeTargetingTest` | Package-based targeting via `include-packages` / `exclude-packages` |
 

integration-tests/pom.xml

@@ -29,6 +29,11 @@
             <version>${project.version}</version>
             <scope>provided</scope>
         </dependency>
+        <dependency>
+            <groupId>io.quarkiverse.goblin</groupId>
+            <artifactId>quarkus-goblin-metrics</artifactId>
+            <version>${project.version}</version>
+        </dependency>
         <dependency>
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-rest</artifactId>

integration-tests/src/main/resources/application.properties

@@ -3,4 +3,5 @@ quarkus.goblin.assault.type=latency
 quarkus.goblin.assault.latency.min-milliseconds=100
 quarkus.goblin.assault.latency.max-milliseconds=500
 quarkus.goblin.target.level=100
+quarkus.micrometer.export.prometheus.step=PT1S
 quarkus.rest-client.sample-client.url=http://localhost:8081