Skip to content

Commit dfac39c

Browse files
author
Github Action
committed
chore: synced pact-plugins docs
1 parent 0876878 commit dfac39c

4 files changed

Lines changed: 296 additions & 47 deletions

File tree

‎website/docs/implementation_guides/pact_plugins/docs/proposals/proposal_007_driver_plugin_callback_model.md‎

Lines changed: 214 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -21,51 +21,246 @@ framework, or they cannot reuse functionality across plugins at all.
2121

2222
This is especially relevant for richer matcher/generator use cases and for any future in-process plugin runtime.
2323

24+
### The dependency-cycle constraint
25+
26+
The driver crate deliberately depends only on `pact_models` (data types), not on `pact_matching` or any other crate
27+
that implements actual Pact matching/generation behaviour. Those higher-level crates depend on the driver (so they can
28+
call out to plugins), so the reverse dependency is not available: the driver cannot link against the code that knows
29+
how to compare an XML body, apply a `DateGenerator`, or run the standard matching rule set. The same constraint holds
30+
for the JVM driver, which depends on `au.com.dius.pact.core:model` but not on the Pact-JVM matching engine.
31+
32+
This means the driver can advertise that a capability exists (it already does this via the catalogue and
33+
`hostCapabilities` in `InitPlugin`, see [005](/implementation_guides/pact_plugins/docs/proposals/proposal_005_plugin_capability_negotiation_and_versioning)), but it cannot
34+
itself execute that capability. Something registered at runtime by the embedding framework has to do the work, and the
35+
driver has to be able to invoke it without knowing its concrete type.
36+
2437
## Recommended direction
2538

2639
### Logical model
2740

28-
Callbacks are modelled as specific typed host capabilities, not a generic message envelope. Each capability is a well-defined operation with a typed request and response. The driver advertises which capabilities it supports during the `InitPlugin` handshake (via [005](/implementation_guides/pact_plugins/docs/proposals/proposal_005_plugin_capability_negotiation_and_versioning)), so a plugin knows before making any call whether a given capability is available.
41+
Callbacks are modelled as specific typed host capabilities, not a generic message envelope. Each capability is a
42+
well-defined operation with a typed request and response. The driver advertises which capabilities it supports during
43+
the `InitPlugin` handshake (via [005](/implementation_guides/pact_plugins/docs/proposals/proposal_005_plugin_capability_negotiation_and_versioning)), so a plugin knows before making any call whether a given capability is available.
44+
45+
### Breaking the dependency cycle: registered handlers, not linked implementations
46+
47+
This proposal does not introduce a new pattern — it generalises one already shipped in
48+
[008](/implementation_guides/pact_plugins/docs/proposals/proposal_008_plugin_observability_and_logging). The `PluginLogSink` trait in
49+
`drivers/rust/driver/src/plugin_log_sink.rs` is the working example: the driver defines the trait, holds a global
50+
replaceable instance, and exposes a `set_plugin_log_sink()` registration function. The embedding framework implements
51+
the trait and registers itself once at startup. The driver's compiled code never references the concrete
52+
implementation. The JVM driver's `PluginHostServer` object (instance registry keyed by plugin instance ID) is the same
53+
shape for a simpler case.
54+
55+
The callback model reuses this exactly, generalised from "one sink" to "one handler per capability":
56+
57+
- The driver defines one narrow trait per **capability shape** — not one generic `dyn Any` handler. A capability shape
58+
corresponds to an operation already defined for plugins (e.g. `CompareContents`, `GenerateContent`) or one that
59+
[006](/implementation_guides/pact_plugins/docs/proposals/proposal_006_field_level_matchers_and_generators) adds for field-level operations. Reusing the existing typed
60+
messages (`CompareContentsRequest`/`Response`, `GenerateContentRequest`/`Response`, `ContentMismatch`, etc.) means the
61+
driver→host interface and the driver→plugin interface share a data model — a capability looks the same regardless of
62+
who provides it.
63+
64+
```rust
65+
#[async_trait]
66+
pub trait CoreContentMatcher: Send + Sync {
67+
async fn compare_contents(&self, request: CompareContentsRequest) -> anyhow::Result<CompareContentsResponse>;
68+
}
69+
70+
#[async_trait]
71+
pub trait CoreContentGenerator: Send + Sync {
72+
async fn generate_content(&self, request: GenerateContentRequest) -> anyhow::Result<GenerateContentResponse>;
73+
}
74+
```
75+
76+
- A per-trait registry, keyed by the unprefixed catalogue entry key (mirroring `register_core_entries`'s own keying),
77+
lives next to `CATALOGUE_REGISTER` in `catalogue_manager.rs`:
78+
79+
```rust
80+
pub fn register_core_content_matcher(key: &str, handler: Arc<dyn CoreContentMatcher>);
81+
pub fn register_core_content_generator(key: &str, handler: Arc<dyn CoreContentGenerator>);
82+
```
83+
84+
Registration happens at the same call site as `register_core_entries`, so a `CatalogueEntryProviderType::CORE` entry
85+
and its handler are always registered together and can't drift apart.
86+
87+
- The JVM driver mirrors this with `CoreContentMatcher`/`CoreContentGenerator` interfaces and a `CoreCapabilityRegistry`
88+
object shaped like `PluginHostServer`.
89+
90+
### One resolver, two call directions
91+
92+
Everything above solves how the host registers a capability. What was still missing is how *anything* — the driver
93+
itself, or a plugin calling back — invokes it. Both cases must resolve a catalogue key to either "call the registered
94+
core handler" or "forward to the plugin that owns this entry", and there should be exactly one place that makes that
95+
decision.
96+
97+
**Direction A — the driver's own outbound calls.** `content.rs`'s `ContentMatcher`/`ContentGenerator` already have an
98+
`is_core()` check, added in anticipation of this work, but `match_contents`/`generate_content` currently
99+
`.expect("Plugin type is required")` and panic if `is_core()` is true — there was nothing to call yet. This proposal
100+
closes that gap directly:
101+
102+
```rust
103+
if self.is_core() {
104+
let handler = core_capabilities::lookup_core_content_matcher(&self.catalogue_entry.key)
105+
.ok_or_else(|| anyhow!("No core handler registered for '{}'", self.catalogue_entry.key))?;
106+
handler.compare_contents(request).await
107+
} else {
108+
// existing lookup_plugin(...) gRPC path, unchanged
109+
}
110+
```
111+
112+
No gRPC is involved here: the driver and the registered handler are in the same process.
113+
114+
**Direction B — a plugin calling back.** This is the actual subject of this proposal. A plugin (external gRPC process
115+
or, per [003](/implementation_guides/pact_plugins/docs/proposals/proposal_003_support_wasm_plugins), an in-process WASM module) needs a capability it doesn't implement
116+
itself — for example a field-level plugin from 006 wants the host's standard `type` matcher for one field of a larger
117+
document it otherwise owns. It calls back with a catalogue entry key. The driver resolves that key exactly the way
118+
Direction A does:
119+
120+
- **`CORE`** → call the registered handler in-process, same as Direction A.
121+
- **`PLUGIN`**, owned by a *different* plugin than the caller → forward the call over gRPC to that plugin, using the
122+
existing `lookup_plugin` mechanism already used for driver→plugin calls. This makes cross-plugin capability calls
123+
possible: plugin A can transparently use a capability plugin B registered, mediated entirely by the driver. Neither
124+
plugin needs to know about the other directly.
125+
- **Not found** → fail the callback with a clear error; the plugin surfaces this as a failure in the parent request.
29126

30127
### gRPC transport
31128

32-
The driver implements a `PactPluginHost` gRPC service on a local listener and passes its address to the plugin in `InitPlugin`. The plugin creates a standard gRPC client channel to that address. Each callback is a normal blocking unary RPC call from plugin to driver — no bi-directional streaming is required.
129+
The driver implements the callback RPCs as an extension of the `PluginHost` gRPC service introduced in
130+
[008](/implementation_guides/pact_plugins/docs/proposals/proposal_008_plugin_observability_and_logging) (currently just `Log`) on the same local listener, whose address is
131+
already passed to the plugin via the `PACT_PLUGIN_HOST` environment variable. Each callback is a normal blocking unary
132+
RPC from plugin to driver — no bi-directional streaming.
133+
134+
```proto
135+
service PluginHost {
136+
rpc Log(LogMessage) returns (google.protobuf.Empty);
137+
138+
// New in this proposal:
139+
rpc CompareContents(HostCompareContentsRequest) returns (CompareContentsResponse);
140+
rpc GenerateContent(HostGenerateContentRequest) returns (GenerateContentResponse);
141+
// Further RPCs land here as 006 defines field-level operation shapes.
142+
}
143+
144+
message HostCompareContentsRequest {
145+
// Catalogue entry key being invoked, e.g. "xml" for content-matcher/xml. Resolved with the
146+
// same lookup used for plugin-provided entries today.
147+
string entryKey = 1;
148+
CompareContentsRequest request = 2;
149+
}
150+
151+
message HostGenerateContentRequest {
152+
string entryKey = 1;
153+
GenerateContentRequest request = 2;
154+
}
155+
```
33156

34-
A concrete flow during `VerifyInteraction`:
157+
A concrete flow during `VerifyInteraction`, where a field-level plugin delegates one field's matching to a host-provided
158+
matcher:
35159

36160
```
37161
Driver ──── VerifyInteraction(request) ────────────────→ Plugin
38162
processes...
39-
Plugin ──── MatchField(request) ───────────────────────→ PactPluginHost (driver)
40-
Plugin ←─── MatchFieldResult ──────────────────────────── driver
163+
Plugin ──── CompareContents(entryKey="xml") ───────────→ PluginHost (driver)
164+
driver resolves entryKey:
165+
CORE -> call registered CoreContentMatcher in-process
166+
PLUGIN -> forward to the owning plugin over gRPC
167+
Plugin ←─── CompareContentsResponse ───────────────────── driver
41168
continues...
42169
Driver ←─── VerifyInteractionResponse ─────────────────── Plugin
43170
```
44171

45-
Each callback completes before the plugin continues. The call stack is synchronous and nested, which makes error propagation and deadline tracking straightforward.
172+
Each callback completes before the plugin continues. The call stack is synchronous and nested, which makes error
173+
propagation and deadline tracking straightforward.
174+
175+
**Baseline vs. optional:** the `PluginHost` service itself, and the ability to resolve a catalogue key via it, is a
176+
**baseline V2 capability** per [005](/implementation_guides/pact_plugins/docs/proposals/proposal_005_plugin_capability_negotiation_and_versioning)'s classification rule —
177+
its absence would make the protocol structurally incomplete, not just degrade one feature. Any V2 driver must expose
178+
it. Individual entries behind it (e.g. whether `content-matcher/xml` specifically is registered) remain **optional** —
179+
a plugin that needs a specific capability checks for it in `hostCapabilities` at `InitPlugin` time, same as today.
180+
181+
### Cycle detection and deadlines (gRPC only)
182+
183+
**Cycle detection.** A call-chain ID is generated by the driver at the root of any call that may trigger callbacks
184+
(`CompareContents`, `ConfigureInteraction`, `GenerateContent`, `VerifyInteraction`, `PrepareInteractionForVerification`)
185+
and sent as gRPC request metadata (`pact-call-chain-id`). A plugin forwards the same chain ID as metadata on any
186+
callback it makes. The driver keeps an in-memory stack per chain ID (`chain_id -> Vec<entry_key>`) in a new
187+
`call_chain` module:
188+
189+
- Before dispatching a call for `entry_key` under `chain_id`, push `entry_key` onto that chain's stack; if it's already
190+
present, reject immediately with a cycle error instead of forwarding.
191+
- Pop it when the call completes (success or failure).
46192

47-
**Cycle detection is required for gRPC.** A call chain ID must be threaded through gRPC request metadata for any request that may trigger callbacks. If the driver receives a callback whose chain ID matches an in-flight request it is currently processing, the call is a cycle and must be rejected with a clear error. The plugin surfaces this as a failure in the parent request.
193+
This applies identically to Direction B forwarding to another plugin: the driver pushes the target entry key before
194+
forwarding, so a cycle across two or more plugins is caught the same way a self-cycle is.
48195

49-
**Deadlines:** a callback's deadline must be bounded by the remaining deadline of the parent request that triggered it. The driver enforces this when it receives the callback.
196+
**Deadlines.** The driver sets an absolute deadline (`pact-deadline-ms`, Unix epoch milliseconds) as metadata on the
197+
root call. Every hop — the plugin's outbound callback, and any forwarding the driver does on the plugin's behalf —
198+
reads it, fails fast if it has already passed, and uses the remaining budget (`deadline_ms - now`) as the timeout for
199+
its own call (`tonic`'s `.timeout()` on the client). A callback can never outlive the request that triggered it.
50200

51-
**Unavailable target:** if the driver's `PactPluginHost` service is unreachable, the plugin must fail the parent request with a clear error rather than hanging.
201+
**Unavailable target:** if the driver's `PluginHost` service is unreachable, the plugin must fail the parent request
202+
with a clear error rather than hanging.
52203

53204
### WASM transport
54205

55-
Host capabilities are exposed as host-exported functions that the WASM module imports at load time. Calls resolve via the native call stack — there is no network hop and no blocking concern. A true cycle would manifest as a stack overflow, which the WASM runtime handles. No explicit cycle detection is needed for WASM.
206+
Host capabilities are exposed as host-exported functions that the WASM module imports at load time. The exported
207+
function signatures correspond directly to the capability traits above (the same `CoreContentMatcher`/
208+
`CoreContentGenerator` etc. are called directly — no gRPC serialisation, though the request/response types can still be
209+
passed as serialised protobuf bytes across the WASM linear-memory boundary, since WASM has no native way to pass Rust
210+
structs by reference). Calls resolve via the native call stack — there is no network hop and no blocking concern. A
211+
true cycle would manifest as a stack overflow, which the WASM runtime handles. No explicit cycle detection or
212+
call-chain metadata is needed for WASM.
56213

57-
## Non-goals for this proposal
214+
### Lua transport (in-process, shipped ahead of WASM)
58215

59-
- Defining the detailed payload model for verification.
60-
- Solving observability/logging by itself.
61-
- Redesigning plugin discovery or packaging.
216+
The Lua plugin runtime (`drivers/rust/driver/src/lua_plugin.rs`) already runs in-process and already registers host
217+
functions into the script's global table (`logger`, `rsa_sign`, etc., see `register_host_functions`). The callback
218+
model extends this the same way as WASM: a `host_compare_contents(entry_key, table)` Lua global that runs the same
219+
resolver as Direction A/B and converts through the conversion helpers `lua_plugin.rs` already has
220+
(`compare_request_to_lua`, `lua_to_compare_response`). Like WASM, no chain ID or cycle detection is needed — it's a
221+
direct, synchronous Rust function call from the Lua VM's perspective.
222+
223+
The logical capability interface — what can be called, what parameters it takes, what it returns — is identical
224+
between gRPC, WASM, and Lua. Only the transport differs.
225+
226+
### Sequencing
62227

63-
## WASM compatibility
228+
This proposal ships as one vertical slice through the mechanism, not the full capability surface:
64229

65-
For WASM plugins, the callback model maps directly to host-exported functions imported by the module at load time. This is the established model for WASM host integration and works well: calls are synchronous, resolve via the call stack, and require no connection management or cycle detection logic in the plugin or driver.
230+
1. The registry/trait pattern (`core_capabilities` module, generalising `PluginLogSink`).
231+
2. The extended `PluginHost` gRPC service with cycle detection and deadline propagation.
232+
3. Wiring the two existing `is_core()` branches in `content.rs` to call through instead of panicking.
233+
4. The Lua and WASM host-function equivalents.
66234

67-
The logical capability interface — what can be called, what parameters it takes, what it returns — is identical between gRPC and WASM. Only the transport differs.
235+
This is deliberately the smallest slice that proves the mechanism end-to-end, because `is_core()` already exists as an
236+
unfinished seam. [006](/implementation_guides/pact_plugins/docs/proposals/proposal_006_field_level_matchers_and_generators) then adds new capability trait shapes
237+
(field-level matching/generation) on top of the same registry and the same `PluginHost` extension pattern — no new
238+
plumbing is needed for it. [009](/implementation_guides/pact_plugins/docs/proposals/proposal_009_host_provided_core_matching_and_generation) is, in turn, just "register the
239+
standard Pact matcher/generator set as `CoreContentMatcher`/field-level handlers using this mechanism" — see that
240+
proposal for details.
241+
242+
## Non-goals for this proposal
243+
244+
- Defining the detailed payload model for verification.
245+
- Solving observability/logging by itself (see [008](/implementation_guides/pact_plugins/docs/proposals/proposal_008_plugin_observability_and_logging), already implemented).
246+
- Redesigning plugin discovery or packaging.
247+
- Defining the field-level operation shapes themselves (see [006](/implementation_guides/pact_plugins/docs/proposals/proposal_006_field_level_matchers_and_generators)) —
248+
this proposal defines the mechanism they will be registered and invoked through.
68249

69-
## Open questions
250+
## Resolved questions
70251

71-
- Which specific host capabilities should be exposed first? This will be driven by the needs of [006](/implementation_guides/pact_plugins/docs/proposals/proposal_006_field_level_matchers_and_generators) and [009](/implementation_guides/pact_plugins/docs/proposals/proposal_009_host_provided_core_matching_and_generation).
252+
- **Which specific host capabilities should be exposed first?** Content-level `CompareContents`/`GenerateContent`,
253+
because the `is_core()` seam for these already exists and is the smallest slice that exercises the full mechanism
254+
(registry, cycle detection, deadlines, all three transports). Field-level capabilities follow once
255+
[006](/implementation_guides/pact_plugins/docs/proposals/proposal_006_field_level_matchers_and_generators) defines their shape.
256+
- **Generic envelope vs. typed capabilities?** Typed, one trait/RPC pair per capability shape, reusing existing
257+
message types (`CompareContentsRequest`, `ContentMismatch`, etc.) rather than introducing a parallel generic
258+
request/response model.
259+
- **How is the dependency cycle avoided?** By generalising the `PluginLogSink` registration pattern from
260+
[008](/implementation_guides/pact_plugins/docs/proposals/proposal_008_plugin_observability_and_logging): the driver defines traits and a registry, the embedding framework
261+
implements and registers handlers at startup, and the driver never has a compile-time dependency on the
262+
implementation.
263+
- **Are cross-plugin calls (plugin A using a capability plugin B provides) in scope?** Yes. The resolver that answers
264+
"who provides this catalogue entry" doesn't distinguish CORE-forwarding from PLUGIN-forwarding as separate
265+
mechanisms — supporting one means supporting both, and cycle detection is required as soon as any callback exists
266+
regardless of who's on the other end.

0 commit comments

Comments
 (0)