Test\Replay\HttpCassette writes a signature for every recorded request (method, URL and body hash), but nothing ever reads it. next() returns only the response half of an interaction, so replay is pure FIFO: the cassette decides what comes back, regardless of what the bridge actually sent.
That leaves half the pipeline uncovered. Contract normalizers and ModelClient payload building can regress without a single replay test noticing. Sending a completely wrong model through the recorded examples of #2437 changes nothing:
// src/platform/src/Bridge/OpenResponses/ModelClient.php
'body' => $this->encodeJsonBody(array_merge($options, ['model' => 'totally-wrong-model'], $payload)),
OK (5 tests, 10 assertions)
A dropped option, a wrong endpoint or a mangled message payload behaves the same way.
The data is already there, so the fix is small: compare the outgoing request against the recorded signature on replay and fail loudly on a mismatch.
Two things to decide:
- Strictness. Failing hard on any mismatch is the point, but it makes every cassette brittle against unrelated payload changes (a new default option would invalidate every cassette at once). An opt-in flag on
CassetteHttpClient, defaulting to on for AbstractBridgeReplayTestCase and off for the examples harness, may be the better balance.
- Granularity. The signature is a hash, so a mismatch can only say "differs", not what differs. Keeping the normalized request body in the cassette (it already is, redacted) allows a readable diff in the failure message, which is what makes such a failure actionable.
Follow-up to #2128, which introduced the scaffolding, and #2437, which puts it in CI as bridge integration testing.
Test\Replay\HttpCassettewrites asignaturefor every recorded request (method, URL and body hash), but nothing ever reads it.next()returns only theresponsehalf of an interaction, so replay is pure FIFO: the cassette decides what comes back, regardless of what the bridge actually sent.That leaves half the pipeline uncovered. Contract normalizers and
ModelClientpayload building can regress without a single replay test noticing. Sending a completely wrong model through the recorded examples of #2437 changes nothing:A dropped option, a wrong endpoint or a mangled message payload behaves the same way.
The data is already there, so the fix is small: compare the outgoing request against the recorded
signatureon replay and fail loudly on a mismatch.Two things to decide:
CassetteHttpClient, defaulting to on forAbstractBridgeReplayTestCaseand off for the examples harness, may be the better balance.Follow-up to #2128, which introduced the scaffolding, and #2437, which puts it in CI as bridge integration testing.