afterScenario hook: match failure does not fail the scenario (need optional strict mode) #2699
Description
Summary
I'm running end-to-end (E2E) validations using the afterScenario hook in Karate, where I re-call a .feature file to validate that a business action (such as updating a preferred language) was propagated correctly to a backend system.
These E2E checks should fail the scenario if there's a mismatch, but currently, even if a match fails inside the afterScenario-invoked .feature, the main scenario still passes, and only a WARN log is printed.
Use Case
In my test pipeline, I want these validations to run only in certain environments (e.g., on deploy to PRE), so I'm triggering them from afterScenario.js conditionally.
Example:
```javascript
function() {
if (karate.env != 'pre') return;
var result = karate.call('classpath:preferredLanguage.feature', {
testRequestBody: karate.request.body
});
// Expected: fail scenario if validation fails
if (!result.pass) {
karate.fail('E2E validation failed: expected ' + result.expected + ', got ' + result.actual);
}
}
Inside preferredLanguage.feature:
* def lang = testRequestBody.preferred_language
* def passed = response.code == lang
* def result = { pass: '#(passed)', expected: '#(lang)', actual: '#(response.code)' }
* return result
Problem
Even when the match fails inside the .feature, the failure is logged as:
WARN com.intuit.karate - afterScenario hook failed: match failed: EQUALS
$ | not equal (STRING:STRING)
'en'
'enrt'
But the test scenario still passes.
Suggested Improvement
Could Karate support a mode like:
javascript
Copy
Edit
karate.configure('strictAfterScenario', true)
So that any uncaught failure in afterScenario causes the main scenario to fail?
Alternatively, should this behavior be documented more explicitly as “by design”?
Thank you!
Thanks for all the amazing work on Karate 🙏 — this framework has greatly improved our E2E testing pipeline, and we’d love to tighten this behavior for specific CI stages.