Skip to content

Add OFREP failure-mode spec and ProviderInitFailureSpec#142

Merged
EtaCassiopeia merged 2 commits into
mainfrom
feat/failure-mode-tests
May 13, 2026
Merged

Add OFREP failure-mode spec and ProviderInitFailureSpec#142
EtaCassiopeia merged 2 commits into
mainfrom
feat/failure-mode-tests

Conversation

@EtaCassiopeia
Copy link
Copy Markdown
Owner

@EtaCassiopeia EtaCassiopeia commented May 13, 2026

Summary

Implements #124 (B1 — OFREP WireMock failure stubs) and #125 (B2 — ProviderInitFailureSpec) from the 1.0-readiness milestone.

Stacked. Base branch is feat/typed-errors-and-ofrep-validation (#139). This branch also merges in feat/init-timeout-and-sync-verify (#138) because both A1 (init timeout) and A4 (typed errors / classifier) are dependencies. The diff against #139 shows the merge of #138 plus the two new specs. Once #138 and #139 land in main, retarget to main.

B1 — OFREP failure-mode integration suite

New ofrep/src/test/scala/zio/openfeature/ofrep/OFREPFailureModeSpec.scala with 5 WireMock-backed tests covering the FeatureFlags layer (not just direct provider calls — the typed-error classifier from #123 only fires there). Findings, all encoded as assertions:

  • HTTP 401 / 5xx: the OFREP contrib provider catches the response internally and returns a ProviderEvaluation with errorCode populated. The ZIO FeatureFlags layer surfaces this as a successful FlagResolution whose errorCode is set — not as a typed Unauthorized error. Operators alert on resolution.errorCode.isDefined. This is documented in the spec preamble so the contract is clear.
  • Connection reset (WireMock Fault.CONNECTION_RESET_BY_PEER) and connection refused (server stopped mid-test): either the contrib provider catches the IOException and returns errorCode, or the throw escapes and the classifier maps it (Unreachable for UnknownHostException/ConnectException, ProviderError as fallback). Both shapes are accepted — the assertion proves the failure doesn't become a silent default-value evaluation.
  • Evaluation timeout (200 ms timeout, 3 s server delay): surfaces as Left(FeatureFlagError.ProviderError(TimeoutException)). Asserted exactly.

The pre-stop / post-stop test also proves the FeatureFlags layer flips behaviour live without process restart, which is important for OFREP-backed apps where the upstream may transiently flap.

B2 — ProviderInitFailureSpec

New core/src/test/scala/zio/openfeature/ProviderInitFailureSpec.scala complementing the existing ProviderInitHardeningSpec (which already covers issue cases 2/3/4). This spec adds the missing cases:

  • Case 1: sync initialize() that throws synchronously → layer build fails with the thrown exception (message preserved).
  • Case 5: async provider fires PROVIDER_ERROR after init → status reflects Error, evaluations fail with ProviderNotReady(Error).
  • Case 6: async provider recovers ERROR → READY → evaluations succeed.
  • Case 7: documents the boundary — when an evaluation method on the underlying provider throws an UnknownHostException, the OpenFeature Java SDK catches it and returns a FlagEvaluationDetails with errorCode populated; the typed Unreachable does not fire on this path. The classifier IS still exercised end-to-end via the OFREP failure suite above (where the throw originates outside the Java SDK's catch), and unit-tested directly in FeatureFlagErrorSpec ([A4] Classify auth/network failures: add Unauthorized/Unreachable to FeatureFlagError #123). This test pins the boundary so a future refactor that changes it shows up here.

Three inline test providers (ThrowingInitProvider, EventDriverProvider, ThrowingEvalProvider) keep the spec free of testkit dependencies — core can't depend on testkit.

Closes #124
Closes #125

@EtaCassiopeia EtaCassiopeia added area:core core module area:ofrep ofrep module labels May 13, 2026
@EtaCassiopeia EtaCassiopeia added this to the 1.0-readiness milestone May 13, 2026
@EtaCassiopeia EtaCassiopeia added area:tests test suite kind:test test-only change labels May 13, 2026
@EtaCassiopeia EtaCassiopeia force-pushed the feat/typed-errors-and-ofrep-validation branch from 2f53550 to ba38ac2 Compare May 13, 2026 15:59
Base automatically changed from feat/typed-errors-and-ofrep-validation to main May 13, 2026 16:03
@EtaCassiopeia EtaCassiopeia force-pushed the feat/failure-mode-tests branch from b13ad71 to 1a3a5ec Compare May 13, 2026 16:28
@EtaCassiopeia EtaCassiopeia merged commit 4be71ea into main May 13, 2026
1 check passed
@EtaCassiopeia EtaCassiopeia deleted the feat/failure-mode-tests branch May 13, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core core module area:ofrep ofrep module area:tests test suite kind:test test-only change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[B2] Add ProviderInitFailureSpec in core/ exercising the new typed errors [B1] Extend OFREP WireMock spec with 4xx/5xx/timeout/connection-reset stubs

1 participant