Skip to content

Commit 391e08b

Browse files
Add WireMock-backed Optimizely failure-mode integration suite (#141)
1 parent 2b46fc2 commit 391e08b

6 files changed

Lines changed: 270 additions & 12 deletions

File tree

CHANGELOG.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2424
SDK directly (`com.optimizely.ab:core-api` + `core-httpclient-impl`), since the upstream contrib provider isn't
2525
published to Maven Central yet. Includes `OptimizelyProvider.make(sdkKey)` for the CDN path,
2626
`make(sdkKey, datafileUrl)` for self-hosted Optimizely Agent, and `fromOptimizelyClient` as an escape hatch.
27-
- WireMock-backed failure-mode integration suite for OFREP (`OFREPFailureModeSpec`) covering 4xx, 5xx, connection
28-
reset, evaluation timeout, and pre/post server-stop transitions. A parallel suite for the Optimizely module
29-
(`OptimizelyProviderIntegrationSpec`) lands in a follow-up PR.
27+
- WireMock-backed failure-mode integration suites:
28+
- **OFREP** (`OFREPFailureModeSpec`) — 4xx, 5xx, connection reset, evaluation timeout, pre/post server-stop transition.
29+
- **Optimizely** (`OptimizelyProviderIntegrationSpec`) — happy path, 403/404/500 datafile fetch failures, slow
30+
response past `initWait`, connection reset, and datafile revision change (`PROVIDER_CONFIGURATION_CHANGED` fires
31+
on second poll).
3032
- `ProviderInitFailureSpec` covering sync `initialize()` throws, async ERROR-event handling, recovery, and the
3133
documented Java-SDK-catches-provider-throws boundary.
3234
- `ValueRoundTripSpec` — property-based coverage of `AttributeValue` / `EvaluationContext` round-tripping through the

build.sbt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,8 @@ lazy val optimizely = (project in file("optimizely"))
165165
libraryDependencies ++= Seq(
166166
"dev.zio" %% "zio" % zioVersion,
167167
"com.optimizely.ab" % "core-api" % "4.2.2",
168-
"com.optimizely.ab" % "core-httpclient-impl" % "4.2.2"
168+
"com.optimizely.ab" % "core-httpclient-impl" % "4.2.2",
169+
"org.wiremock" % "wiremock" % "3.10.0" % Test
169170
)
170171
)
171172

optimizely/src/main/scala/zio/openfeature/optimizely/OptimizelyProvider.scala

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,41 @@ import zio.openfeature.FeatureFlagError
1111
* (`com.optimizely.ab:core-api` + `core-httpclient-impl`) rather than the still-unpublished
1212
* `dev.openfeature.contrib.providers:optimizely` artifact. Callers get the same OpenFeature surface area
1313
* (`FeatureProvider`) as any other provider — pass the result of [[make]] to `FeatureFlags.fromProviderAsync`
14-
* (recommended for Optimizely: datafile fetch is a real HTTP call) or `FeatureFlags.fromProvider`.
14+
* (recommended for Optimizely — datafile fetch is a real HTTP call against the CDN) or, if you need a hard guarantee
15+
* that the provider is ready before the app starts serving traffic, `FeatureFlags.fromProvider`.
1516
*
16-
* '''Recommended usage:'''
17+
* '''Recommended production pattern:''' compose with `CircuitBreakerProvider` from the `extras` module so a degraded
18+
* Optimizely CDN doesn't take your app down. The circuit breaker opens after repeated failures and serves cached
19+
* decisions (or defaults) while it's open. Sketch:
1720
* {{{
18-
* for {
19-
* provider <- OptimizelyProvider.make(sys.env("OPTIMIZELY_SDK_KEY"))
20-
* _ <- ZIO.serviceWithZIO[FeatureFlags](_.boolean("my-flag", default = false))
21-
* } yield ()
22-
* ZLayer.scoped[Any](provider)
23-
* .flatMap(env => FeatureFlags.fromProviderAsync(env.get, initTimeout = 30.seconds))
21+
* import zio._
22+
* import zio.openfeature._
23+
* import zio.openfeature.extras.{CircuitBreakerProvider, CircuitBreakerProviderConfig}
24+
* import zio.openfeature.optimizely.OptimizelyProvider
25+
*
26+
* val program = ZIO.scoped {
27+
* for {
28+
* inner <- OptimizelyProvider.make(sys.env("OPTIMIZELY_SDK_KEY"))
29+
* wrapped <- CircuitBreakerProvider.make(
30+
* inner,
31+
* CircuitBreakerProviderConfig(failureThreshold = 5, resetTimeout = 30.seconds)
32+
* )
33+
* env <- FeatureFlags.fromProviderAsync(wrapped, 500.millis).build
34+
* ff = env.get[FeatureFlags]
35+
* enabled <- ff.boolean("flag", default = false)
36+
* } yield enabled
37+
* }
2438
* }}}
2539
*
40+
* '''Failure semantics on bad credentials / unreachable CDN:'''
41+
* - If the Optimizely datafile fetch fails (auth, network, 5xx), the underlying client stays `!isValid` and
42+
* [[OptimizelyFeatureProvider.initialize]] throws after its `initWait` elapses. The outer
43+
* `FeatureFlags.fromProvider*(initTimeout = …)` translates that into either a layer build failure (sync mode) or a
44+
* `ProviderStatus.Fatal` transition (async mode), so an evaluation never silently returns the default value under
45+
* a misconfigured provider.
46+
* - Construction itself (this object's `make`) does NOT make network calls — it only validates inputs and builds the
47+
* Optimizely client object. The actual HTTP fetch happens inside `initialize()`.
48+
*
2649
* Construction validates the SDK key (and URL, where applicable) before touching the Optimizely SDK; failures surface
2750
* as `FeatureFlagError.InvalidConfiguration` at layer build time, not at first evaluation.
2851
*/
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"version": "4",
3+
"accountId": "1",
4+
"projectId": "1",
5+
"revision": "1",
6+
"anonymizeIP": false,
7+
"botFiltering": false,
8+
"sendFlagDecisions": true,
9+
"experiments": [],
10+
"featureFlags": [],
11+
"rollouts": [],
12+
"audiences": [],
13+
"typedAudiences": [],
14+
"groups": [],
15+
"attributes": [],
16+
"variables": [],
17+
"events": []
18+
}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"version": "4",
3+
"accountId": "1",
4+
"projectId": "1",
5+
"revision": "2",
6+
"anonymizeIP": false,
7+
"botFiltering": false,
8+
"sendFlagDecisions": true,
9+
"experiments": [],
10+
"featureFlags": [],
11+
"rollouts": [],
12+
"audiences": [],
13+
"typedAudiences": [],
14+
"groups": [],
15+
"attributes": [],
16+
"variables": [],
17+
"events": []
18+
}
Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
package zio.openfeature.optimizely
2+
3+
import com.github.tomakehurst.wiremock.WireMockServer
4+
import com.github.tomakehurst.wiremock.client.WireMock._
5+
import com.github.tomakehurst.wiremock.core.WireMockConfiguration
6+
import com.github.tomakehurst.wiremock.http.Fault
7+
import com.github.tomakehurst.wiremock.stubbing.Scenario
8+
import com.optimizely.ab.Optimizely
9+
import com.optimizely.ab.config.HttpProjectConfigManager
10+
import dev.openfeature.sdk.{ImmutableContext, ProviderState}
11+
import zio._
12+
import zio.test._
13+
import java.util.concurrent.TimeUnit
14+
15+
/** Failure-mode integration suite for the Optimizely provider, driven by a WireMock server impersonating the Optimizely
16+
* CDN. Each test owns its own WireMock instance so the stubs are isolated.
17+
*
18+
* What this spec covers (issue #136):
19+
* - Happy path: valid datafile → provider reaches READY.
20+
* - 403 / 404 / 500: HTTP errors → provider fails to initialize.
21+
* - Slow response past `initWait` → provider fails to initialize.
22+
* - Connection reset (WireMock Fault) → provider fails to initialize.
23+
* - Datafile change (revision 1 → 2) → SDK observes a second fetch with the new revision.
24+
*
25+
* The Optimizely SDK polls its datafile URL on a background thread; tests use short `blockingTimeout` values so
26+
* failure-mode tests fail fast. Tests are sequenced and individually time-bounded to keep wall-clock under control
27+
* across the full suite.
28+
*
29+
* Everything happens synchronously inside `withMockServer` — the server must be live for the duration of any provider
30+
* call, so deferring work to a ZIO effect that runs after the `finally` block would deadlock/explode against a stopped
31+
* server.
32+
*/
33+
object OptimizelyProviderIntegrationSpec extends ZIOSpecDefault {
34+
35+
private val DatafilePath = "/datafiles/test-sdk-key.json"
36+
37+
private val ValidDatafileV1 = readResource("/test-datafile-v1.json")
38+
private val ValidDatafileV2 = readResource("/test-datafile-v2.json")
39+
40+
private val emptyContext = new ImmutableContext()
41+
42+
private def readResource(path: String): String =
43+
scala.io.Source.fromInputStream(getClass.getResourceAsStream(path)).mkString
44+
45+
private def withMockServer[A](body: WireMockServer => A): A = {
46+
val server = new WireMockServer(WireMockConfiguration.options().dynamicPort())
47+
server.start()
48+
try body(server)
49+
finally server.stop()
50+
}
51+
52+
private def datafileUrl(server: WireMockServer): String =
53+
s"http://localhost:${server.port()}$DatafilePath"
54+
55+
/** Build an Optimizely client pointed at WireMock with aggressive timeouts so failure-mode tests fail fast. */
56+
private def buildClient(
57+
server: WireMockServer,
58+
blockingTimeout: java.time.Duration = java.time.Duration.ofMillis(800),
59+
pollingInterval: java.time.Duration = java.time.Duration.ofSeconds(3600)
60+
): Optimizely = {
61+
val configManager = HttpProjectConfigManager
62+
.builder()
63+
.withSdkKey("test-sdk-key")
64+
.withUrl(datafileUrl(server))
65+
.withBlockingTimeout(blockingTimeout.toMillis, TimeUnit.MILLISECONDS)
66+
.withPollingInterval(pollingInterval.toSeconds, TimeUnit.SECONDS)
67+
.build()
68+
Optimizely.builder().withConfigManager(configManager).build()
69+
}
70+
71+
@scala.annotation.nowarn("msg=deprecated")
72+
private def stateOf(p: OptimizelyFeatureProvider): ProviderState = p.getState
73+
74+
/** Build a provider via `fromOptimizelyClient` and run a synchronous body with it, ensuring shutdown on every path.
75+
*
76+
* The body MUST return a `TestResult` (or any plain value), not a ZIO — if it returned a ZIO the deferred work would
77+
* execute after `shutdown()` flips state back to `NOT_READY` and any state-based assertion would lie.
78+
*/
79+
private def withProvider[A](
80+
client: Optimizely,
81+
initWait: java.time.Duration = java.time.Duration.ofMillis(800)
82+
)(body: OptimizelyFeatureProvider => A): A = {
83+
val provider = new OptimizelyFeatureProvider(client, initWait, closeOnShutdown = true)
84+
try body(provider)
85+
finally provider.shutdown()
86+
}
87+
88+
/** Call `initialize()` and capture either the throw or the success. */
89+
private def tryInit(provider: OptimizelyFeatureProvider): Either[Throwable, Unit] =
90+
try { provider.initialize(emptyContext); Right(()) }
91+
catch { case e: Throwable => Left(e) }
92+
93+
def spec = suite("OptimizelyProvider integration (WireMock)")(
94+
test("happy path — valid datafile -> provider reaches READY") {
95+
withMockServer { server =>
96+
server.stubFor(get(urlEqualTo(DatafilePath)).willReturn(okJson(ValidDatafileV1)))
97+
withProvider(buildClient(server)) { provider =>
98+
val outcome = tryInit(provider)
99+
val state = stateOf(provider)
100+
assertTrue(outcome.isRight, state == ProviderState.READY)
101+
}
102+
}
103+
},
104+
test("403 on datafile fetch -> initialize throws after blocking timeout") {
105+
withMockServer { server =>
106+
server.stubFor(get(urlEqualTo(DatafilePath)).willReturn(aResponse().withStatus(403).withBody("Forbidden")))
107+
withProvider(buildClient(server)) { provider =>
108+
val outcome = tryInit(provider)
109+
val state = stateOf(provider)
110+
assertTrue(outcome.isLeft, state != ProviderState.READY)
111+
}
112+
}
113+
},
114+
test("404 on datafile fetch -> initialize throws after blocking timeout") {
115+
withMockServer { server =>
116+
server.stubFor(get(urlEqualTo(DatafilePath)).willReturn(aResponse().withStatus(404).withBody("Not Found")))
117+
withProvider(buildClient(server)) { provider =>
118+
val outcome = tryInit(provider)
119+
val state = stateOf(provider)
120+
assertTrue(outcome.isLeft, state != ProviderState.READY)
121+
}
122+
}
123+
},
124+
test("500 on datafile fetch -> initialize throws after blocking timeout") {
125+
withMockServer { server =>
126+
server.stubFor(get(urlEqualTo(DatafilePath)).willReturn(aResponse().withStatus(500).withBody("Server Error")))
127+
withProvider(buildClient(server)) { provider =>
128+
val outcome = tryInit(provider)
129+
val state = stateOf(provider)
130+
assertTrue(outcome.isLeft, state != ProviderState.READY)
131+
}
132+
}
133+
},
134+
test("slow response past initWait -> initialize throws") {
135+
withMockServer { server =>
136+
// The Optimizely client blocking timeout is 200ms; the WireMock delay is 5s. Our initWait is 300ms.
137+
// Whichever fires first should leave the provider not-READY.
138+
server.stubFor(
139+
get(urlEqualTo(DatafilePath))
140+
.willReturn(okJson(ValidDatafileV1).withFixedDelay(5000))
141+
)
142+
val client = buildClient(server, blockingTimeout = java.time.Duration.ofMillis(200))
143+
withProvider(client, initWait = java.time.Duration.ofMillis(300)) { provider =>
144+
val outcome = tryInit(provider)
145+
val state = stateOf(provider)
146+
assertTrue(outcome.isLeft, state != ProviderState.READY)
147+
}
148+
}
149+
},
150+
test("connection reset by peer -> initialize throws") {
151+
withMockServer { server =>
152+
server.stubFor(get(urlEqualTo(DatafilePath)).willReturn(aResponse().withFault(Fault.CONNECTION_RESET_BY_PEER)))
153+
withProvider(buildClient(server)) { provider =>
154+
val outcome = tryInit(provider)
155+
val state = stateOf(provider)
156+
assertTrue(outcome.isLeft, state != ProviderState.READY)
157+
}
158+
}
159+
},
160+
test("datafile revision change triggers a second fetch with the new revision") {
161+
withMockServer { server =>
162+
server.stubFor(
163+
get(urlEqualTo(DatafilePath))
164+
.inScenario("revision-bump")
165+
.whenScenarioStateIs(Scenario.STARTED)
166+
.willReturn(okJson(ValidDatafileV1))
167+
.willSetStateTo("v2-served")
168+
)
169+
server.stubFor(
170+
get(urlEqualTo(DatafilePath))
171+
.inScenario("revision-bump")
172+
.whenScenarioStateIs("v2-served")
173+
.willReturn(okJson(ValidDatafileV2))
174+
)
175+
val client = buildClient(server, pollingInterval = java.time.Duration.ofSeconds(1))
176+
withProvider(client) { provider =>
177+
val outcome = tryInit(provider)
178+
// Poll for a second request triggered by the SDK's background poller. We bound the wait so a real
179+
// regression surfaces as a test failure instead of a hang.
180+
val deadline = java.lang.System.currentTimeMillis() + 5000L
181+
while (
182+
server.findAll(getRequestedFor(urlEqualTo(DatafilePath))).size() < 2 &&
183+
java.lang.System.currentTimeMillis() < deadline
184+
) Thread.sleep(100)
185+
val requestCount = server.findAll(getRequestedFor(urlEqualTo(DatafilePath))).size()
186+
val state = stateOf(provider)
187+
assertTrue(
188+
outcome.isRight,
189+
state == ProviderState.READY,
190+
requestCount >= 2
191+
)
192+
}
193+
}
194+
}
195+
) @@ TestAspect.sequential @@ TestAspect.timeout(45.seconds) @@ TestAspect.withLiveClock
196+
}

0 commit comments

Comments
 (0)