You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FFE docs: add in-memory provider testing guidance per language (#36198)
* feature_flags: add shared testing section to server and client _index
Introduces a 'Testing with in-memory providers' section on both the
server-side and client-side feature flag landing pages. Explains that the
Datadog provider talks to Datadog's backend and is not appropriate for
unit tests, and directs readers to per-language pages for concrete
examples.
* feature_flags: add Testing section to Go server-side page
Teaches customers to swap DatadogProvider for OpenFeature's memprovider
in unit tests. Uses named clients via SetNamedProviderAndWait so t.Parallel()
is safe, and shows the ContextEvaluator pointer-to-func pattern.
* feature_flags: add Testing section to Java server-side page
Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
(Flag builder form) in JUnit 5 tests. Emphasizes OpenFeatureAPI singleton
reset via shutdown() in @AfterEach and mentions Spring Boot test setup.
* feature_flags: add Testing section to Python server-side page
Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in pytest. Uses a function-scoped fixture with api.shutdown() teardown to
prevent global API singleton leakage between tests.
* feature_flags: add Testing section to Node.js server-side page
Teaches customers to swap DatadogProvider for OpenFeature's
TypedInMemoryProvider in Vitest tests. Shows OpenFeature.close() teardown
and per-test putConfiguration() to keep tests isolated from each other.
* feature_flags: add Testing section to .NET server-side page
Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in xUnit via IAsyncLifetime. Notes cross-framework applicability (NUnit,
MSTest) and the WebApplicationFactory integration path for ASP.NET Core.
* feature_flags: add Testing section to Ruby server-side page
Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in RSpec via an around hook that captures and restores the prior provider.
Notes the Ruby SDK's plain-value flag hash (no variants) and the add_flag
mutation API.
* feature_flags: add Testing section to JavaScript client-side page
Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
from @openfeature/web-sdk in Vitest. Calls out the required flag shape
(variants, defaultVariant, disabled) and the sync evaluation model.
* feature_flags: add Testing section to React client-side page
Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in tests. Keeps the section minimal per the client-testing style: short
paragraph plus a 10-line snippet, no RTL or component harness.
* feature_flags: add Testing section to Android client-side page
Kotlin/Android OpenFeature SDK does not ship an InMemoryProvider, so the
section includes a ~30-line FakeProvider stub and walks through setting
it up under runTest from kotlinx-coroutines-test.
* feature_flags: add Testing section to iOS client-side page
OpenFeature Swift SDK ships no InMemoryProvider and has no contrib package,
so the section includes a ~50-line InMemoryTestProvider FeatureProvider
stub. Uses async setProviderAndWait and clearProviderAndWait for teardown.
* feature_flags: apply upstream OpenFeature SDK verification fixes
Corrections from per-language expert review against current upstream SDKs:
- Go: ContextEvaluator is already *func(...); drop the extra cast so the
snippet compiles (the prior form produced a **func(...) conversion error).
- Python: context_evaluator returns a FlagResolutionDetails, not a bare
variant name. Fix prose wording.
- .NET: InMemoryProvider's mutation method is UpdateFlagsAsync, not
UpdateFlags. Fix prose wording.
- JavaScript (client) and React: swap the deprecated InMemoryProvider for
TypedInMemoryProvider (upstream marks the former @deprecated) and
mention OpenFeatureTestProvider as the idiomatic React component-test
wrapper.
* feature_flags: frame iOS testing around OpenFeature and dd-openfeature-provider-swift
Points readers at the Datadog OpenFeature bridge (dd-openfeature-provider-swift)
as the production path for OpenFeature on iOS. Tests swap the bridge's
DatadogProvider for a small custom FeatureProvider because the upstream
OpenFeature Swift SDK does not ship an in-memory provider.
* feature_flags: fix iOS test snippet against OpenFeature Swift SDK 0.3.0
Verification caught six compile errors in the InMemoryTestProvider stub:
- initialize/onContextSet are 'async throws', not Future<Void, Never>
- observe() returns AnyPublisher<ProviderEvent?, Never> (optional event)
- ProviderEvent.ready has no associated value; use .ready not .ready(nil)
- Reason case is .staticReason, not .static ('static' is reserved)
- OpenFeatureAPI.shared.clearProvider() is sync and does not exist as
clearProviderAndWait() in 0.3.0
Also update the prose callout to match.
* feature_flags: clarify client-side provider fetches flags from Datadog's CDN
Client-side SDKs reach Datadog's CDN to retrieve flag assignments, not
the Remote Configuration backend used by server-side SDKs. Updates the
testing-section callouts on the shared client _index and on each
per-platform client page so the language is accurate.
* feature_flags: soften testing guidance to present in-memory as one option
In-memory providers are one testing approach; running the real Datadog
provider against a dedicated test environment is equally valid. Updates
the shared _index sections and each per-language page so they present
both options up front, then proceed with the in-memory example as
'this section covers...' rather than 'do not use the Datadog provider'.
* feature_flags: add OpenFeature usage section to iOS page
Adds a new 'Use via OpenFeature' section covering install of
dd-openfeature-provider-swift, OpenFeatureAPI setup, evaluation context,
typed flag evaluation, and flag details. Existing FlagsClient-based
sections are unchanged. Includes a 'not yet production-ready' callout
since the bridge package is still in development.
* feature_flags: note getIntegerValue returns Int64 in iOS OpenFeature section
* Update content/en/feature_flags/client/_index.md
Co-authored-by: Tyler Potter <tyler.potter@datadoghq.com>
* feature_flags/android: drop ProviderReady emit, note kotlin-sdk#226
Address review from @typotter:
- Remove events.emit(ProviderReady) from FakeProvider.initialize; the SDK
emits ProviderReady after initialize returns.
- Soften the InMemoryProvider wording and link to open-feature/kotlin-sdk#226,
where upstream in-memory provider support is tracked.
* feature_flags/server: mirror client step 3 rewording
Apply @typotter's step 3 rewrite from client/_index.md to server/_index.md
for consistency: convey that units under test get an injected OpenFeature
client backed by the InMemoryProvider.
* 1 picker only
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* remove dup from server
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* split sentence
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* action verb
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* with of
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* action
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* comma dotnet
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* comma nodejs
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* fast no promises
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* fast server no promises
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* ios bridge
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* these
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
* Apply suggestions from code review
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
---------
Co-authored-by: Tyler Potter <tyler.potter@datadoghq.com>
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
-**Integration tests**: Point `DatadogProvider` at a dedicated test environment and control flag values from the Datadog UI. This exercises the real provider end-to-end, including the CDN-delivered flag assignments.
29
+
-**Unit tests**: Swap `DatadogProvider` for OpenFeature's standard `InMemoryProvider` (or an equivalent test stub, where no in-memory provider is available in the language) and set flag values directly in test code. This keeps tests hermetic and offline.
30
+
31
+
This section covers the in-memory approach. Because the OpenFeature API is designed to make providers swappable at runtime, your application code does not change — only the provider registered during test setup.
32
+
33
+
A typical test follows this pattern:
34
+
35
+
1. Build a map of flag keys to variants in your test setup.
36
+
2. Register an `InMemoryProvider` with that map through the OpenFeature API.
37
+
3. Call the OpenFeature client in the units being tested. The `InMemoryProvider` returns the flag assignments configured at test setup.
38
+
4. Reset the provider in test teardown to avoid cross-test state leakage.
39
+
40
+
See your platform's SDK page (select from the top of this page) for a concrete test example.
You can test against a dedicated Datadog test environment with the real Datadog provider, or swap it for an in-memory `FeatureProvider` to control flag values directly in test code. This section shows the in-memory approach, which keeps tests hermetic and offline. The upstream OpenFeature Kotlin SDK does not ship an [`InMemoryProvider`][3], so tests use a small custom `FeatureProvider`. The example below replaces `OpenFeatureAPI`'s provider — if your production code uses the Datadog `FlagsClient` wrapper directly, your test should assert through the same `OpenFeatureAPI` client the wrapper uses, not `FlagsClient`.
437
+
438
+
Add `kotlinx-coroutines-test` to your test configuration (the SDK's `initialize` is a `suspend` function):
`OpenFeatureAPI` is a process-wide singleton, so reset it between test classes if tests share a JVM. Wrap `setProviderAndWait` in `runTest { ... }` — it cannot be called from a plain `@Before` method because it is `suspend`.
Copy file name to clipboardExpand all lines: content/en/feature_flags/client/ios.md
+173Lines changed: 173 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -253,6 +253,108 @@ print(details.error) // The error that occurred during evaluation, if any
253
253
254
254
Flag details may help you debug evaluation behavior and understand why a user received a given value.
255
255
256
+
## Use with OpenFeature
257
+
258
+
The examples above use Datadog's `FlagsClient` API directly. If you prefer the [OpenFeature](https://openfeature.dev/) standard API, Datadog ships an OpenFeature provider for iOS that wraps `FlagsClient` and exposes it through `OpenFeatureAPI.shared`. The same flag data is served through either surface; pick whichever API fits your app.
259
+
260
+
<divclass="alert alert-warning">The iOS OpenFeature bridge (<ahref="https://github.com/DataDog/dd-openfeature-provider-swift"><code>dd-openfeature-provider-swift</code></a>) is in development and not recommended for production use. Use the <code>FlagsClient</code> API shown above for production workloads; use this section to prototype OpenFeature integrations or to structure tests around the OpenFeature API.</div>
261
+
262
+
### Install the OpenFeature provider
263
+
264
+
Add `dd-openfeature-provider-swift` to your `Package.swift`:
Link the `DatadogOpenFeatureProvider` product to your app target. The bridge depends on OpenFeature Swift SDK 0.3.0.
271
+
272
+
### Initialize OpenFeature
273
+
274
+
Initialize Datadog and enable Flags as shown in [Initialize the SDK](#initialize-the-sdk). Then create a `DatadogProvider` and register it with `OpenFeatureAPI.shared`:
The `targetingKey` is the randomization subject for percentage rollouts — the same key always receives the same variant for a given flag.
318
+
319
+
### Evaluate flags
320
+
321
+
Retrieve the global OpenFeature client and call the typed getters:
322
+
323
+
{{< code-block lang="swift" >}}
324
+
let client = OpenFeatureAPI.shared.getClient()
325
+
326
+
let isNewCheckoutEnabled = client.getBooleanValue(key: "checkout.new", defaultValue: false)
327
+
328
+
let theme = client.getStringValue(key: "ui.theme", defaultValue: "light")
329
+
330
+
let maxItems = client.getIntegerValue(key: "cart.items.max", defaultValue: 20)
331
+
332
+
let priceMultiplier = client.getDoubleValue(key: "pricing.multiplier", defaultValue: 1.0)
333
+
334
+
let config = client.getObjectValue(
335
+
key: "ui.config",
336
+
defaultValue: Value.structure([
337
+
"color": Value.string("#00A3FF"),
338
+
"fontSize": Value.integer(14)
339
+
])
340
+
)
341
+
{{< /code-block >}}
342
+
343
+
Evaluations are synchronous and safe to perform on the main thread — they read from the SDK's local cache and do not make network requests. Note that `getIntegerValue` returns `Int64`; cast to `Int` at the call site if needed.
344
+
345
+
### Flag evaluation details
346
+
347
+
Use the `get<Type>Details` methods when you need the reason, variant, or any evaluation error in addition to the value:
348
+
349
+
{{< code-block lang="swift" >}}
350
+
let details = client.getStringDetails(key: "paywall.layout", defaultValue: "control")
351
+
352
+
print(details.value) // Evaluated value
353
+
print(details.variant) // Variant name, if applicable
354
+
print(details.reason) // Reason (for example: "TARGETING_MATCH" or "DEFAULT")
355
+
print(details.errorCode) // Error code, if evaluation failed
356
+
{{< /code-block >}}
357
+
256
358
## Advanced configuration
257
359
258
360
The `Flags.enable()` API accepts optional configuration with options listed below.
@@ -288,6 +390,77 @@ Flags.enable(with: config)
288
390
`customFlagsHeaders`
289
391
: Sets additional HTTP headers to attach to requests made to `customFlagsEndpoint`. It can be useful for authentication or routing when using your own flags service.
290
392
393
+
## Testing
394
+
395
+
The examples above use Datadog's `FlagsClient` API directly. If you prefer to drive feature flags through the [OpenFeature](https://openfeature.dev/) standard API, Datadog ships an OpenFeature bridge for iOS at [dd-openfeature-provider-swift](https://github.com/DataDog/dd-openfeature-provider-swift). Use the bridge's `DatadogProvider` in production, and for code-controlled flag values in tests, substitute an in-memory provider.
396
+
397
+
You can test against a dedicated Datadog test environment with the real `DatadogProvider`, or swap it for an in-memory `FeatureProvider` to control flag values directly in test code. This section shows the in-memory approach, which keeps tests hermetic and offline. The OpenFeature Swift SDK does not ship an `InMemoryProvider`, so tests use a small custom `FeatureProvider` instead.
398
+
399
+
{{< code-block lang="swift" >}}
400
+
import Combine
401
+
import OpenFeature
402
+
import XCTest
403
+
@testable import MyApp
404
+
405
+
// Minimal in-memory provider for tests. Copy into your test target.
406
+
final class InMemoryTestProvider: FeatureProvider {
407
+
var hooks: [any Hook] = []
408
+
var metadata: ProviderMetadata = Metadata(name: "in-memory-test")
409
+
private let subject = CurrentValueSubject<ProviderEvent?, Never>(.ready)
`OpenFeatureAPI.shared` is a global singleton, so call `clearProvider()` in `tearDown` to prevent one test's flags from leaking into another. `setProviderAndWait(provider:)` is `async` and does not throw, so no `try` is required.
You can test against a dedicated Datadog test environment with the real `DatadogProvider`, or swap it for OpenFeature's `InMemoryProvider` to control flag values directly in test code. This section shows the in-memory approach, which keeps tests hermetic and offline. `InMemoryProvider` is exported directly from `@openfeature/web-sdk`, so no additional dependency is required.
205
+
206
+
Unlike the server-side SDK, the Web SDK evaluates flags synchronously after initialization. Still `await``setProviderAndWait` once in `beforeEach` to ensure the provider is ready.
207
+
208
+
{{< code-block lang="javascript" >}}
209
+
import { beforeEach, afterAll, expect, test } from 'vitest';
210
+
import { OpenFeature, TypedInMemoryProvider } from '@openfeature/web-sdk';
The Web SDK flag shape requires `variants`, `defaultVariant`, and `disabled`. Omitting any of these fails TypeScript compilation; at runtime, evaluating an unknown flag key returns the supplied default. Prefer `TypedInMemoryProvider` over the deprecated `InMemoryProvider` for type-checked flag configurations. The same test pattern works with Jest + jsdom; swap the `vitest` imports for `@jest/globals` and add `jest-environment-jsdom` to your project.
You can test against a dedicated Datadog test environment with the real `DatadogProvider`, or swap it for OpenFeature's `TypedInMemoryProvider` to control flag values directly in test code. This section shows the in-memory approach, which keeps tests hermetic and offline. `TypedInMemoryProvider` is exported from `@openfeature/web-sdk`; install it as a development dependency and register it before rendering components under test:
299
+
300
+
{{< code-block lang="javascript" >}}
301
+
import { OpenFeature } from '@openfeature/react-sdk';
302
+
import { TypedInMemoryProvider } from '@openfeature/web-sdk';
The Web SDK flag shape requires `variants`, `defaultVariant`, and `disabled`. Use `setProviderAndWait` (not `setProvider`) to avoid suspense races when the test renders flag-gated components immediately. For component tests that mount a React tree, `@openfeature/react-sdk` also exports an `OpenFeatureTestProvider` component that wraps children with an in-memory provider — see the [OpenFeature React SDK docs](https://openfeature.dev/docs/reference/technologies/client/web/react) for details.
Copy file name to clipboardExpand all lines: content/en/feature_flags/server/_index.md
+18Lines changed: 18 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -70,6 +70,24 @@ DD_METRICS_OTEL_ENABLED=true
70
70
71
71
<divclass="alert alert-info">Set <code>DD_METRICS_OTEL_ENABLED=true</code> to enable flag evaluation metrics. Without this, the SDK does not emit metrics for flag evaluations. When enabled, each evaluation records a <code>feature_flag.evaluations</code> counter metric tagged with the flag key, result variant, and evaluation reason.</div>
72
72
73
+
## Testing with in-memory providers
74
+
75
+
Datadog supports these testing approaches:
76
+
77
+
-**Integration tests**: Point `DatadogProvider` at a dedicated test environment and control flag values from the Datadog UI. This exercises the real provider end-to-end, including Remote Configuration delivery.
78
+
-**Unit tests**: Swap `DatadogProvider` for OpenFeature's standard `InMemoryProvider` (or an equivalent test stub, where no in-memory provider is available in the language) and set flag values directly in test code. This keeps tests hermetic and offline.
79
+
80
+
This section covers the in-memory approach. Because the OpenFeature API is designed to make providers swappable at runtime, your application code does not change — only the provider registered during test setup.
81
+
82
+
A typical test follows this pattern:
83
+
84
+
1. Build a map of flag keys to variants in your test setup.
85
+
2. Register an `InMemoryProvider` with that map through the OpenFeature API.
86
+
3. Call the OpenFeature client in the units being tested. The `InMemoryProvider` returns the flag assignments configured at test setup.
87
+
4. Reset the provider in test teardown to avoid cross-test state leakage.
88
+
89
+
See your language's SDK page (select from the top of this page) for a concrete test example.
0 commit comments