Skip to content

feat: In-memory provider (#208)#226

Open
mstepien wants to merge 3 commits into
open-feature:mainfrom
mstepien:208-in-memory-provider
Open

feat: In-memory provider (#208)#226
mstepien wants to merge 3 commits into
open-feature:mainfrom
mstepien:208-in-memory-provider

Conversation

@mstepien
Copy link
Copy Markdown

@mstepien mstepien commented Apr 3, 2026

InMemoryProvider, Gherkin based tests

This PR introduces the InMemoryProvider for the OpenFeature Kotlin SDK and validates its implementation using the official OpenFeature specification.

  • Adds the InMemoryProvider for managing and evaluating flags entirely in memory, including ContextEvaluator and Flag models.
  • Links the open-feature/spec repository as a Git submodule to utilize the official cross-platform test suite.
  • Adds End-to-End (E2E) testing suite (EvaluationSteps, GherkinSpecTest) using Cucumber to verify the new provider adheres to the OpenFeature Gherkin specification. Evaluation targets evaluation.feature file and ignores evaluation_v2.feature.
  • Updates the GitHub Actions ci.yaml workflow to ensure Git submodules are checked out during CI test runs.

Related Issues

Fixes #208

Notes

Because this PR introduces the open-feature/spec as a Git submodule, developers pulling or checking out this branch locally may need to initialize submodules by running:

git submodule update --init --recursive

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an InMemoryProvider implementation for the Kotlin SDK, along with supporting classes like Flag and ContextEvaluator. It also integrates Gherkin-based E2E tests to ensure compliance with the OpenFeature specification. Feedback focuses on improving thread safety for the provider's internal state, ensuring type safety during flag evaluation (especially when using custom evaluators), resolving nullability mismatches in provider metadata, and enforcing invariants in the Flag data class to prevent invalid configurations.

@mstepien mstepien force-pushed the 208-in-memory-provider branch 3 times, most recently from d02500b to e9af646 Compare April 3, 2026 11:59
@mstepien mstepien marked this pull request as ready for review April 3, 2026 15:08
@dd-oleksii
Copy link
Copy Markdown
Contributor

This is user-facing API change, so the PR name should use feat: prefix (test: is for test-only changes)

@mstepien mstepien changed the title test: In-memory provider (#208) feat: In-memory provider (#208) Apr 7, 2026
Copy link
Copy Markdown
Contributor

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid implementation with a clean InMemoryProvider and a nice Flag/Builder API. Nothing major from my pass outside of what has already been flagged/is in discussion.

Comment thread kotlin-sdk/src/jvmTest/kotlin/dev/openfeature/kotlin/sdk/e2e/EvaluationSteps.kt Outdated
Comment thread kotlin-sdk/src/jvmTest/kotlin/dev/openfeature/kotlin/sdk/e2e/EvaluationSteps.kt Outdated
@mstepien mstepien force-pushed the 208-in-memory-provider branch 2 times, most recently from 7f67cf9 to b825f75 Compare April 8, 2026 16:10
Copy link
Copy Markdown
Contributor

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things I noticed on the latest commits — great progress overall.

Copy link
Copy Markdown
Contributor

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few paths still don't have unit test coverage:

  • The disabled flag path (return defaultValue with Reason.DISABLED)
  • contextEvaluator returning null (fallback to defaultVariant with Reason.DEFAULT)
  • updateFlags/updateFlag event emission (that ProviderConfigurationChanged fires with the correct flagsChanged set)

@mstepien mstepien force-pushed the 208-in-memory-provider branch from b825f75 to 56cdedd Compare April 9, 2026 10:58
@mstepien mstepien requested a review from typotter April 10, 2026 10:21
Copy link
Copy Markdown
Contributor

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes, Marcin

Sorry for missing this earlier, but I think we need to relook at the ContextEvaluator interface.

@mstepien mstepien force-pushed the 208-in-memory-provider branch 2 times, most recently from 02ca4bc to eaeb895 Compare April 14, 2026 13:02
@mstepien mstepien requested a review from typotter April 14, 2026 14:05
@mstepien mstepien force-pushed the 208-in-memory-provider branch from ebe0c00 to eaa5335 Compare April 16, 2026 09:24
leoromanovsky added a commit to DataDog/documentation that referenced this pull request Apr 22, 2026
Address review from @typotter:
- Remove events.emit(ProviderReady) from FakeProvider.initialize; the SDK
  emits ProviderReady after initialize returns.
- Soften the InMemoryProvider wording and link to open-feature/kotlin-sdk#226,
  where upstream in-memory provider support is tracked.
@typotter
Copy link
Copy Markdown
Contributor

Code review

Found 2 issues:

  1. kotlin.concurrent.Volatile is JVM-only — will fail to compile on linuxX64, js, iosArm64, iosSimulatorArm64. Replace with AtomicRef from kotlinx-atomicfu (move from commonTest to commonMain), or restructure to avoid the field entirely.

import kotlin.concurrent.Volatile
class InMemoryProvider(initialFlags: Map<String, Flag<*>> = emptyMap()) : FeatureProvider {
override val hooks: List<Hook<*>> = emptyList()
override val metadata: ProviderMetadata = InMemoryProviderMetadata("InMemoryProvider")
private val flagsState = MutableStateFlow<Map<String, Flag<*>>>(initialFlags.toMap())
@Volatile
private var state: OpenFeatureStatus = OpenFeatureStatus.NotReady

  1. context-aware E2E fixture: contextEvaluator returns "INTERNAL"/"EXTERNAL" (uppercase) but the variant keys are "internal"/"external" (lowercase). containsKey always misses — every evaluation falls back to defaultVariant with Reason.ERROR instead of producing Reason.TARGETING_MATCH. Fix: return lowercase strings from the evaluator.

"context-aware" to Flag.builder<String>()
.variant("internal", "INTERNAL")
.variant("external", "EXTERNAL")
.defaultVariant("external")
.contextEvaluator { _, ctx ->
if (ctx?.getValue("customer")?.asBoolean() == false) "INTERNAL" else "EXTERNAL"
}

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@mstepien
Copy link
Copy Markdown
Author

kotlin-sdk/kotlin-sdk/src/commonMain/kotlin/dev/openfeature/kotlin/sdk/providers/memory/InMemoryProvider.kt

Volatile should work without changes. And should take effects on platform with multithreading. Project compiles fine on all non-JVM targets without failing.
Starting with Kotlin 1.9.0, kotlin.concurrent.Volatile was introduced into the common standard library and made stable for Kotlin multiplatform.

kotlin-sdk/kotlin-sdk/src/jvmTest/kotlin/dev/openfeature/kotlin/sdk/e2e/EvaluationSteps.kt

Good catch, fixed case sensitivity issue

Signed-off-by: Marcin Stepien <marcin.stepien@fluxon.com>
Signed-off-by: Marcin Stepien <marcin.stepien@fluxon.com>
Signed-off-by: Marcin Stepien <marcin.stepien@fluxon.com>
@mstepien mstepien force-pushed the 208-in-memory-provider branch from eaa5335 to e7494ef Compare April 28, 2026 12:20
leoromanovsky added a commit to DataDog/documentation that referenced this pull request Apr 30, 2026
* feature_flags: add shared testing section to server and client _index

Introduces a 'Testing with in-memory providers' section on both the
server-side and client-side feature flag landing pages. Explains that the
Datadog provider talks to Datadog's backend and is not appropriate for
unit tests, and directs readers to per-language pages for concrete
examples.

* feature_flags: add Testing section to Go server-side page

Teaches customers to swap DatadogProvider for OpenFeature's memprovider
in unit tests. Uses named clients via SetNamedProviderAndWait so t.Parallel()
is safe, and shows the ContextEvaluator pointer-to-func pattern.

* feature_flags: add Testing section to Java server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
(Flag builder form) in JUnit 5 tests. Emphasizes OpenFeatureAPI singleton
reset via shutdown() in @AfterEach and mentions Spring Boot test setup.

* feature_flags: add Testing section to Python server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in pytest. Uses a function-scoped fixture with api.shutdown() teardown to
prevent global API singleton leakage between tests.

* feature_flags: add Testing section to Node.js server-side page

Teaches customers to swap DatadogProvider for OpenFeature's
TypedInMemoryProvider in Vitest tests. Shows OpenFeature.close() teardown
and per-test putConfiguration() to keep tests isolated from each other.

* feature_flags: add Testing section to .NET server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in xUnit via IAsyncLifetime. Notes cross-framework applicability (NUnit,
MSTest) and the WebApplicationFactory integration path for ASP.NET Core.

* feature_flags: add Testing section to Ruby server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in RSpec via an around hook that captures and restores the prior provider.
Notes the Ruby SDK's plain-value flag hash (no variants) and the add_flag
mutation API.

* feature_flags: add Testing section to JavaScript client-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
from @openfeature/web-sdk in Vitest. Calls out the required flag shape
(variants, defaultVariant, disabled) and the sync evaluation model.

* feature_flags: add Testing section to React client-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in tests. Keeps the section minimal per the client-testing style: short
paragraph plus a 10-line snippet, no RTL or component harness.

* feature_flags: add Testing section to Android client-side page

Kotlin/Android OpenFeature SDK does not ship an InMemoryProvider, so the
section includes a ~30-line FakeProvider stub and walks through setting
it up under runTest from kotlinx-coroutines-test.

* feature_flags: add Testing section to iOS client-side page

OpenFeature Swift SDK ships no InMemoryProvider and has no contrib package,
so the section includes a ~50-line InMemoryTestProvider FeatureProvider
stub. Uses async setProviderAndWait and clearProviderAndWait for teardown.

* feature_flags: apply upstream OpenFeature SDK verification fixes

Corrections from per-language expert review against current upstream SDKs:

- Go: ContextEvaluator is already *func(...); drop the extra cast so the
  snippet compiles (the prior form produced a **func(...) conversion error).
- Python: context_evaluator returns a FlagResolutionDetails, not a bare
  variant name. Fix prose wording.
- .NET: InMemoryProvider's mutation method is UpdateFlagsAsync, not
  UpdateFlags. Fix prose wording.
- JavaScript (client) and React: swap the deprecated InMemoryProvider for
  TypedInMemoryProvider (upstream marks the former @deprecated) and
  mention OpenFeatureTestProvider as the idiomatic React component-test
  wrapper.

* feature_flags: frame iOS testing around OpenFeature and dd-openfeature-provider-swift

Points readers at the Datadog OpenFeature bridge (dd-openfeature-provider-swift)
as the production path for OpenFeature on iOS. Tests swap the bridge's
DatadogProvider for a small custom FeatureProvider because the upstream
OpenFeature Swift SDK does not ship an in-memory provider.

* feature_flags: fix iOS test snippet against OpenFeature Swift SDK 0.3.0

Verification caught six compile errors in the InMemoryTestProvider stub:

- initialize/onContextSet are 'async throws', not Future<Void, Never>
- observe() returns AnyPublisher<ProviderEvent?, Never> (optional event)
- ProviderEvent.ready has no associated value; use .ready not .ready(nil)
- Reason case is .staticReason, not .static ('static' is reserved)
- OpenFeatureAPI.shared.clearProvider() is sync and does not exist as
  clearProviderAndWait() in 0.3.0

Also update the prose callout to match.

* feature_flags: clarify client-side provider fetches flags from Datadog's CDN

Client-side SDKs reach Datadog's CDN to retrieve flag assignments, not
the Remote Configuration backend used by server-side SDKs. Updates the
testing-section callouts on the shared client _index and on each
per-platform client page so the language is accurate.

* feature_flags: soften testing guidance to present in-memory as one option

In-memory providers are one testing approach; running the real Datadog
provider against a dedicated test environment is equally valid. Updates
the shared _index sections and each per-language page so they present
both options up front, then proceed with the in-memory example as
'this section covers...' rather than 'do not use the Datadog provider'.

* feature_flags: add OpenFeature usage section to iOS page

Adds a new 'Use via OpenFeature' section covering install of
dd-openfeature-provider-swift, OpenFeatureAPI setup, evaluation context,
typed flag evaluation, and flag details. Existing FlagsClient-based
sections are unchanged. Includes a 'not yet production-ready' callout
since the bridge package is still in development.

* feature_flags: note getIntegerValue returns Int64 in iOS OpenFeature section

* Update content/en/feature_flags/client/_index.md

Co-authored-by: Tyler Potter <tyler.potter@datadoghq.com>

* feature_flags/android: drop ProviderReady emit, note kotlin-sdk#226

Address review from @typotter:
- Remove events.emit(ProviderReady) from FakeProvider.initialize; the SDK
  emits ProviderReady after initialize returns.
- Soften the InMemoryProvider wording and link to open-feature/kotlin-sdk#226,
  where upstream in-memory provider support is tracked.

* feature_flags/server: mirror client step 3 rewording

Apply @typotter's step 3 rewrite from client/_index.md to server/_index.md
for consistency: convey that units under test get an injected OpenFeature
client backed by the InMemoryProvider.

* 1 picker only

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* remove dup from server

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* split sentence

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* action verb

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* with of

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* action

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* comma dotnet

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* comma nodejs

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* fast no promises

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* fast server no promises

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* ios bridge

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* these

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* Apply suggestions from code review

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

---------

Co-authored-by: Tyler Potter <tyler.potter@datadoghq.com>
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
estherk15 pushed a commit to DataDog/documentation that referenced this pull request May 4, 2026
* feature_flags: add shared testing section to server and client _index

Introduces a 'Testing with in-memory providers' section on both the
server-side and client-side feature flag landing pages. Explains that the
Datadog provider talks to Datadog's backend and is not appropriate for
unit tests, and directs readers to per-language pages for concrete
examples.

* feature_flags: add Testing section to Go server-side page

Teaches customers to swap DatadogProvider for OpenFeature's memprovider
in unit tests. Uses named clients via SetNamedProviderAndWait so t.Parallel()
is safe, and shows the ContextEvaluator pointer-to-func pattern.

* feature_flags: add Testing section to Java server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
(Flag builder form) in JUnit 5 tests. Emphasizes OpenFeatureAPI singleton
reset via shutdown() in @AfterEach and mentions Spring Boot test setup.

* feature_flags: add Testing section to Python server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in pytest. Uses a function-scoped fixture with api.shutdown() teardown to
prevent global API singleton leakage between tests.

* feature_flags: add Testing section to Node.js server-side page

Teaches customers to swap DatadogProvider for OpenFeature's
TypedInMemoryProvider in Vitest tests. Shows OpenFeature.close() teardown
and per-test putConfiguration() to keep tests isolated from each other.

* feature_flags: add Testing section to .NET server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in xUnit via IAsyncLifetime. Notes cross-framework applicability (NUnit,
MSTest) and the WebApplicationFactory integration path for ASP.NET Core.

* feature_flags: add Testing section to Ruby server-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in RSpec via an around hook that captures and restores the prior provider.
Notes the Ruby SDK's plain-value flag hash (no variants) and the add_flag
mutation API.

* feature_flags: add Testing section to JavaScript client-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
from @openfeature/web-sdk in Vitest. Calls out the required flag shape
(variants, defaultVariant, disabled) and the sync evaluation model.

* feature_flags: add Testing section to React client-side page

Teaches customers to swap DatadogProvider for OpenFeature's InMemoryProvider
in tests. Keeps the section minimal per the client-testing style: short
paragraph plus a 10-line snippet, no RTL or component harness.

* feature_flags: add Testing section to Android client-side page

Kotlin/Android OpenFeature SDK does not ship an InMemoryProvider, so the
section includes a ~30-line FakeProvider stub and walks through setting
it up under runTest from kotlinx-coroutines-test.

* feature_flags: add Testing section to iOS client-side page

OpenFeature Swift SDK ships no InMemoryProvider and has no contrib package,
so the section includes a ~50-line InMemoryTestProvider FeatureProvider
stub. Uses async setProviderAndWait and clearProviderAndWait for teardown.

* feature_flags: apply upstream OpenFeature SDK verification fixes

Corrections from per-language expert review against current upstream SDKs:

- Go: ContextEvaluator is already *func(...); drop the extra cast so the
  snippet compiles (the prior form produced a **func(...) conversion error).
- Python: context_evaluator returns a FlagResolutionDetails, not a bare
  variant name. Fix prose wording.
- .NET: InMemoryProvider's mutation method is UpdateFlagsAsync, not
  UpdateFlags. Fix prose wording.
- JavaScript (client) and React: swap the deprecated InMemoryProvider for
  TypedInMemoryProvider (upstream marks the former @deprecated) and
  mention OpenFeatureTestProvider as the idiomatic React component-test
  wrapper.

* feature_flags: frame iOS testing around OpenFeature and dd-openfeature-provider-swift

Points readers at the Datadog OpenFeature bridge (dd-openfeature-provider-swift)
as the production path for OpenFeature on iOS. Tests swap the bridge's
DatadogProvider for a small custom FeatureProvider because the upstream
OpenFeature Swift SDK does not ship an in-memory provider.

* feature_flags: fix iOS test snippet against OpenFeature Swift SDK 0.3.0

Verification caught six compile errors in the InMemoryTestProvider stub:

- initialize/onContextSet are 'async throws', not Future<Void, Never>
- observe() returns AnyPublisher<ProviderEvent?, Never> (optional event)
- ProviderEvent.ready has no associated value; use .ready not .ready(nil)
- Reason case is .staticReason, not .static ('static' is reserved)
- OpenFeatureAPI.shared.clearProvider() is sync and does not exist as
  clearProviderAndWait() in 0.3.0

Also update the prose callout to match.

* feature_flags: clarify client-side provider fetches flags from Datadog's CDN

Client-side SDKs reach Datadog's CDN to retrieve flag assignments, not
the Remote Configuration backend used by server-side SDKs. Updates the
testing-section callouts on the shared client _index and on each
per-platform client page so the language is accurate.

* feature_flags: soften testing guidance to present in-memory as one option

In-memory providers are one testing approach; running the real Datadog
provider against a dedicated test environment is equally valid. Updates
the shared _index sections and each per-language page so they present
both options up front, then proceed with the in-memory example as
'this section covers...' rather than 'do not use the Datadog provider'.

* feature_flags: add OpenFeature usage section to iOS page

Adds a new 'Use via OpenFeature' section covering install of
dd-openfeature-provider-swift, OpenFeatureAPI setup, evaluation context,
typed flag evaluation, and flag details. Existing FlagsClient-based
sections are unchanged. Includes a 'not yet production-ready' callout
since the bridge package is still in development.

* feature_flags: note getIntegerValue returns Int64 in iOS OpenFeature section

* Update content/en/feature_flags/client/_index.md

Co-authored-by: Tyler Potter <tyler.potter@datadoghq.com>

* feature_flags/android: drop ProviderReady emit, note kotlin-sdk#226

Address review from @typotter:
- Remove events.emit(ProviderReady) from FakeProvider.initialize; the SDK
  emits ProviderReady after initialize returns.
- Soften the InMemoryProvider wording and link to open-feature/kotlin-sdk#226,
  where upstream in-memory provider support is tracked.

* feature_flags/server: mirror client step 3 rewording

Apply @typotter's step 3 rewrite from client/_index.md to server/_index.md
for consistency: convey that units under test get an injected OpenFeature
client backed by the InMemoryProvider.

* 1 picker only

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* remove dup from server

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* split sentence

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* action verb

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* with of

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* action

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* comma dotnet

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* comma nodejs

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* fast no promises

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* fast server no promises

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* ios bridge

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* these

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

* Apply suggestions from code review

Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>

---------

Co-authored-by: Tyler Potter <tyler.potter@datadoghq.com>
Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add InMemory Provider and Basic E2E Tests

3 participants