StickyBucketing does not work when there are more than 1 running experiment

[Arideno]

As FeatureEvaluator creates new EvalContext every time, the StickyBucketService does not work properly, as it is always having only one assignment document in stickyBucketAssignmentDocs. So stickyBucketAssignmentDocs is always resets to [:] here.

[vazarkevych]

Hi @Arideno. Please let us check that and we will provide a pull request with solution.

[vazarkevych]

Hi @Arideno This issue was fixed in this PR

[Arideno]

@vazarkevych Thank you!

[Arideno]

@vazarkevych Unfortunately, I still observe this issue. Only one experiment result is written to disk

[vazarkevych]

Hi @Arideno,

Thanks for your update. I checked the logic and unit tests, and it seems that sticky bucketing behaves as expected in most cases. Only one experiment result is stored per user context per feature.

So if a feature has multiple rules or experiments, only the “winning” experiment (the one that the user falls into according to the hash/fallback attributes) is written to stickyBucketAssignmentDocs.

The situation you described, where only one experiment is written, is actually expected behavior. Sticky bucketing does not store multiple experiment results simultaneously; it only keeps the assignment for the experiment the user is currently bucketed into.

If your goal is to see multiple assignments (like in unit tests), you need different user identifiers or different features, because the SDK maps one assignment per user per experiment.

[Arideno]

@vazarkevych I have one attribute and two different experiments, each using a different feature key. A user gets their variation IDs in both experiments, so I think both variation IDs (for the two different experiments) should be stored in the sticky bucket. Without this behavior, a user could receive a different variation ID if, for example, the coverage changes.

[vazarkevych]

Hi @Arideno ,

You're absolutely right, and thank you for the additional details! I found the actual issue - there was a bug in the Swift implementation of getStickyBucketAssignments that was causing the stickybucket assignments to be incorrectly cleared. Here PR with fix

[vazarkevych]

Hi @Arideno . Could you please confirm that this will resolve the issue and that we can close it after the merge?

[Arideno]

Hi, @vazarkevych. Unfortunately, I still experience the same issue

[vazarkevych]

Hi @Arideno . Could you please provide more details on how you are using StickyBucketing, maybe with a code example? It would help us reproduce this on our side.

[Arideno]

@vazarkevych I have 2 experiments set up and running. After evaluating all features on the start of the app (after sdk init), there is only one record in Sticky Bucketing text file on disk. I assume there should be all the experiments. The experiments' coverage is 100%, so I participate in both tests. Here is my sdk init code:

sdk = GrowthBookBuilder(
                apiHost: BuildConfiguration.growthBookApiHost,
                clientKey: BuildConfiguration.growthBookClientKey,
                encryptionKey: BuildConfiguration.growthBookEncryptionKey,
                attributes: [
                    // Attributes
                ],
                trackingCallback: { experiment, result in
                    // Tracking
                },
                refreshHandler: { [weak self] isRefreshed in
                    guard let self, !isRefreshCalled else { return }
                    isRefreshCalled = true
                    continuation.resume(returning: isRefreshed)
                }
            )
            .setStickyBucketService()
            .initializer()

[vazarkevych]

Hi @Arideno . Thank you for providing the details. We will check them and get back to you.

[vazarkevych]

Hi @Arideno . Just wanted to clarify: did you check that in the current pull request? It hasn’t been released yet. Also if you could please send us a response it would be helpful to fix the issue.

[Arideno]

Hi, @vazarkevych. I've just checked the behavior on pr's branch and still encounter the same issue. Here is SB file with only one feature saved, but there are like 7 experiments running at this time...

[vazarkevych]

Hi @Arideno . We have uploaded the updates. Please check our PR — looking forward to your feedback.