[ABTesting] Cache variations of logged out context experiments #8806
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| import AutomatticTracks | ||
|
|
||
| /// For getting the variation of a `ABTest` | ||
| public protocol ABTestVariationProvider { | ||
| /// Returns the `Variation` for the provided `ABTest` | ||
| func variation(for abTest: ABTest) -> Variation | ||
| } | ||
|
|
||
| /// Default implementation of `ABTestVariationProvider` | ||
| public struct DefaultABTestVariationProvider: ABTestVariationProvider { | ||
| public init() { } | ||
|
|
||
| public func variation(for abTest: ABTest) -> Variation { | ||
| abTest.variation ?? .control | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,27 @@ | ||||||||||||||||||||||||||||||||||
| import AutomatticTracks | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| /// Cache based implementation of `ABTestVariationProvider` | ||||||||||||||||||||||||||||||||||
| /// | ||||||||||||||||||||||||||||||||||
| public struct CachedABTestVariationProvider: ABTestVariationProvider { | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| private let cache: VariationCache | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| public init(cache: VariationCache = VariationCache(userDefaults: .standard)) { | ||||||||||||||||||||||||||||||||||
| self.cache = cache | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| public func variation(for abTest: ABTest) -> Variation { | ||||||||||||||||||||||||||||||||||
| guard abTest.context == .loggedOut else { | ||||||||||||||||||||||||||||||||||
| return abTest.variation ?? .control | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| if let cachedVariation = cache.variation(for: abTest) { | ||||||||||||||||||||||||||||||||||
| return cachedVariation | ||||||||||||||||||||||||||||||||||
| } else if let variation = abTest.variation { | ||||||||||||||||||||||||||||||||||
| try? cache.assign(variation: variation, for: abTest) | ||||||||||||||||||||||||||||||||||
| return variation | ||||||||||||||||||||||||||||||||||
| } else { | ||||||||||||||||||||||||||||||||||
| return .control | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||
| if let cachedVariation = cache.variation(for: abTest) { | |
| return cachedVariation | |
| } else if let variation = abTest.variation { | |
| try? cache.assign(variation: variation, for: abTest) | |
| return variation | |
| } else { | |
| return .control | |
| } | |
| if let variation = abTest.variation { | |
| try? cache.assign(variation: variation, for: abTest) | |
| return variation | |
| } else if let cachedVariation = cache.variation(for: abTest) { | |
| return cachedVariation | |
| } else { | |
| return .control | |
| } |
Contributor
Author
There was a problem hiding this comment.
Done in c0708bf
Earlier, I prioritised cached value to display that we return the cached value if it is available. Because, for a logged-out experiment, it is perfectly valid to get the variation value only once and use it without refresh.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| import AutomatticTracks | ||
|
|
||
| enum VariationCacheError: Error { | ||
| case onlyLoggedOutExperimentsShouldBeCached | ||
| } | ||
|
|
||
| public struct VariationCache { | ||
| private let variationKey = "VariationCacheKey" | ||
|
|
||
| private let userDefaults: UserDefaults | ||
|
|
||
| public init(userDefaults: UserDefaults) { | ||
| self.userDefaults = userDefaults | ||
| } | ||
|
|
||
| func variation(for abTest: ABTest) -> Variation? { | ||
| guard abTest.context == .loggedOut else { | ||
| return nil | ||
| } | ||
|
|
||
| guard let data = userDefaults.object(forKey: variationKey) as? Data, | ||
| let cachedVariations = try? JSONDecoder().decode([CachedVariation].self, from: data), | ||
| case let variation = cachedVariations.first(where: { $0.abTest == abTest })?.variation | ||
| else { | ||
| return nil | ||
| } | ||
|
|
||
| return variation | ||
| } | ||
|
|
||
| func assign(variation: Variation, for abTest: ABTest) throws { | ||
| guard abTest.context == .loggedOut else { | ||
| throw VariationCacheError.onlyLoggedOutExperimentsShouldBeCached | ||
| } | ||
|
|
||
| var variations = userDefaults.object(forKey: variationKey) as? [CachedVariation] ?? [] | ||
| variations.append(CachedVariation(abTest: abTest, variation: variation)) | ||
|
|
||
| let encodedVariation = try JSONEncoder().encode(variations) | ||
| userDefaults.set(encodedVariation, forKey: variationKey) | ||
| } | ||
| } | ||
|
|
||
| public struct CachedVariation: Codable { | ||
| let abTest: ABTest | ||
| let variation: Variation | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| import XCTest | ||
| @testable import Experiments | ||
|
|
||
| final class CachedABTestVariationProviderTests: XCTestCase { | ||
| func test_variation_is_control_when_the_value_does_not_exist() throws { | ||
| // Given | ||
| let userDefaults = try XCTUnwrap(UserDefaults(suiteName: UUID().uuidString)) | ||
|
|
||
| // When | ||
| let cache = VariationCache(userDefaults: userDefaults) | ||
| let provider = CachedABTestVariationProvider(cache: cache) | ||
|
|
||
| // Then | ||
| XCTAssertEqual(provider.variation(for: .applicationPasswordAuthentication), .control) | ||
| } | ||
|
|
||
| func test_correct_variation_is_returned_after_caching_it() throws { | ||
| // Given | ||
| let userDefaults = try XCTUnwrap(UserDefaults(suiteName: UUID().uuidString)) | ||
| let cache = VariationCache(userDefaults: userDefaults) | ||
| let provider = CachedABTestVariationProvider(cache: cache) | ||
|
|
||
| // When | ||
| try cache.assign(variation: .treatment, for: .applicationPasswordAuthentication) | ||
|
|
||
| // Then | ||
| XCTAssertEqual(provider.variation(for: .applicationPasswordAuthentication), .treatment) | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| import XCTest | ||
| @testable import Experiments | ||
|
|
||
| final class VariationCacheTests: XCTestCase { | ||
| func test_variation_is_nil_when_the_value_does_not_exist() throws { | ||
| // Given | ||
| let userDefaults = try XCTUnwrap(UserDefaults(suiteName: UUID().uuidString)) | ||
|
|
||
| // When | ||
| let cache = VariationCache(userDefaults: userDefaults) | ||
|
|
||
| // Then | ||
| XCTAssertNil(cache.variation(for: .applicationPasswordAuthentication)) | ||
| } | ||
|
|
||
| func test_correct_variation_is_returned_after_setting_it() throws { | ||
| // Given | ||
| let userDefaults = try XCTUnwrap(UserDefaults(suiteName: UUID().uuidString)) | ||
| let cache = VariationCache(userDefaults: userDefaults) | ||
|
|
||
| // When | ||
| try cache.assign(variation: .treatment, for: .applicationPasswordAuthentication) | ||
|
||
|
|
||
| // Then | ||
| XCTAssertEqual(cache.variation(for: .applicationPasswordAuthentication), .treatment) | ||
| } | ||
|
|
||
| func test_it_throws_when_trying_to_cache_logged_in_experiment() throws { | ||
| // Given | ||
| let userDefaults = try XCTUnwrap(UserDefaults(suiteName: UUID().uuidString)) | ||
| let cache = VariationCache(userDefaults: userDefaults) | ||
|
|
||
| // When | ||
| XCTAssertThrowsError(try cache.assign(variation: .treatment, for: .aaTestLoggedIn)) | ||
|
||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: should we leave a comment on why we are caching only for the logged-out context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if we need to restrict to logged-out experiments only 🤔 We are requesting only experiments for specific contexts so it should be safe to apply this for all contexts right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hope you mean that we only request experiments for current contexts from the server. Yes, I agree.
But, I would still prefer only to cache logged-out experiments to minimize the impact of this change. Also, we don't face any crossover issues with logged-in experiments.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense, please go ahead with this solution 👍