-
Notifications
You must be signed in to change notification settings - Fork 121
[REST API] Introduce A/B test and remove feature flag #8744
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REST API] Introduce A/B test and remove feature flag #8744
Conversation
You can test the changes from this Pull Request by:
|
Generated by 🚫 dangerJS |
ealeksandrov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works as described ![]()
- ✅ control login flow
- ✅ treatment login flow
- ✅ self-hosted site
- ✅ treatment self-hosted site
- ✅ treatment self-hosted site with application password disabled
One note: when I log out in app settings - I always get login flow for control group (3 buttons), after app restart it resets to treatment (2 buttons).
|
Thanks for the review, @ealeksandrov 🙇
AFAIK this could be because ABTest variant info is cleared during logout, and the login screen is shown before we fetch the variations again from the server. |
|
I have updated the abacus experiment to a newer version. I added a new version to address the feedback I received on the experiment proposal. pbxNRc-2i4-p2 I have validated this new experiment with my sandbox. Could you kindly validate this with your sandbox? Thank you! |
ealeksandrov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All flows still work as described ![]()
Part of: #8679
Description
This PR adds a new A/B test experiment for the REST API project to enable application password authentication. Experiment details: pbxNRc-2i4-p2
Testing instructions
Testing the
controlvariationTesting the
treatmentvariationWe will have to sandbox and assign a particular variation to the device to test the variations.
Prerequisites:
ifconfig) in the local matchine's/etc/hostsExPlatServiceL47 whereanon_idis set in the URL requestPATCH /wpcom/v2/experiments/0.1.0/assignmentswith the followingbodyand make sure it's successful:1. Prologue screen and site address login flow.
2. Application password disabled screen
Pre-requisite: Disable application password on your self-hosted site (e.g. by installing WordFence plugin Free version is enough for testing).
Screenshots
Application password disabled:

RELEASE-NOTES.txtif necessary.