- Optimizely X Full Stack
THIS ARTICLE WILL HELP YOU:
- Run A/B tests on features
- Test your features without deploying code
Feature tests are A/B tests you conduct on features. Feature tests have all the properties of an A/B test, plus the ability to:
Trigger the test using the isFeatureEnabled API
Toggle the feature ON and OFF when defining a variation
Specify feature variable values as part of a feature configuration for a specific variation
Create a feature test
Feature Management adds a new type of experiment to the Experiments dashboard: Feature Test. To create a new feature test:
Navigate to Experiments > Create New…
Select Feature Test from the dropdown menu
Choose the feature you want to test or click Create New Feature... to add a new feature.
After you select a feature, Optimizely automatically generates an experiment key by appending “_test” to the end of the feature key for the feature you selected.
You can edit the experiment key if you like, as long as you always use a unique key.
Create feature test variations
Optimizely automatically suggests variation keys for your feature tests, because variation keys aren't required to setup a feature test. Optimizely will assign a user to a variation, then return the values associated with the assigned variation for isFeatureEnabled and for your feature configuration (i.e. variables). By default, we’ll provide two variations with the keys “variation_1” and “variation_2”. Like experiment keys, you can edit variation keys.
If you add additional variations, we will provide automatic suggestions according to the variation number: “variation_3”, “variation_4”, and so on. Deleting a variation will not affect the automatic numbering of our automatic variation key suggestions.
Feature toggles and configurations
Feature test variations include a feature toggle and the feature configuration (if one exists). By default, the toggle will be set to ON and the configuration default values will load:
A common feature test includes a feature with no configuration, with one variation set to test “toggle=ON” and another variation set to test “toggle=OFF.” This allows you to test the performance of you application in its current form v. its performance with your new feature enabled.
If the feature includes a feature configuration and you set a variation to “toggle=OFF,” Optimizely will disable the option to modify variable values and revert to the default variable values.
To create variations using feature configurations, update the variable values under each variation:
When this experiment is live, the getFeatureVariable APIs will return the values specified for the variation assigned to a visitor. Experimenting using a feature configuration allows you to iterate on a feature in between code deploys. You can run a sequence of experiments with different combinations of variable values to determine the optimal experience for your users.
If a feature test is running on a feature that uses a feature configuration, the feature configuration is locked until you pause the test.
Launch a feature test
Like A/B tests, you will assign audiences (optional), metrics, traffic allocation, and mutually exclusive groups (optional) to feature tests. After saving your changes, launch your feature test using the same workflow as you would use for A/B tests in environments.
For every feature test running on the feature, we report:
Experiment Key: unique identifier for the experiment. Click to open the Experiment modal.
Type: Feature Test. We plan to include other types of tests in the future.
Status: current status of the test (Draft, Running, or Paused)