Skip to main content
menu-icon.png

 

x
Optimizely Knowledge Base

Interaction between feature rollouts and feature tests

Relevant products:
  • Optimizely X Full Stack

THIS ARTICLE WILL HELP YOU:
  • Understand how feature rollouts and feature tests interact
  • Use feature rollouts and feature tests more effectively

Feature tests take precedence over feature rollouts. In other words, if your feature has both a feature test and feature rollout, the feature test is evaluated first. If a user does not qualify for the feature test, the feature rollout is evaluated.

To understand how this works in practice, imagine the following scenario. You've deployed a feature to your application and you've created both a feature test and feature rollout for that feature:

  • Feature test running with an audience and a traffic allocation

  • Feature rollout running with an audience and a traffic allocation

A series of users are handled by your application and are evaluated according to their audience attributes and your feature test and feature rollout rules.  The following chart depicts all possible outcomes:

User Feature Test Audience Feature Test Traffic Allocation Feature Rollout Audience Feature Rollout Traffic Allocation Result
user1 pass pass N/A N/A Feature test
user2 pass fail pass pass Feature rollout
user3 fail N/A pass pass Feature rollout
user4 fail N/A pass fail No action
user5 fail N/A fail N/A No action

 

Interactions in multiple feature tests

In each environment, you can only run one feature test on each feature at a time -- you can't run two tests on the same feature simultaneously (unless your feature is assigned to a mutually exclusive group). You can have as many draft, paused, and archived tests on a feature as you like.

If you try to launch a second feature test while another feature test is running, you’ll see a warning that only one feature test is allowed at a time (unless the tests are mutually exclusive).

Use mutually exclusive groups to run concurrent production feature tests. If feature tests are running in a mutually exclusive group, Optimizely evaluates a user’s mutually exclusive group first in the order of operations after calling isFeatureEnabled.  After the mutually exclusive group is assigned, Optimizely evaluates the feature test first. If a user does not quality for the feature test, the feature rollout is evaluated.