Skip to main content
menu_icon.png

 

x
Optimizely Knowledge Base

Six steps to create an experiment in Optimizely X Web

There are two versions of Optimizely
What version do you have?
X
Optimizely Classic
This is what the Optimizely Classic user interface looks like.
Optimizely X
This is what the Optimizely X user interface looks like.
. If you're using Optimizely Classic, check this article out instead.
RELEVANT PRODUCTS:
  • Optimizely X Web Experimentation

THIS ARTICLE WILL HELP YOU:
  • Set up an A/B test with Optimizely X Web Experimentation

Optimizely makes it easy to A/B test your site. Use the Editor to design variations, define where the experiment runs and who sees it, set events to measure success, and QA. Once your experiment looks and works the way you'd like, publish it live to the world!

Want to learn how to build an experiment in just six short minutes and follow along? Just watch this video.

 

Want to dig a little deeper? Check out our Optimizely Academy course on how to build an experiment in Optimizely X.

Let's get started.

  1. Navigate to the Experiments dashboard.

  2. Click Create New.

  3. Select A/B Test from the dropdown.
    create-exp.png

Before you can run an experiment, you'll need to implement the snippet and create at least one page and metric in Optimizely X Web. If your account uses custom snippets, you might see more than one snippet listed. In this case, choose the snippet you want to use for your experiment.

This checklist helps you set up Optimizely. This experiment design template can help you document and share the details of your experiment.

1. Add pages

Next, tell Optimizely where your experiment will run.

Click () to add any existing pages to your experiment. Or, click Create New Page and set up a new page.

2. Add audiences

Next, decide who will see your experiment.

Click () to add any existing audiences to your experiment. Or, click Create New Audience to build a new audience.

You can combine multiple audiences in your experiment using AND and OR conditions. For example, you can target the experiment to visitors who qualify for the "Social Butterflies" audience AND the "Luxury Travelers" audience -- a visitor must qualify for both. Or you can target the experiment to "Social Butterflies" AND "Luxury Travelers" OR 5x Buyers. In this case, a visitor only needs to qualify for one of these audiences to enter the experiment.

Here's how to target multiple audiences:

  1. Click () to add more than one audience.

  2. Use the dropdown to specify how you'd like to combine your audiences.

    Match any audienceA visitor is included if they match any of the audiences you add.
    Match all audiences: A visitor is included if they match all of the audiences you add.
    Custom: Use AND and OR conditions to specify exactly how you want visitors to qualify.

3. Add metrics

How will you measure success in your experiment? Add the metrics that measure success for your experiment. If the changes in your variation are successful, what will you measure to show improvement?

Create a metric out of an existing event or create a new event for the experiment.

To add an existing event, click ().  

The first metric you add is the primary metric for the experiment. This metric determines whether your experiment "wins" or "loses," and should track an event that's directly affected by the changes you make in your experiment.

Optimizely's Stats Engine uses false discovery rate control to make sure that the primary metric reaches statistical significance as quickly as possible. You can also add secondary metrics to measure the downstream effects of your experiment. The top five secondary metrics in your list will reach statistical significance quickly. Additional secondary metrics may take longer to reach statistical significance. Stats Engine ensures that secondary metrics don't affect the speed of your primary metric to help you make quick, data-driven business decisions.

To complete the experiment setup, you must add at least one metric now. Don't worry, you can always change, remove, or add new metrics to your experiment later.

4. Set traffic allocation

Decide how traffic will be split between your variations. Optimizely randomly allocates traffic into different variations, including the original. If you like, you can change the percentage of traffic that goes into each variation. Or, you can leave it as is.

Click Save to complete the experiment workflow and start creating your variations.

Once you save, you can also change the total traffic that goes into the experiment, as a whole.

traffic-allocation.png

You'll also use this sidebar to change the components that are attached to your experiment: metrics, audiences, and pages.

5. Create a variation

Once you've completed the experiment setup, use the Visual Editor to make changes to your variations.

  1. Select the variation you want to edit.

  2. To make visual changes, select the element you want to modify. Then, use the navigation in the left rail to make changes to the experience.

    edit_element.gif

    In the editor left rail, you can:

    • Fine-tune the jQuery selector that you're editing
    • Change the layout, including the element's visibility or position; or even rearrange the element on the page
    • Modify or replace typography, images, background styles or borders
    • Add inline CSS
    • Change the timing of the change from synchronous to asynchronous 
  3. Click Save.

  4. Repeat these steps for all the variations you'd like to create. To add more variations, click New Variation.

To make changes with custom code in JS or CSS, click the () icon.

6. QA and Publish

Now that you’ve set up all components of your experiment, preview it to make sure it looks and works the way you intend.

  1. Use the Preview button to view visual changes for your variation.


     

    Check out this article to QA the functionality of your campaign with the Preview tool.

    Not seeing your changes in Preview mode? Make sure your snippet is implemented on this page and configured to include Optimizely X. For deeper QA functionality, learn about advanced QA and troubleshooting.

  2. Once everything looks and works the way you'd like, click Publish to set the experiment live for visitors.

Congratulations! You've created and launched an experiment in Optimizely X Web Experimentation.