- Set up an A/B test with Optimizely
Optimizely makes it easy to A/B test your site. Use the Editor to design variations, define where the experiment runs and who sees it, set goals, and QA. Once you're ready, you'll go ahead and publish your experiment live to the world!
Watch this video for a quick overview of the six steps. Or, dive deeper into each step below.
Materials to prepare
- Review Experiment Plan
- URL scope
- Define test duration
- Identify upcoming site changes
- Identify potential overlapping experiments
- Identify integrations overlap (eVars, custom variables, etc.)
- Resources / code
- Gather creative assets
- Required resources from other team (copy, another team to build a page out or URLs from another team, you might be waiting on Marketing to make campaigns)
- Timeline for test creation
- Confirm that the QA environment is the same as the production environment (if applicable)
- Review Experiment Plan
- Program Manager
- Power User or Developer (if code-heavy)
Actions you'll perform
- Notify QA team of the experiment that will be running on Development environment (if applicable)
- Create an Experiment within QA project (if applicable)
- Six steps to create an Experiment
- Set up all integrations
- Reusable assets from this experiment
- Variation code
- Analytics code
- Custom event or revenue code
- Completed experiment(s)
- Completed goals(s)
What to watch out for
- Ill-defined scope for new functionality
- Lack of useful goals added
- An experiment timeline that overlaps with other site changes
Before you build your first test, create an experiment plan for more impactful results.
1. Create a variation
Start by using the Optimizely Editor to create the variations you scoped in your test plan.
First, navigate to the Home page and click New Experiment. Enter the URL of the website you’d like to test and name the experiment.
Once you load the site, you can use Optimizely’s Visual Editor to make changes to the page. To read more about creating variations, check out this article on using the Visual Editor or watch the video below.
If you’re code-savvy, use the Code Editor to create variations.
If you’ve already implemented the Optimizely snippet, you should be able to load the site without problems. But if you run into trouble, check this article about Editor loading issues.
2. Set Goals
To set up a goal, click the Goals icon () to open the Experiment Goals dialog. Click Create a new goal or Add a saved goal.
Create your new goal in the Create goal modal.
Learn more in this goal settings article, or get an overview with this short video:
Learn the strategy of setting different goals by industry vertical in this article about defining primary, secondary, and monitoring goals.
By default, Optimizely sets engagement as your primary goal: the goal that measures the success of your experiment. Use the Experiment Goals modal to change or remove the engagement goal.
You can also track add a revenue goal to your experiment. Read this article to learn more about revenue tracking goals.
At this stage, you’ll also want to turn on any analytics integrations, such as SiteCatalyst or Google Analytics, that you’ll use in this experiment. Read this article to learn more about our integrations in Optimizely.
3. Define URL targeting
Once you create your variations, you’ll tell Optimizely where the experiment will run. Use URL Targeting to define the pages included in the experiment.
Click Options and select URL Targeting.
Are you running an experiment on a single page? You’ll probably want to use our default: simple match. If you’re changing an element on multiple pages or site-wide, you’ll probably want to use substring match instead. Optimizely’s URL Match Validator helps you ensure that your experiment includes the right pages.
Read this article to learn more about URL Targeting.
4. Create audiences
Next, decide who can see your experiment. Use the Audience Builder to define the conditions a visitor must meet to see the experiment you’ve created.
Click the Audiences icon () and Create a New Audience. Drag and drop conditions to build your Audience.
5. Set traffic allocation
Once you’ve defined who can see the experiment and where it will run, decide how traffic will be split between the variations. Optimizely randomly allocates traffic into different variations, including the original. You can control the percentage of traffic that goes into each variation. You also control the total traffic that goes into the experiment, as a whole.
Click Options and select Traffic Allocation.
Learn more about traffic allocation, pausing variations, and pushing all traffic to a winning variation in our Traffic Allocation article. Or, watch this short video:
6. QA your experiment and publish it
Now that you’ve set up all components of your experiment, preview it to make sure it looks and works the way you intend.
In the variation tab, click Preview to open a new browser window in Preview mode.
Read this article to learn more about what you can do with Preview mode, or watch this short video:
Then, use the QA checklist from your experiment plan to perform more rigorous QA. Preview mode provides a great preliminary check, but you should always perform an in-depth QA review before pushing your experiment live to the world.
Read this article to learn more about how to QA your experiment. If you don’t have separate staging and production environments, take care when you QA. We recommend setting a test cookie as a best practice.
Once that’s done, go to Visual Editor and click Start Experiment. Or, click the start () button on the Home page. Congratulations! You’ve launched your experiment and now you’re ready to monitor results.
This article is part of a series. Click to learn more about the strategy and practice of experience optimization.