Getting started and wondering what to test? This article outlines a top 10 list of the most common, relatively simple experiments that people run as they're getting started with Optimizely. It also provides you with guidance on how to build them on your own site.

For each experiment, we'll share the story of someone who wants to use A/B testing to improve the user experience on his or her website and an experiment idea to test out the hypotheses. The ideas and implementation details are intended to be adaptable and to help kick-start your own testing opportunities. We'll also link to relevant Knowledge Base articles if you'd like to expand on what you learn here. 

 
Tip:

Looking for the definitive guide to getting up and running with Optimizely? Learn and practice your Optimizely skills and web optimization strategy in Optimizely Academy.

Have a question about getting your experiment up and running? Head over to the Optimizely Community and join the discussion.

Site-wide changes

This month, John is promoting a new product that he feels will be a hit. The ad for this new product is currently poorly highlighted and appears as one of the many promo boxes on the homepage. John speculates that adding a promotional banner site-wide will increase the click-through rate and purchase conversions.

Setting up the experiment:

  • First, determine where the banner would fit best on your site.
  • Then, work with your creative team to design a promotional image that is the correct dimensions and is in line with your site’s style guide.
  • Once you have the banner ready to go, open up the Optimizely editor and load one of the pages where you’d like the new image to appear.
  • Click the relevant location (in this case we will add the banner above the top navigation) and select Insert Image. You have the option to enter the URL of the new image, or upload the image right from your desktop.
  • Once the image is on the page, you can use the Move and Resize feature to fine-tune the element’s location and sizing.
  • To ensure that your experiment runs site-wide, click Options > URL Targeting and set the URL targeting condition to be a substring match for yourdomain.com. A substring match will allow the experiment to run on any page that contains the string “yourdomain.com” (which should be every page on your site.) Note: you can use this targeting concept for any experiments that require a site-wide change.
  • To make sure your experiment is tracking goals that are meaningful, set up a click goal on the banner itself, a pageview goal on the promotional product’s cart page, and pageview goal on the purchase confirmation page. The results of your experiment will prove once and for all if the addition of a site-wide promotional banner ads value to the customer’s experience.

Before

After (Added a promotion underneath the search bar)

Testing the CTA: Call to Action

Joe has a theory that he can increase the click through rate on his “Add to Cart” CTA by testing out different versions of the button’s language and color.

Setting up the experiment:

  • What do you think will cause visitors to respond positively to the “Add to Cart” CTA? Does text outperform symbols? Does a short CTA outperform a more detailed CTA? Does red outperform blue? Brainstorm some ideas with your team and put your predictions to the test.
  • Load one of your product pages into the editor and select the CTA button. Choose Edit Element > Edit Style > Color and Background to modify the background color of the button.
  • For your second variation, click the CTA again and choose Edit Text to modify the copy. Note: if your buttons are images and are not created with CSS properties, then you will have to click on the image and swap it out for the new, updated image.
  • Set up a click goal on the CTA so you can accurately determine which button is generating the highest number of clicks. It would also be smart to set up a pageview goal on the purchase confirmation page so you can measure how the changes are impacting overall conversions.

For additional details on the results of the Fab.com experiment that tested different CTAs, check out our customer story here. You can also check out our blog post on CTA optimization here.

Before

After ("Add to Cart" button is changed to red)

 

Focusing on your Call to Action within the Checkout Funnel

Kate has recently found that a large number of visitors are reaching the “Cart Review” and then bouncing. She thinks she can increase the percentage of visitors who progress to the next stage in the funnel by adding a second “Proceed to Checkout” button above the product table. This will ensure that the CTA always appears both above and below the fold.

Setting up the experiment:

  • For this experiment, you will need to add a new button that functions in the same way as your original button. You may want to contact your technical team to attain the exact HTML needed to duplicate the button and make it fully functional.
  • Load the cart page into the editor. Before adding the new HTML, it's a best practice to identify the correct container (i.e. the correct <div> element) for the button. You can do this using the Select Container feature. Specific positioning will ensure that your button does not end up floating in an undesirable location on the page.
  • Once you have the correct container selected, use the Insert HTML feature to add your new button in the appropriate position. 
  • To set your experiment’s targeting conditions, click Options > URL Targeting and add the relevant URL. In most cases, a simple match to your cart’s URL will work.
  • To measure the performance of your test and confidently determine whether additional CTAs add value to your checkout flow, you will need to add goals to your experiment.
  • To add a click goal to your new button, click on the CTA and select Track Clicks from the dropdown menu. This will pop up a Goal Dialog that will let you give the goal a name and adjust the goal settings.
  • Be sure to also add the original button to the goal (a single goal can track clicks on multiple elements). This will give you a comprehensive view of clicks on a “Proceed to Checkout” button, rather than just clicks on one button or the other.

Before

After ("Update" and "Checkout" buttons added)

Removing Distractions from the Checkout Funnel

Greg thinks that the navigation bar within his checkout funnel is distracting to customers. He believes he can increase the number of successful conversions by keeping visitors on the page and removing the navigation bar.

Setting up the experiment:

  • To get started with this test, load the first checkout page of your funnel into the editor.
  • Click the navigation bar and choose Select Container from the dropdown menu. You will be able to choose the appropriate container that selects the entire navigation bar, rather than just one of the tab elements. Now, click Remove > Remove element from page.
  • Feel free to repeat this process for any banners or icons that appear in the header and may be a potential distractor.
  • To ensure the navigation bar is removed from every page in your checkout funnel, you will need to set your targeting conditions to a substring match. In Greg’s case, all of his checkout pages contain the string “yourdomain.com/checkout.”
  • As a baseline metric, it is beneficial to add click goals to the navigation bar elements in the Original version. Be sure to set up a pageview goal on the purchase confirmation page so you can confidently determine whether a clutter-free checkout funnel is better for your bottom line.

Before

After (Nav removed)

Rearranging the Order of your Subscription Price Points

Tommy’s product page lists three different subscription types that start from the cheapest on the left and increase in price as you move to the right. There is also a “best-deal” banner on the cheapest subscription. Tommy has a hypothesis that by removing the “best-deal” banner and rearranging the subscription types so that the most expensive subscription is on the left, he will be able to increase revenue.

Setting up the experiment: 

  • Load the product page into the editor and select the column with the most expensive pricing structure.
  • Make sure to use the Select Container feature to grab the entire element.
  • Then, click Rearrange and drag the column so it moves all the way to the left. (Note: be sure to avoid using the “Move and Resize” feature to do this.) You will see the other two columns adjust accordingly.
  • Do the same thing for the column with the cheapest pricing structure, but move it all the way to the right.
  • Click the “best deal” banner image and hide it using the Remove feature.
  • Tommy only needs this experiment to run on the pricing page. Click Options > URL Targeting and set the targeting to be a simple match to the appropriate URL.
  • The key metric for determining success of the experiment here will be revenue. You can add revenue tracking to your experiment.
  • To track engagement with the page, it is also recommended that you create click goals on each of the subscription levels.

Highlighting Site Navigation Elements

Ricky has a hypothesis that increasing the size of the breadcrumbs on his product pages will allow his customers to navigate more easily between different pages to find the item they want, and will ultimately increase purchases as a result.

Setting up the experiment:

  • Breadcumbs are designed to help users navigate between products and categories. On your site, identify which groupings of pages use breadcrumbs and identify how specific the navigation levels can get.
  • Within the editor, click the breadcrumbs text and use “Select Container” to make sure you are changing the font size of all the possible levels at once, rather than just one product name.
  • Click Edit Element > Edit Style > Text > Font Size and adjust the font size to be a few pixels larger.
  • Importantly, this experiment should run on all the pages that you identified earlier. In Ricky’s case, he needs to set the URL targeting to match all of his product pages. In most cases, a substring match will work but Regular Expressions or Custom Tags can also come in handy if your URLs contain dynamic product names/IDs.
  • To track the performance of a larger font size, add a click goal to the breadcrumb elements and to the “Add to Cart” buttons. There may be other relevant clicks goals you wish to create that will help to deduce whether the size of the breadcrumbs have an effect on user interaction.
  • Finally, don’t forget to set up a pageview goal on your confirmation page so you can easily determine how the test is impacting your high-level objectives.

Before

After (More prominent breadcrumb)

Personalizing Pages based on Cookies

Pam knows that her site adds a unique cookie to a visitor who has read more than 2 articles on a certain topic. Pam hypothesizes that she can increase engagement on the site and increase click-through rates on articles if she can customize the homepage based on a reader’s preferences. As an example, Pam would like to feature a hero banner that highlights an article related to entrepreneurship for visitors who frequently browse articles related to this topic.

Setting up the experiment: 

  • To save time, Pam wants to run a proof-of-concept experiment and isolate only her most common article topic. She has decided that she will test personalizing the homepage for visitors who have an interest in the environment.
  • Load the homepage into the editor and swap out the main image with the new, personalized image. To do this, simply click the image and select Change Image from the dropdown menu. As a future experiment idea, Pam has noted that she also wants to test personalizing the recommended content to match the reader’s interest area.
  • How will Pam ensure that only visitors with the “environment” cookie are seeing the updated image? She does not want to show an image featuring endangered animals to her readers who have no interest in this topic. Optimizely allows you to target an experiment based on a variety of conditions known to be true about the visitor -- one of these options is cookie targeting.
  • Click the Audiences button, then Create New Audience. In the Audiences menu, drag Cookie into the Audience Conditions section and add the relevant cookie’s name and value.
  • As goals for the experiment, set up a click goal on the homepage banner to effectively determine whether personalized content drives more clicks on high-value areas.

Personalizing Pages based on Search Campaigns

Wesley is a marketing manager who has been tasked with optimizing paid search campaigns. He is currently running 2 campaigns that are advertising his company’s new e-book. Each of these campaigns are using slightly different messaging to target different groups of people and interest areas. Wesley believes he can increase conversions on the e-book landing page by customizing the header, sub-header and call to action copy to match a specific ad’s messaging (rather than use a generic landing page for all ads).

Setting up the experiment: 

  • To get started, you will first need to identify how visitors arriving from each ad campaign can be differentiated from one another. In Wesley’s case, he has created unique utm_campaign names that will be associated with the ads.
  • The easiest way to set up this test is to create 2 separate experiments -- one for each campaign.
  • Load your landing page loaded into the editor, click on the header and select Edit Text from the dropdown menu.
  • Customize the copy to match the specific campaign's messaging. Do the same for any other content on the page that you’d like to modify. For this particular experiment, Wesley is going to focus on the discounted price.
  • Repeat this process for your second ad campaign.
  • How will Wesley ensure that the correct traffic sees the experiment? He does not want a visitor who clicked on an ad featuring a discount to land on a page that completely downplays the price.
  • You can specify an audience for an experiment using a variety of conditions known to be true about the visitor -- one of these options is query string parameter targeting. Click Audiences > Create New Audience and add the relevant parameter’s name and value.
  • Set up a click goal on the landing page’s Call To Action so you can confidently measure whether the original or customized version drives more conversions.

 

Optimizing Lead Generation Forms

Mark relies heavily on landing pages to generate leads for his sales team. He believes that changing the order and increasing the size of his input fields will increase conversions and result in more leads.

Setting up the experiment:

  • To get started with this experiment, load your landing page into the editor.
  • Click one of the input fields and select Edit Element > Edit Style
  • The Edit Style feature lets you easily make changes to the background color, border size, text font etc.
  • The Dimensions tab allows you to change the height and width of the input field. Once you have finished adjusting the size and coloring of your input fields, click the text titles and repeat the process.
  • With a little jQuery know-how, you can save some time and modify the styling of all of the input fields at once.
  • To change the order of input fields, simply click the field you’d like to move and select Rearrange from the dropdown menu. You will be able to drag this field into the desired location and the other input fields will adjust accordingly.
  • Click Options > URL Targeting to set up the URL targeting conditions for this experiment. Since your experiment should only run on one page, a simple match to the landing page URL will do the trick.
  • How will you confidently determine whether the larger input fields and new order had an effect on sign-ups? Set a click goal on the “Sign up” CTA so you can track which version generated a higher click-through rate.
  • If you would like to collect more granular data, you can set up custom events that track when visitors begin typing in a specific input field. Custom events such as this one will allow you to evaluate whether a certain field is generating dropoff.
  • Learn to build a test that streamlines the fields in your leadgen form, step-by-step. 

Before

After (New input fields)

Promoting a Value Proposition

Allison’s site offers free shipping on all online orders. Currently the “Free shipping” value proposition is hidden below the product description. Allison believes adding “Free shipping” text adjacent to the “Add to Cart” button will increase clicks on the CTA as well as increase overall purchase conversions.

Setting up the experiment:

  • Emphasizing free shipping on your product pages is a great way alleviate fears of extra charges. On product pages, it’s crucial that you are able to turn a visitor’s purchase intention into action; details can make all the difference.
  • Load one of your product pages into the editor.
  • Click the space right above the “Add to Cart” CTA, select Insert Text, and type the desired promotional message.
  • Once the free shipping text has been added to the page, you can modify the styling and position as necessary by using Edit StyleMove and Resize, or Rearrange.
  • Ideally, the experiment will run on all product pages to ensure a large sample size and to ensure consistency across the site. Click Options > URL Targeting and use a substring match to meet the needs of your test.
  • To accurately measure the success of this experiment, it will be important to add a click goal for the “Add to Cart” CTA and to add a pageview goal on the purchase confirmation page. For this experiment, it would also be interesting to track revenue in order to deduce whether highlighting free shipping increases overall revenue.

For additional details and ideas, check out our blog article on emphasizing the return policy.

Before

After ("Ships Free" added above "Add to Cart)

 

 
Tip:

Tell us about your experiment successes and see how others are seeing success with Optimizely in the Optimizely Community.