- Set the right goals for an experiment
- Decide where you'll track macro or micro conversions
- Get statistically significant results more often
Choosing the right metrics to track impact is a key part of good experiment design.
A conversion is an event that has a measurable impact on your business. Typically, the most important conversion events for a company are revenue and top-level metrics that tie directly into revenue, such as purchase rate or average order value.
In A/B testing, however, you seldom end up with actionable results if you rely only on these conversion events because they require huge shifts in behavior or large sample sizes to detect.
In this article, we show you how to balance “macro” conversions like revenue with “micro” conversions that track visitor behaviors that are closely related to the changes you’re testing. Knowing when to track micro-conversions instead of macro will help improve your win rate and show value when improving the user experience.
Macro-conversions are events that translate directly into revenue. Below are a few examples, by industry. Each macro-conversion is tied to the vertical’s business model. Notice that they usually occur at the end of the funnel and the conversion rate likely to be rather low.
|Industry||Most common macro-conversions|
|Media||Subscriptions, ad clicks|
Macro-conversions easily translate into revenue and are therefore very popular events to track in experiments and campaigns.
The problem with macro conversions is:
They don’t occur very often
They usually occur at the end of the funnel
Micro-conversions occur during the user journey as visitors travel one step of the funnel to another. Each part of the visitor's experience may have different micro-conversions.
Here are a few examples of micro-conversions for an e-commerce site (more examples here):
|Customer journey||Sample micro-conversions|
|Funnel step||Common macro-conversions|
|Homepage||Searches submitted, category clicks, featured products clicks|
|Search results / category page||Product clicks, filter/sort usage|
|Product page||Add-to-cart clicks|
|Shopping cart||Continue-to-cart clicks|
|Cart checkout||Fill-in payment details, purchase confirmation page views (macro)|
Each one of these micro-conversions is a step on the way to the purchase confirmation pageviews, which is the macro-conversion in this case. Micro-conversions are important milestones on the way to revenue, and occur far more often.
Micro-conversions are harder to translate into revenue. If you increase the number of searches from the homepage, you get more visitors to the next step of the funnel. But it’s hard to say how many more purchases you should expect from those events.
Optimize the metrics in your experiment design
The most efficient way to overcome the shortcomings of each type of event is to track both.
The problem with macro conversions is that they only measure home runs, but there are many ways to win in baseball.
- Hazjier Pourkhalkhali, Senior Strategy Consultant at Optimizely
Imagine that you run an e-commerce website and you’re testing a new algorithm that selects products to feature on the homepage. Your main goal is to sell as many products as possible, so you decide to measure the number of purchases (a macro-conversion). If you sell more items with the new algorithm than with the old one, then it’s a winner. Unfortunately, the purchase rate for the average e-commerce business usually doesn’t exceed 3-4%. Detecting a change to that small percentage requires a high volume of visitors in the experiment. If your company has low-to-medium traffic, this could take a couple of months.
Tracking micro-conversions is an effective strategy in this case. They usually occur more often than macro-conversions because they’re closer to the tested change. So, you're more likely to detect a change in a reasonable period of time.
However important revenue is to a company, not all value that you will provide for your users will be directly captured as more revenue. Delighting your users by providing them with a top-notch experience is likely to increase important high-level goals such as user satisfaction, retention, or better engagement.
- Lev Tatarov, Strategy Consultant at Optimizely
Primary and secondary events are another way to think about your metrics strategy. Read about in this article.
Example: E-commerce metrics
Imagine you run an experiment on the product page where you highlight a certain discount. You hope to make a purchase more attractive to visitors and increase revenue.
The macro-conversion for case is purchases. If you think you can get a statistically significant answer on that metric -- that's great! But in most cases, purchase completions is a few steps away from the product page. So, it's more effective to track a micro-conversion like clicks to the Add-to-Cart button.
Clicks to Add-to-Cart button is the event that's closest to the change on the page and most likely to be affected. You have a much higher likelihood of getting a statistically significant result.
But how do you measure overall impact?
Let’s assume these baseline conversion rates for each step in the funnel:
Product page -> Shopping cart: 20%
Shopping cart -> Cart checkout: 75%
Cart checkout -> Purchase completion: 60%
If you see a 10% increase to the add-to-cart rate, you can estimate the effect on purchases by multiplying the rates.
Estimated increase in purchases = 10% * 20% * 75% * 60% = 0.9%
This type of calculation helps you estimate the overall value you'd expect if this change were implemented on the site.