Skip to main content
menu_icon.png

Everything you need to switch from Optimizely Classic to X in one place: See the Optimizely X Web Transition Guide.

x
Optimizely Knowledge Base

How Optimizely counts conversions

relevant products:
  • Optimizely Classic
  • Optimizely Classic Mobile
  • Optimizely X Web Experimentation
  • Optimizely X Web Personalization
  • Optimizely X Web Recommendations
  • Optimizely X Full Stack
  • Optimizely X OTT
  • Optimizely X Mobile

THIS ARTICLE WILL HELP YOU:
  • Understand how Optimizely counts conversion events
  • View examples for how conversions are counted under a few, specific conditions
  • Determine how results in Optimizely Classic and Optimizely X are counted differently

Imagine that you’re running an experiment on all the product detail pages on your site. A user lands on one of those pages and enters the experiment. They’re bucketed into a certain variation and convert on an event you’re tracking: clicks to the Add-to-Cart button. As the user navigates around your site, they convert a few more times. How exactly does Optimizely count those events and display them on your Results page?

This article explains how Optimizely counts conversions for a variation or personalized experience; it's Optimizely’s attribution model. Conversion events are attributed slightly differently in Optimizely X than in Optimizely Classic. They are also a few major differences in how conversion events are attributed in Experimentation versus Personalization. This article explains these different calculations.

We’ll also cover how conversion events are counted under a few, specific conditions:

  • When you target an experiment to certain users via audience conditions

  • When you filter the results only for a certain audience segment

  • When you filter results only for a certain date range

Jump straight to the general rules.

Here’s a diagram that follows a single user’s activity on your site:

example timeline for a single user

We’ll use this example to follow how Optimizely’s different products count conversions and how events are counted in specific situations.

  • E is a conversion event, such as a click to the Add-to-Cart button.

  • D is a decision point, when Optimizely decides whether the user is bucketed into a certain variation based on URL and audience targeting conditions. In this example, all decision points in the timeline are for the same variation.

  • A session is a period of activity for a user. An existing session ends and a new session begins after 30 minutes of inactivity on the site. The maximum session length is 24 hours, at which point a new session automatically begins (even if the user wasn’t inactive for 30 minutes).

  • Unless otherwise noted, we'll always assume that the user is shown the same variation in Sessions 1 and 3.

The example in the timeline diagram shows the user’s activity on your site. In the middle of Session 1, the user lands on a URL where the experiment is running, and a decision (D) is made based on targeting conditions. The user is either shown the variation or not and converts two more times. After 30+ minutes of inactivity, a fresh session begins. The user fires the conversion event (E) five times during Session 2. In Session 3, the user converts twice, and a decision (D) is made again -- perhaps that the user has returned to a product detail page and the decision is evaluated again. The same user also fires a conversion event (E) five more times during Session 4.

Below, we’ll use this same timeline to walk through how the conversion events are counted in different products and under certain conditions.

General rules

Here are a few basic rules that Optimizely follows when counting conversion events.

User scoping versus session scoping

User-level scoping

When a user is shown a variation, all subsequent conversion events from that user are attributed to that variation, for as long as the experiment is active.

Products with user-level scoping:

Optimizely Classic
Optimizely X Web Experimentation
Optimizely X Web Recommendations
Optimizely X Full Stack
Optimizely X Mobile
Optimizely X OTT

Session-level scoping

When a user is shown a variation, all subsequent conversion events that are fired during that user session are attributed to that variation. 

Products with session-level scoping:

Optimizely X Web Personalization

Common counting scenarios

Audience targeting

Optimizely Classic and Optimizely X use different attribution rules for users who initially qualify for and later do not qualify for the same experiment.

Example: A user is bucketed into an experiment that targets signed-in visitors and is shown a variation. The same user returns the following day, triggering another decision for that experiment. She no longer qualifies because she is not signed in.

Optimizely Classic attributes all events from this user from the moment she's bucketed into a variation until the experiment ends.

Optimizely X attributes all events from this user from the moment she was bucketed until she's disqualified from the experiment.

Filtering by segment

Optimizely Classic counts all events that are fired by the user while she is in this segment.

In Optimizely X, if the user is in a given segment at the end of a session, then all events in that session are attributed to that segment when results are filtered by that segment.

Filtering by date range

Optimizely Classic will count all events that are fired by the user during the given time range.

In Optimizely X, if the session starts in a given date range, then all events in that session are counted when results are filtered by that date range.

Resetting results

In Optimizely Classic, events are counted immediately after the reset point.

In Optimizely X, events in new sessions that begin after the reset point are counted.

Below are a few examples that explain how events are scoped to the user or session level and how this impacts counting.

All Optimizely products, except Personalization, have user-scoped results. The timeline below shows how conversion events (E) for a single user are counted when results are scoped to the user.

Optimizely counts 16 total conversion events (E) and attributes them to one user. This is user-scoped event attribution. When the user is shown a variation, all subsequent conversion events from that user are attributed to that variation, for as long as the experiment is active.

Users = 1

Total conversions = 16

Unique converting users = 1

Optimizely X Web Personalization is the only product that has session-scoped results. Here’s the same event timeline as above, but counted by session instead of user.
session-scoped countingWhen the user is shown a variation, all subsequent conversion events that are fired during that user session are attributed to that variation. Optimizely uses session-scoped metrics in Personalization because it’s not uncommon for a single user to see several different personalized experiences over time in behaviorally-targeted campaigns.

Sessions = 2

Total conversions = 4

Unique converting sessions = 2

Next, see how event counting works under several specific conditions: including or excluding users based on audience conditions, filtering the Results page by segment, and filtering the Results page by time range.

Audience targeting

Optimizely lets you target experiments and personalized experiences to groups of users (called audiences) who share attributes that you define. This section explains how Optimizely attributes a single user’s events over time as they navigate around the site.

When the user first visits the site, Optimizely buckets them as a signed-in visitor and shows them a variation during Session 1. When the user returns the following day, they trigger another decision for that experiment in Session 3, and so no longer qualify because they’re now signed out.

Here’s an example for Optimizely Classic:

Optimizely Classic attributes all events from this user from the moment they are bucketed (Session 1) until the experiment ends.

Users = 1

Total conversions = 16

Unique converting users = 1

Here’s an example for all Optimizely X products, except Personalization:

All user-scoped Optimizely X products attribute all events from this user from the moment they are bucketed (Session 1) until the moment they are disqualified from the experiment (Session 3).

Users = 1

Total conversions = 9

Unique converting users = 1

Here's an example for Personalization:

During Session 1, the user is bucketed an experiment targeting signed-in visitors. Only the subsequent events fired during that user session are attributed to the variation.

Sessions = 1

Total conversions = 2

Unique converting sessions = 1

"Purchase" event example

Below is an example that highlights the difference in how Optimizely Classic and Optimizely X count conversions.

Let's imagine again that a visitor to your site is bucketed into a variation during Session 1, and the visitor triggers a decision for the experiment in Session 3 that means they no longer qualify. Now, suppose this visitor makes a purchase after they're disqualified in Session 3.

In Optimizely Classic, all events are attributed to this visitor until the experiment ends, so the unique converting user count would be 1. In Optimizely X, events are no longer attributed to this visitor after they're disqualified, so the unique converting user count would be 0. In other words, in Optimizely Classic, you have 100% conversion, but in Optimizely X, you have 0% conversion—for the same visitor and the same event.

So if you are running experiments on an e-commerce site, switching to Optimizely X could mean that you see fewer conversions based on purchase events. This is because Optimizely X doesn't attribute events to visitors after they're disqualified from an experiment.

Filtering by segment

When you look at your results in Optimizely, you can use segmentation to see how a subset of users respond to the variation (instead of your users overall). This section explains how Optimizely attributes conversion events when filter your results by segment. If the user is in a given segment at the end of a session, then all events in that session are attributed to that segment when results are filtered by that segment.

These are the basic counting rules for segmentation:

  • Optimizely Classic: If the user is in a given segment, all events fired are attributed to that segment.

  • Optimizely X: If the user is in a given segment at the end of a session, then all events in that session are attributed to that segment when results are filtered by that segment.

Here’s an example for Optimizely Classic:

All events that fire while the user is in the segment are counted. When the user is no longer in the segment, the event is not attributed.

Users = 1

Total conversions = 12

Unique converting users = 1

Here’s an example for all Optimizely X products, except Personalization:

If the user is in a given segment at the end of a session (like Sessions 1, 2, and 3), then all events in that session are attributed to that segment when results are filtered by that segment.

Users = 1

Total conversions = 11

Unique converting users = 1

Here’s an example for Personalization:

When the user is in a given segment at the end of a session (like Sessions 1 and 3), all subsequent conversion events that are fired during that specific user session are attributed to the variation.

Sessions = 2

Total conversions = 4

Unique converting sessions = 2

Filtering by date range

In Optimizely, you can also filter your results by date range. For example, you might want to see conversions for Jan. 1 - Jan. 20 on your Results page. This section explains how Optimizely attributes conversion events when filter your results by date range.

Here are the basic counting rules for date ranges:

  • In Optimizely Classic, all events that fire during the date range are counted.

  • In Optimizely X, if a session starts in the filter’s date range, then all events in that session are counted when results are filtered by that date range.

Here’s an example for Optimizely Classic:

All events that are fired during the date range are counted. 

Users = 1

Total conversions = 11

Unique converting users = 1

Here’s an example for Optimizely X Web, except Personalization:

If the session starts in the given date range (like Sessions 2, 3, and 4), all events in that session are counted when results are filtered by that time range.

Users = 1

Total conversions = 14

Unique converting users = 1

Here’s an example for Personalization:When a user is bucketed into the experiment and the session starts in the date range (Session 3), all subsequent conversion events that are fired during that user session are attributed to the variation.

Sessions = 1

Total conversions = 2

Unique converting sessions = 1

Here's an example in Full Stack:

Like all Optimizely X products except Personalization, Full Stack, Mobile, and OTT count events based on user-level scoping. If the session starts in the date range (like Sessions 3 and 4), all subsequent conversion events that are fired after the user is bucketed into the experiment are attributed to the variation

Resetting results

In Optimizely, you can reset your Results page. This section explains how a user’s events are attributed to an experiment or campaign when you reset the page.

When you reset your results, all sessions that began before the reset point are discarded, even if those sessions persist beyond that point.

Here's how to reset results in Optimizely ClassicOptimizely X (except Personalization), and Optimizely X Web Personalization.

Here are the basic rules:

  • In Optimizely Classic, events are counted immediately after the reset point.

  • In Optimizely X, events in new sessions that begin after the reset point are counted.

Here’s an example for Optimizely Classic:

Users = 1

Total conversions = 12

Unique converting users = 1

Here’s an example for Optimizely X:

Users = 1

Total conversions = 9

Unique converting users = 1

Here's a detailed diagram of how Optimizely counts events when the Results page is reset, by Ali Rizvi.