Skip to main content
menu-icon.png

 

x
Optimizely Knowledge Base

What are monthly active users (MAUs) in Optimizely

THIS ARTICLE WILL HELP YOU:
  • Define how monthly active users (MAUs) are counted from a technical perspective
  • Distinguish between monthly active users (MAUs) and impressions
  • Forecast monthly active users and view monthly active users consumption

Starting September 2020, Optimizely has introduced a simplified usage billing component: Monthly Active Users (MAUs), which replaces impressions

Your Monthly Active Users (MAUs) is the number of unique users used in a decision or tracking event. It is a measure of the overall traffic where you are using the snippet, APIs, or SDKs to do something, specifically:

  • Experiment Evaluation
  • Personalization Campaign Evaluation
  • Feature Flag/Rollout Evaluation
  • Event Tracking

Unlike impressions, it is not a measure of what percent of that traffic you’re experimenting on - every user that gets evaluated is counted. This allows you to run large-scale experiments at 100% traffic in order to reach statistical significance more quickly.

This article provides an overview of how MAUs work at Optimizely from a technical perspective.

Full Stack

In Full Stack, Optimizely counts a monthly active user each time a decision or tracking event is sent for a unique user ID:

  • When the optimizelyClientInstance.activate()method is used (experiment evaluation).

  • When the optimizelyClientInstance.isFeatureEnabled()method is used (feature flag/rollout evaluation).

  • When the optimizelyClientInstance.track()method is used (tracking event).

Users are counted even if they receive a disabled flag as a result of activate or isFeatureEnabled because a decision event was made.

Optimizely Web

In Optimizely Web, MAUs include the number of unique anonymous IDs that the Web snippet used in a decision or tracking event:

  • Experiment Evaluation
  • Personalization Campaign Evaluation
  • Event Tracking

These are only counted on the pages where experiments/campaigns are running or events are being tracked - an empty snippet won’t count toward MAUs.

De-Duplication

As long as it’s the same unique user ID, we dedupe. For example, if a user goes to a page with one project's snippet and then a page with another project's snippet, only one MAU is counted as long as it's the same unique user ID.

If you use both Optimizely Web and Optimizely Full Stack, you can override anonymous Web user IDs with known Full Stack user IDs. Reach out to your Optimizely account contact for more information.

Decision event

Each time a Full Stack experiment or a page within a Web experiment is activated, a decision request is sent to Optimizely. Decision requests look like this:

decisionrequest.png

In the request payload, the decision attribute indicates the experiment that it applies to.

Example

Let's walk through an example scenario. There are three multipliers:

  • Experiments

  • Pages (as defined in Optimizely)

  • Pageviews

Imagine that your company, Attic and Button, is experimenting on www.atticandbutton.us. Consider a visitor who starts by visiting the Attic and Button homepage, where there are three experiments running. One of these experiments has two Optimizely pages that both target the homepage:

Experiment 1

Experiment 2

Experiment 3

Three bucketing decisions are made on whether to include that user in the experiment or not, but since the user ID is unique, this will only count towards one monthly active user (MAU). This is in contrast to impressions, which would have counted four impressions for this one user.

maus_1.jpg

If the visitor refreshes the page, they will still only count as one monthly active user. This is in contrast to impressions, which would have generated another four impressions, making the total eight impressions for this one particular user.

maus_2.jpg

Now, suppose that you’re running a search algorithm experiment with Full Stack on the homepage too. When a visitor types a search term, the results are refreshed without reloading the page. The Full Stack SDK makes a decision for a variation every time a new search is done. This means that if a visitor searches for "shirts," changes their search to "denim shirts," then changes their search again to "button down shirts," only one monthly active user would be counted since it's based on a unique user ID. This is in contrast to impressions, which would have counted three impressions, making the total usage count now 11 impressions.

maus_3.jpg

Verifying monthly active users with results export

Optimizely uses the server timestamp to calculate monthly active users, as opposed to the timestamp on the client device where the monthly active user originated. Doing so makes it possible to accurately verify monthly active users  all the way down to the experiment level.

You can use Optimizely's results export to get a complete list of all monthly active users that occurred within a specific time period. You can then compare that information to your invoice, or determine whether any of your experiments are generating more monthly active users than they should be. To learn how to access that data, check out our developer documentation article on data export services in Optimizely, and follow the instructions for results export (and not raw data export).

How to Forecast MAUs

In order to forecast MAUs, we recommend you ask the following questions:

  1. Where will you be using Optimizely? 
    These could be multiple channels, regions, business units, etc. (e.g. Android, iOS, website, etc.)
  2. How many unique users/visitors do you have per month? 
    Include unique users/visitors, not pageviews. Excludes bot traffic. We recommend you get these number from internal analytics, but they can also be estimated from SimilarWeb. Note: Provide the average across the year, note the peak.
  3. How complete is your Optimizely implementation?
    If the SDK/snippet runs for most users (e.g. feature flags in app navigation, or experiments on PDP layout), this should be 90-100%
    If Optimizely is only used on a small part of the site (eg. personalizing a few landing pages, rolling out minor features only) this could be as low as 5-10%
  4. How much annual growth do you expect?
    Year over year, how much more traffic do you expect?

MAUs are a shared pool between Web and Full Stack products, and roll over the next month if not used.