Skip to main content
menu-icon.png

 

x
Optimizely Knowledge Base

Experiments in OpX

You can use OpX experiments to evaluate multiple versions of a layout, process, or flow to understand which version best improves your team operations and customer outcomes. 

You can also use OpX experiments to manage feature deployments, which can help reduce the risk involved in deploying changes to mission-critical Salesforce components. 

Here are some idea for how your team can use experiments:

  • Lightning page layout experimentation

  • Flow Builder experimentation

  • Process Builder experimentation

  • Custom backend process automation and experimentation

  • Feature management

Create an experiment

To create a new experiment record, follow these steps: 

  1. Click the dropdown caret next to the Experiments tab and click + New Experiment.

  1. Enter the Optimizely name for this experiment—and, if desired, a description—in the appropriate fields.

  2. Select either A/B or Multi-armed bandit from the Type dropdown.
    You should select A/B if you want greater certainty and statistical significance to support findings; if you expect a large sample size; or if you will be using the experiment for feature management. However, if you want to optimize for specific outcomes, or if you expect a limited sample size and are only interested in the directionality of findings, select Multi-armed bandit.
    Do not use Multi-armed bandit if you plan to use the experiment for feature management.

  3. From the Subtype dropdown, select Lightning Page Builder (if you are using the Lightning Page Builder experiment template to experiment with different page layouts), Flow Builder (if you are using the Lightning Flow Builder experiment template to experiment with different screen flows), or Other (for a different, custom use case).
    Do not use Fullstack or Web for your experiment subtype. These options are used exclusively for tracking traditional digital experiments run outside Salesforce.

  4. From the Status dropdown, select Not Started, Paused, or Running. We recommend a status of Not Started or Paused until your experiment has been configured.

  5. When you’re done, click Save.

Optimizely automatically generates a unique experiment ID and experiment key when you save a new experiment; for this reason, we do not recommend manually entering information into these fields.

Additionally, the Campaign field must remain empty at this stage. 

When you’re done, your experiment record should look something like this:

How your experiment status affects user experience

When you create a new experiment, you will generally give it a status of Not Started or Paused. You’ll change it to Running later, once you’ve had a chance to configure the experiment.

Changing the status to Running allows you to associate the experiment with a variation experience. Once you have completed all experiment configuration and your variations experiences are being served to users, changing the experiment status will affect how your users experience your site.

If you run an experiment and then change its status to Paused, OpX will by default show the first variation you create, regardless of the status of that variation. For example, if you were to pause a page layout experiment where your first variation is your standard page layout, and the second variation is your alternate page layout, your users would see your standard layout while the experiment is paused. Variation performance is not tracked when an experiment is paused.

Once an experiment’s status has been set to Running, it cannot be reset to Not Started again. However, setting it to Paused will allow you to temporarily stop the experiment. 

You’ll need to pause an experiment before editing it. Note that not all fields may be edited; once an experiment is running, you can no longer modify the Optimizely Experiment Name or Experiment Key fields.

You can archive an experiment when you no longer need it. This will prevent further changes to the experiment. You can also un-archive an experiment by changing its status to Paused.

Configure associated records

After you have created an experiment record, you’ll need to define the variations, events, metrics, and target audiences that will be associated with the experiment. In general, those steps will go like this:

  1. Define your metrics and events. You can select existing metrics and events, or you can create new ones (you’ll have to configure any new ones).

  2. Create new variation records.

  3. Create new variation experiences.

  • If you’re creating a page layout experiment, be sure to activate the experiment record before creating an associated variation experience; the experiment will run once the Lightning record page layout is activated.

  • If you’re creating a flow experiment, remember to activate experiment records after you create your flows, but before you activate the variation experience. The experiment will run once the experiment record is activated.

  1. QA your experiment.

  2. Review your experiment results.