Skip to main content
menu_icon.png

 

x
Optimizely Knowledge Base

Use environments to QA your experiment code

relevant products:
  • Optimizely X Full Stack

THIS ARTICLE WILL HELP YOU:
  • Use environments to QA your experiment code before deploying it
  • Explore use cases for Optimizely environments
  • Run your experiment code in different environments
  • Manage your environments

The Optimizely environments feature lets you view and QA your experiment code before you reveal it on your website. Whether or not you already use a formal deployment environment, you can create environments to ensure the quality of your experiment code in an independent setting before you show it to users.

Essentially, Optimizely environments allows a single experiment to be running in some contexts and paused in others. You can turn an experiment on and run it for your team so they can QA it, but leave it turned off for users. Environments enable you to do this without duplicating your code, Optimizely project, or experiments within your project.

Environments are currently available only for Optimizely X Full Stack projects.

How environments work

Every Optimizely project starts with one environment (and one datafile) named Production. When you set up an environment, Optimizely generates a corresponding datafile. The datafiles generated by these additional environments represent the current states of all experiments within your Optimizely project. Each environment-specific datafile represents only the state of the experiments within that environment.

For example, imagine you have three experiments: sort-by-name, show-subtotal, and warn-empty-cart. You set up a new environment named Testing so you can QA your experiment code before you deploy in the Production environment. Suppose that:

  • You completed QA for the sort-by-name experiment and deployed it.

  • You are currently completing QA for the show-subtotal experiment.

  • Your warn-empty-cart experiment has completed QA but is not yet deployed.

Here’s an overview of the statuses of your three experiments in your two environments:

Experiment

Testing environment activity state and datafile status

Production environment activity state and datafile status

sort-by-name

Paused*

Running

show-subtotal

Running

Not started

warn-empty-cart

Paused

Not started

*You can keep experiments running in your testing environment that are also running in production if you prefer.

Now, imagine that the project owner creates a third environment: Development. Optimizely will generate an additional datafile named Development. Every experiment will be in the “Not started” activity state in Development, and the Development datafile will show experiment status as “Not started.”

When your experiment code is fully functional and ready to QA, switch it to the Testing environment by pointing your code to the Testing datafile.

Environment security

Each datafile has a different key at the end of the CDN URL, which you specify when you set up the environment. You can choose to use a straightforward key like “staging”, or obfuscate the URL by generating your own universally unique identifier (UUID) so that it won’t be easily guessed.

Improve security by restricting permissions for starting and stopping experiments in an environment to collaborators with Publisher permissions. This limits the number of people who can change your experiments’ statuses or move them from one environment to another.

We recommend keeping access open to everyone in local or development environments and restricting access in production environments. By default, the production environment is limited to Publishers and above, which is the same permission level that applies if you aren’t using the environments feature.

Use cases for environments

Here are two examples of environments use cases. You can have an unlimited number of environments, and there are many more possibilities for using them than we have described here. Use as many environments as your workflow requires. 

In general, the fewer environments you use, the simpler the workflow. If you use two or more environments, they’ll be grouped together on your experiment list as “Other Environments.”

Three-stage testing

The three-stage testing use case involves these separate, sequential environments:

  • Local: code is running only for the developer who is writing it. You don’t need to create a separate Local environment for each developer, just one environment where the experiment will run during development.

  • Staging: code is shared with the larger development team and QA.

  • Production: code is deployed and running for all users.

Two-stage “sandbox” QA

The two-stage “sandbox” QA use case is a common setup in web development and involves these separate, sequential environments:

  • Local: code is running only for the development team and QA.

  • Production: code is deployed and running for all users.

You can use the "sandbox" approach even if you don’t have separate deployment environments for your application. One common use case is to toggle between the local and production files for debugging. For example, you could load the local file when a certain “test” cookie is enabled, when a certain URL parameter is attached (like ?debug=true), or based on some other check that matches your internal workflow.

Create environments

By default, every project starts with a single default environment with a single datafile inside it that includes everything from the project. You can rename the default environment and add more environments as needed:

  1. Navigate to the Settings dashboard.

  2. Click New Environment…

  3. Enter the environment name and a datafile key. Specify the permission level you want to set for experiments in the environment.
    create-env.jpg

  4. Click Create Environment and Datafile.

  5. Implement the datafile in your application.

Learn more about accessing the datafile and best practices for datafile management.

Experiments are paused by default in any new environments you create, even if they were already running in the default environment.

Change experiment status in an environment

When you use multiple environments for a project, experiment status is no longer either “Running” or “Paused” on the Experiments dashboard. Instead, each experiment's status will be specified with icons that represent the state of the experiment in each environment:

environments.png

To change an experiment's status in an environment:

  1. Click the icon for the experiment.

  2. Select Run or Pause for the environment where you want to change the experiment's status.
    env-status.png

Learn how to designate which environment is shown to users in our article about best practices for datafile management.

Rename an environment

To rename an environment:

  1. Navigate to the Settings dashboard.

  2. Click the icon for the environment you want to rename.

  3. Click Settings.
    env-settings.png

  4. Enter the new name for the environment and click Save Environment.
    rename-env-2.jpg

Archive an environment

You can archive environments if they aren’t being used anymore (except the primary environment, which can't be archived). Archiving deactivates all the datafiles in an environment, replacing them with an empty file, and removes the environment as an option for experiments.

If experiments are running in the environment when you archive it, they will stop because archiving empties the datafile.

To archive an environment:

  1. Navigate to the Settings dashboard.

  2. Click the icon for the environment you want to archive.

  3. Click Archive.
    archive-env.png

Results by environment

To make sure traffic to your experiment in a staging environment isn’t mixed in with production environment results, reset your results when starting in production to clear any experiment conversions.

As an alternative, create an attribute in each environment and use segmentation to filter your results by attribute.