Skip to main content
menu_icon.png

 

x
Optimizely Knowledge Base

Get started with Optimizely X Full Stack

Skip Ahead

 
 
Relevant Products:
  • Optimizely X Full Stack

THIS ARTICLE WILL HELP YOU:
  • Get started with Optimizely X Full Stack
  • Create experiments in the programming language of your choice, including: Python, Ruby, Java, Node, PHP, C#, and JavaScript
  • Create, test, and rollout features with Optimizely Feature Management
  • Run experiments anywhere in your technology stack, using one of Optimizely’s SDKs
  • Use Optimizely’s Stats Engine to measure the impact of your experiments

Optimizely X Full Stack helps you run Optimizely experiments in any application or on any connected device. It enables product and engineering teams to build, deploy, test, and roll out features anywhere in their technology stack, with server-side and client-side testing on a single testing platform.

Use Full Stack to develop and experiment in areas like pricing, search result algorithms, or redesigns. A/B test across your technology stack or slowly roll out a new feature.

Full Stack 2.0 introduces a set of new capabilities collectively called Feature Management. Feature Management is designed to make it easy for product development teams to build, deploy, and iterate on features. Feature Management introduces several new concepts that you should become familiar with:

  • Features are the building blocks of your Full Stack experimentation program. Develop a feature, deploy your feature using the isFeatureEnabled API (the equivalent of a feature flag), then create and launch experiments without deploying again. Toggle your feature on/off for specific users by running feature tests and feature rollouts.

  • Feature configurations allow you to define variables that parameterize your features. Set the value of the variables in your feature configuration as part of an experiment to iterate on your feature in between code deploys. Use Optimizely’s getFeatureVariable APIs to access your feature configuration variable values in your application.

  • Feature tests are A/B tests tailor-made for features. Feature tests allow you to test a new feature against your existing user experience, or to iterate on an feature by defining variations based on feature configurations.
    Note that feature tests do not require using the activate API. Instead, isFeatureEnabled does all the work for you. When you call isFeatureEnabled, Optimizely checks for active feature tests, evaluates whether the current user is eligible for those tests, and activates the tests if the user is eligible.

  • Feature rollouts are the output of your product development process. When you’re ready to expose a new feature to your users, create a feature rollout, set a traffic allocation, and, if appropriate, target an audience. Leverage feature rollouts to deploy the results of your feature tests to the right users, at the right time, with the appropriate safeguards in place.

Finally, we continue to support traditional A/B tests:

  • A/B tests are one-off experiments designed to answer specific questions. A/B tests use the activate API, which returns the variation key corresponding to the variation assigned to a particular user. Leverage A/B tests to answer one-off questions while using feature tests to iterate on features over time.

Full Stack 2.0 introduces several other new conveniences, including environments, which allow you to map Optimizely to your development process; notification listeners, which allow you easily integrate Full Stack experiments with other services; and bucketing IDs, which allow you to assign users to variations with an identifier other than the user's ID (e.g., an Account ID).

To learn more about the new concepts introduced in Full Stack 2.0, see the following articles:

Use Full Stack SDKs to run tests in different languages and in any application. Selecting a language when creating a Full Stack SDK project allows us to show you appropriate code samples in the UI and inform you of SDK updates. Otherwise, our Full Stack SDKs are language agnostic, meaning that you can use a datafile from any Full Stack project type to initialize an Optimizely client within your app, regardless of which language you've designated for your project.

To learn more or add Full Stack to your account, contact your Customer Success Manager.

FAQs

Below, we answer some frequently asked questions.

How does Optimizely X Full Stack affect my application's performance?

Using server-side approaches with Full Stack is significantly faster than client-side approaches.

With client-side approaches, you add a JavaScript snippet to your web page. When a user opens that page, their browser can take hundreds of milliseconds or more to download this snippet and execute your experiments.

In contrast, when using server-side approaches with Full Stack, the backend does the work beforehand. The experiment code runs before the web page ever loads. Full Stack uses in-memory bucketing so it already has all of the information it needs locally and users don't perceive any noticeable latency.

We’ve built our SDKs so you can split traffic to experiments without any network requests. All decisions are made in-memory based on the datafile cached in your application so there is negligible impact on latency.

Although you may be familiar with other platforms that call out to a third-party server to make experiment decisions, Full Stack makes all of these decisions in memory using a cached copy of the datafile. In other words, Full Stack is faster because it doesn't make any blocking API requests to get decisions about which experiment variant to use; the device (your mobile phone or server) running the code makes that decision in less than a millisecond.

The bucketing approach described above means that Full Stack won't slow down the end user's experience. However, there are still a few performance considerations to keep in mind as you scale your usage of Full Stack:

  • When and how often to download the datafile that in-memory bucketing uses. Manage this by limiting the files size download frequency.
  • When and how often to send data about conversion events for tracking. You can manage this by batching events and using asynchronous event dispatching (having the app send data out-of-band later).

For more information, see:

Please contact us if you’re interested in seeing performance benchmarks for any of our SDKs.

How can I use Feature Management in Full Stack 2.0?

Optimizely must whitelist you for Feature Management because all of your installed SDKs must meet the minimum version requirements. Older SDKs are incompatible with Feature Management. To get whitelisted for Feature Management, file a support ticket.

What SDKs are compatible with Feature Management?

The minimum SDK versions for Feature Management are:

  • JS: 1.4.3 and later

  • Ruby: 1.2.0 and later

  • Python: 1.1.1 and later

  • PHP: 1.1.1 and later

  • Java: 1.6.0 and later

  • C#: 1.2.0 and later

  • Node: 1.2.3 and later

How do I target my experiment to a group of users?

Our SDKs allow you to conditionally activate experiments based on user attributes you provide. Define user attributes in the Optimizely app as well as audiences consisting of one or more user attributes. An attribute is comprised of an ID, a key, and a segment ID. All of these are strings. Therefore, any values that are stored as attributes must be converted to strings.

How do I QA or preview my experiment?

Use our whitelisting feature to force users into specific variations for QA purposes. The whitelist feature allows you to specify a list of user IDs and their corresponding variations. We're working on adding an option to force users into variations from the SDK as well as a preview mode for our iOS and Android SDKs.

How do I make experiments mutually exclusive?

SDK projects support mutually exclusive experiments out of the box. Simply create a group and assign experiments to that group.

Do your SDKs handle bot detection?

Our SDKs support bot detection as of version 2.1. Enabling bot filtering requires enabling the feature on your project Settings dashboard. Once the setting is enabled, events sent from web browsers have bot filtering applied automatically. Applying bot filtering to events sent from other environments requires configuration. For more context, read our developer documentation about bot filtering.

How do I access the raw events I’ve sent to Optimizely?

Our raw data export feature allows you to export all of the events you’ve sent to Optimizely on a daily basis.

What user IDs should I use in activate() and track()?

We designed our SDKs to be platform-agnostic, so you can use whatever user ID makes the most sense for your experiments. If you’re experimenting on anonymous users, we recommend using a cookie, device ID, or an automatically generated ID from your analytics provider. If you’re experimenting on a logged-in application or device or experimenting across multiple channels, you can use a hashed email address, universal user identifier (UUID), or any other unique ID.

How does Optimizely consistently bucket users across SDKs?

All of our SDKs use deterministic bucketing via MurmurHash3 to determine what experiments and variations should be active for a user. This ensures that users will be given the same treatment across multiple visits or on different channels. We also ensured that all of our SDKs give the same output no matter what language you’re using.

Can I use the same datafile across different SDKs?

Yes. The datafile is language-agnostic, and you can use the same datafile from the same project across different SDKs and you would get consistent bucketing across them. This way, you can activate and track the exact same experiments across different SDKs and get consistent results.

How does Optimizely work with multiple apps and services?

It is possible to perform the traffic splitting in one backend using your SDK of choice and track conversion events in multiple different places using other SDKs, such as other backend services and mobile clients, or from the browser with our JavaScript SDK.

How does datafile management work across the Optimizely SDKs?

The recommended way to manage the datafile  across multiple SDKs is to have a centralized service that subscribes to the datafile webhook on your project and pushes updates to other services that subscribe to datafile file changes. Those services in turn would re-instantiate their Optimizely client using whichever Optimizely SDK they have.

How do I track user events that occur client-side in a web browser?

We created a lightweight JavaScript SDK that can be used for tracking conversion events from a web browser. Note that this is different than our standard JavaScript snippet, so it won’t interfere with any experiments that you’re running client-side in the browser.

How is Full Stack compatible with caching?

Many of our customers are using a CDN to cache content, which means that using one of Optimizely’s SDKs in the backend may require custom configuration. There are several ways you can use Full Stack in your caching architecture:

1. (Recommended) Use an off-the-shelf load balancer provided by your CDN.

a. Implement one of our SDKs at this layer to run bucketing logic.
b. Use the variation key you get from bucketing and the URL as part of the cache key to determine which variation to show.

2. Build your own load balancer on top of your CDN and perform bucketing logic in it to determine which variation to show the user.

3. Use CDN side-scripting such as Edge Side Includes (ESI) for Akamai to run bucketing logic to determine the correct variation to return.

4. Run inline JavaScript code in the of the page.

a. Add a script with our JavaScript SDK and datafile.
b. Script will run bucketing logic, set a cookie or URL query param, and reload the page.
c. The caching layer should use the query param or cookie along with the URL as the cache key to determine which variation to show.