- Implement the changes from a successful variation permanently on your site
- Involve a developer in your optimization process
At the end of your testing cycle, you’ll decide which changes to implement permanently on your site. When you decided on a winning variation, you likely sent all traffic to that experience by pausing your running experiment, duplicating it, and allocating all traffic to the winning experience. Next, you'll work with a developer to integrate these changes permanently into your site's codebase.
Sometimes, the moment of the handoff to a developer can be a delicate one. If the optimization team hasn’t prepared the developer for implementation, the project may hit a slow point or even be de-prioritized. For example, a new feature may test well -- but take a month of your developer's time to build.
By looping your developers in early, you'll smooth the process of implementing permanent changes from testing and build a stronger optimization culture in the process. Read on for more about how and when to integrate developers into your testing cycle.
- Code from Optimizely variations
- Documented changes from your roadmap
- Identify all new functionality of the variation
- Identify regression test cases, where changes may break functionality on your site
- Identify upcoming site changes
People and resources
- QA team
- Verify that new code functions correctly
- Verify changes for regression testing
- Update production environment
- Update staging environment
- Notify QA team, Marketing, Support, and Engineering
- Updated versions of pages on developer environment
- Updated version of pages on production environment
- Not sharing about new site changes that might affect other experiments with the entire team
This article is a part of the Optimization Methodology series.
What the developer should know
To implement the changes that you decide to make, your developer will translate Optimizely’s jQuery variation code into the framework that your site is built with. In the beginning, you might be able to add Optimizely’s variation code directly to your site, but over time this process won’t be sustainable when you implement changes from multiple experiments. A developer will translate the variation code and integrate it into a part of your site’s framework.
This process requires familiarity with both your site’s codebase and Optimizely. It may also require the developer to translate some of Optimizely’s client-side script into server-side script. Help smooth your developer’s path by providing context around the experiment you’ve run -- and Optimizely.
A few things your developer should know about Optimizely:
The order of execution in Optimizely's snippet
How Optimizely evaluates variation code
Documentation for developers
Also, share your results document with the developer. A detailed plan provides context for the changes that will be implemented.
Your results document also provides a glimpse into the power of Optimizely to create a quick proof-of-concept before fully dedicating developer resources. Only site changes that have been tested and validated with data are passed to the development team for implementation.
Integrate your communications about the site build with your developers’ communication systems -- often JIRA or Trello. Create an unified record so potential communication gaps - such as the overlapping deployment of live features and experiments - don’t cause conflicts that break the site.
The workflow might look something like this:
Create a test based on your experiment plan, with your developer’s input.
If a variation is successful, duplicate the experiment and set traffic allocation to 100% to show all visitors.
Share results with stakeholders, including the developer.
Developer enters Optimizely experiment and reviews variation code with the project manager.
Developer abstracts the code and builds a permanent feature in the production environment; then, verifies that the new feature works correctly.
Developer notifies the optimization program manager when the feature goes live.
Optimization program manager turns the experiment off and archives it.
Implement the same feature in your development environment, if you have one.
Integrate your implementation process with the systems your developer uses to track projects to execute the handoff smoothly.
Get a developer involved early
As your optimization program matures, start to incorporate your testing program with your developer’s sprint cadence. Involve the developer in the testing cycle too.
As you do so, you may notice a decrease in the time it takes to translate successful changes to your live site. But the effort it takes to set up a new test may slightly increase. These slight shifts are good signs: your developer is becoming more deeply involved in testing. You’re probably running more complex experiments, too.
And by involving your developer early, you smooth the way for the very last step -- optimizing your live site. Optimizely makes it easy to create variations with static HTML, but a developer can help you design more efficient, dynamic experiments. Changes that you make with the Visual Editor generate code that’s executed in the browser after your site loads. Inefficient changes can balloon into hundreds or even thousands of lines of variation code.
For example, you may want to make a button more prominent, so you use the Move and Resize feature and edit the color a few times to get it just right. With every change, a new line of code is generated. Hundreds of lines of code can cause performance issues such as flashing.
A developer can also think ahead about how to abstract variation code into the code base. Then, when you decide deploy changes to the live site, they can build new or altered features more efficiently.