- Build a business intelligence report based on multiple data sources
- Generate hypotheses for testing and personalization
- Use a template to create your own intelligence report
Ideation, the process of generating powerful hypotheses to drive experiments and campaigns, is one of the most important steps in experience optimization. It’s also one of the hardest.
Why? Optimizely's 2015 Benchmark Report suggests that optimization programs face three hurdles when it comes to ideation:
- Many companies over-rely on quantitative and behavioral data for ideation
- Most organizations use too few data sources as inputs for their hypotheses
- Teams don't build a consistent process for consulting the data sources they use
Without a structured framework supporting the creative stage of experience optimization, it can be difficult to consistently generate impactful ideas. If your team needs help with ideation, consider building a business intelligence report to turn your data into action.
A business intelligence report gathers key insights about your site and industry into a single document. When you brainstorm hypotheses, the report helps foster creativity and ground your team's idea generation in data.
When creating your report, draw from a variety of sources to build different perspectives into brainstorm process. Consult direct data, or intelligence directly about your business such as heat maps, personas, and web analytics. Look to indirect data for industry expertise such as best practices and competitors' sites. Goal trees help you connect your optimization ideas to company goals.
An intelligence report rich in stories about your customer journey will help your program plan impactful ways to optimize your site. Use insights from your report to generate ideas for testing and personalization.
Click to create your own business intelligence report from this template.
Materials to prepare
People and Resources
- Program Manager
- Design resources
- Collect data for the intelligence report
- Analyze your data
- Create a report
What you end up with
- A business intelligence report
What to watch out for
- Remember to perform this process year-round
- Don't neglect to use this report for decision-making about the site in general, instead of just for ideation
Consult multiple data sources
Direct data, indirect data, and goal trees all provide different perspectives on your site experience.
Together, they provide context and insights beyond what you might see in a single source.
For example, imagine that your web analytics report (direct data) reveals that visitors drop out of the funnel on a certain page. As a result, you may decide to test a clearer CTA to push visitors further down the funnel. But in the process of reviewing competitors’ sites (indirect data), you notice another site personalizing their CTA. After reading about this idea on an industry blog, you decide that this tactic has the potential to be highly impactful. At this point, you weigh your options and decide to test whether personalizing the CTA works with your visitors.
In this example, the direct source - your web analytics - tells you where to optimize. Indirect sources complement this data by suggesting how you might approach this opportunity. When used together, different types of sources shed light on questions surfaced by another data source.
Build multiple sources into your intelligence report to gain more context for your visitors’ behaviors and generate meaningful hypotheses.
Build a business intelligence report
The comprehensiveness and depth of your intelligence report will depend on your program’s resources and maturity level. But any program, no matter the size, can use the report to strengthen its ideation process.
Here are a few steps to help you get started:
- Create a timeline for producing your report. Choose a final deadline before your next major planning session.
- Outline the inputs for your report:
- Identify all of the data sources that your organization currently uses.
- Use the list of direct and data sources below as guides to identify potential sources to add. Look for resources you don’t currently use but can feasibly incorporate given team constraints. Focus on sources that are impactful for your business.
- Remember, the more data sources you employ, the more context you’ll build around your visitor experience.
- Assign responsibility for different aspects of the report to internal and external stakeholders. For example, ask the marketing team to review value propositions on competing sites or the web analyst to provide funnel segmentation analysis.
- Identify a few key insights from each data source to include in the report.
- Create summaries of these insights and gather them into a single deliverable.
- Once your report is ready, introduce it and socialize its use at the beginning of your planning meeting. Use the insights in the report to generate ideas for your campaigns and experiments.
- Return to the report at a regular cadence.
The following is a list of data sources to consider for your intelligence report.
Direct data, or what you know about yourself:
- web analytics
- results from prior experiments
- voice of the customer
- heat maps
- user testing
- user personas
- customer decision models
- lifetime value (LTV)
- customer acquisition cost (CAC)
To learn more, read about how to use direct data sources to build an intelligence report.
Indirect data, or information that you gather about your industry:
- your competitors’ sites
- best-in-class experiences
- user groups
- academic literature
To learn more, read about how to use indirect data sources to build an intelligence report.
Goal mapping helps you align your campaigns and experiments with the company's goals. Add your goal tree to your intelligence report to focus ideation on KPIs adn leading indicators that you're optimizing for.
To learn more, read about building a goal map to drive impact for your program.
Review at a regular cadence
Prepare your business intelligence report so it's ready for review at major team planning and decision-making sessions. Do this bi-weekly, quarterly, or monthly, depending on how often your team meets for ideation.
Focus on stories and insights
To get the most out of your report, fill it with it with meaningful insights and data that has a story to tell about your visitors. An intelligence review should feed your team's creative process; it helps you bridge the gap between raw data and hypothesis.
Use takeaways, summaries, and quick insights in your report to tell stories about your customers' desires, optimization opportunities, and competitive advantages. Consider leaving out other metrics that are more important for a performance report on KPIs and revenue; for ideation, this information can be a distraction.
By building an intelligence report to help drive a structured, data-driven ideation process, you bridge the gap between data, A/B testing strategy, and personalization strategy to lay the groundwork for a sustainable program.