Skip to main content
menu-icon.png

 

x
Optimizely Knowledge Base

Access Optimizely raw data

 
relevant products:
  • Optimizely X Web Experimentation
  • Optimizely X Full Stack

THIS ARTICLE WILL HELP YOU:
  • Access Optimizely raw data
  • Determine what resources you will need to use the raw data
  • Understand how the data export process works and what you can expect

To calculate results, Optimizely relies on event data obtained through your experiments. When an experiment activates, we capture a decision event. When a page activates, we capture a view activation. If a visitor converts on one of your metrics, we capture a conversion event.

On the Results page, you can export final calculated results to a CSV file, but you might also want to look into each individual event and the metadata it might offer. We recognized the value of being able to examine raw event-level data and developed an export process that gives developers direct and secure access to their raw data through Amazon S3

Keep reading to learn more about how to access the raw data. Amazon S3's cloud-based online storage is a technical developer feature that will require some development effort on your end to set up automatic data ingestion or export one-off data. Optimizely support can help by transmitting and validating credentials using the steps described below. However, we don’t recommend any specific tools or applications to access the data, and Optimizely Support can't provide ad hoc data exports.

Check out our developer documentation for technical details about Optimizely's raw data.

Currently, the Raw Data Event Export feature is available on select plans; please reach out to your Customer Success Manager if you wish to utilize this feature. If you do not have a CSM, submit a ticket to the developer support team to verify your plan and eligibility.

Access Optimizely raw data via Amazon S3

Optimizely can provide you with a set of credentials you can use to access our Amazon S3 bucket. We need to transmit these credentials in a secure way—so emailing them won't suffice!

We use PGP encryption to facilitate secure transmission for Amazon S3 credentials. Here's the step-by-step process:

  1. Generate a public and private PGP key.

  2. File a support ticket to request raw data access. Make sure to include your public PGP key in your ticket.

  3. Optimizely Support will send you an encrypted file with your Amazon S3 credentials.

  4. Decrypt the credentials from Optimizely Support.

Validate your credentials

Next, you'll need to verify that the credentials from Optimizely Support provide you with access to the raw data:

  1. Install AWS command line tools.

  2. Run aws configure to input your access key and secret access key.

  3. List the files in your directory with aws s3 ls s3://optimizely-export-ng/<account id>/.
    The final forward-slash is necessary because your credentials will only provide access to one folder inside the Optimizely Amazon S3 bucket. Here's a link where you can access your account ID

You should see a list of your projects that have had running experiments:

us-troddy:~ troddy$ aws s3 ls s3://optimizely-export-ng/2779270234/
                           PRE 3219400867/
                           PRE 7620220811/
                           PRE 8431340120/
us-troddy:~ troddy$ 

If you see An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied , please try step 2 again and double-check that you copied and pasted your credentials correctly. Then, confirm that you’re including the final forward-slash after your account ID.

Raw data location

Your raw data will be within s3://optimizely-export-ng/<account id>/. You can do a recursive list operation to see the full structure within your directory. 

Troubleshooting

  • I don't see status.yaml

    If you do not see a status.yaml file, the raw data is still copying into the S3 bucket. When the status.yaml file appears, the process is complete and it is safe to begin copying the raw data.

  • The status.yaml file says export failed.

    Please file a support ticket so we can look into this issue.

  • I need a script or code for accessing my data.

    We do not recommend specific programs for accessing raw data through Amazon S3 or provide scripts for data retrieval. You may need technical help on your end to develop a solution that meets your business needs.

  • My raw data doesn't match the results page.

    It's important to understand that the raw event data is not raw results data. The data we provide access to is a raw list of events we received — it is not a list of raw results data. Event data goes through an attribution process described in our knowledge base here. Our raw data comes from before attribution occurs.

    Raw event data will contain events from users who may or may not count for an experiment. Events may exist in exports outside of the time frame in which an experiment ran. Re-creating the results page's numbers out of raw data will be non-trivial as it would require re-creating our entire attribution model in your queries.

  • The password column is missing in the credentials file I received from Technical Support.

    We do not provide access to the AWS web console which is why there is nothing in the password column. All you need is the AWS_ACCESS_KEY_ID and the AWS_SECRET_ACCESS_KEY to access your data.