To understand your report, let's walk through the columns you'll see on your reports page. These columns will be slightly different if you're viewing a report for an A/B experiment or a multivariate experiment, but the interpretation is the same.
Estimated conversion rate range provides the most immediate insight into overall performance. You'll view this column to see how well each combination or variation is performing relative to your original content. If you're a numbers type, you can view the numerical range to the left, but we've visually displayed the performance in the bars to the right: when the bar is green, we're confident that a combination is performing better than the original. Yellow bars mean that we're not sure yet -- it could be doing better, or worse. Gray bars mean that it's performing on the same level as the original. A red bar means that the combination isn't doing as well as the original.
The chance to beat original column, immediately to the right, displays the probability that a combination will be more the successful than the original version. When numbers in this column are high, perhaps around 95%, that means a given combination is probably a good candidate to replace your original content. Low numbers in this column mean that the corresponding combination is a poor candidate for replacement.
Improvement displays the percent improvement over the original combination or variation. Because this percentage is a ratio of the conversion rate of a combination to the conversion rate of the original column, it will often vary widely. We suggest that you only concentrate on the improvement when a large amount of data has been collected and it can be considered more reliable.
Conversions/visitor is just that -- the raw data of how many conversions per visitor a particular combination generated.
If you're running a multivariate test, you'll notice that you have two sub-tabs: reports by combination, or by page section.
A combination report will show the performance results for all of the page combinations made from the page section variations you created for your experiment. By seeing how well a particular combination performs in comparison with the original, you can choose the most successful one to improve your business.
In contrast, the page section report focuses on which variations of each page section performed best. Keep in mind that simply picking the best-performing variations for each page section may not be as effective as picking a winning combination, since there may be interactions among variations that the page section report doesn't capture.
Page section reports have the same columns as combination reports, plus one more: relevance rating.
Relevance rating shows how much impact a particular page section has on your experiment. For example, if your headline page section showed a relevance rating of 0, you'd know that the headlines you used did not significantly distinguish themselves. Alternatively, a relevance rating of 5 for your image page section would show that there were one or more images which significantly differentiated themselves from the others, and that the images page section is important for conversions.
