This handy calculator helps you estimate the potential duration of your experiment. Try out various numbers of combinations and see how they affect the length of the experiment. For pages with very high traffic, the differences may be negligible.
Keep in mind that the calculated experiment duration is an estimate only. Not only will actual traffic and conversion rates vary in a real experiment, the actual improvement experienced with some combinations will drastically affect duration. If, for example, one combination exhibits a significantly higher conversion rate than any other, the experiment will conclude more quickly.
Estimate Test Duration# Test combinations # Page views per day % Visitors in experiment % Current conversion rate % Expected improvement |
The factors used in this calculator are:
Test combinations
This value is determined by the number of page sections and variations you've selected. To calculate, multiply the number of variations for each section together. For example, if there are three variations for the first section, and two variations for the second, the number of combinations would be 3 x 2 = 6. Keep in mind that this number can add up fast. If you designate six sections and want to test three variations for each, the number of combinations would be 3 x 3 x 3 x 3 x 3 x 3 = 729! The amount of time it will take to run your experiment is linearly proportional in the number of combinations you have created. Fewer combinations mean less time it will take to run your experiment.
Page views
This value reflects the average amount of traffic your test page gets per day. All other factors being equal, experiments on pages with higher traffic will achieve results faster.
% Visitors
This value indicates the percentage of visitors you want to participate in the experiment. A value of "50" would include only half of all visits; in effect, it halves the amount of traffic. The lower you set this value, the longer an experiment will take.
Conversion rate
This value should be set to the conversion rate the web page has experienced in the past with the original content. The lower the conversion rate of your website, the longer your experiment will take; sites that convert at 2% will take roughly half the time to finish than sites that convert at 1%.
Expected improvement
This value indicates how much better you expect at least one of the new combinations to perform over the original content. Expected improvement is expressed as a percentage; if the current conversion rate is 15% and you hope to double that to 30%, then you're looking for expected improvement of 100%. If you don't have a figure in mind, keep the default of 10%. Improvement is the single most dominant factor determining duration. A good lift is usually the result of a high-performing variation. If you can think of a significantly better headline, for example, than the original headline, then your experiment will reveal a winner and finish more quickly.
