CRO Test Significance Calculator

Has your test reached statistical significance?

Know when to
stop your CRO test.

Converted’s CRO Test Significance Calculator is a fast and easy way to check whether a test is at a point where you can effectively draw a conclusion. Testing software should offer this, but just in case you’ve run your own!

Significance Calculator
to begin enter your details below

Sessions (original)

Sessions (variation)

Conversions (original)

Conversions (variation)

Calculate

Your P Value is{{PValue.toFixed(3)}}

Your Test Is{{significant}}

Statistical significance is all about the probability of whether the results you see in a test are a fluke or not. So if you have a significance score of 95%, it is in fact saying that there is only a 5% chance that your Null Hypothesis is true and that you can be 95% certain that the data you are seeing from the test is not random. Statistical significance is used to help you understand the confidence you should have is a set of test results.
Null Hypothesis can be most easily explained as the following: ‘If I run this CRO A/B test then the new variation will not deliver a significant change.’ It means the hypothesis of your test is that there won’t be a significant change. By adopting the Null Hypothesis it means you set out in every test to prove yourself wrong. Not prove that your new variation is right. This is a key difference in mindset and approach. The P-Value is the figure that will help disprove your null hypothesis.
The easiest way we can explain the P Value is that if it is converted into a percentage (a 0.25 P Value becomes 25%), this is the percentage chance that your test is a fluke and your Null Hypothesis is true. P-Value runs between 0-1 with a low P Value number (say 0.05 for example) indicating a strong probability that your null hypothesis is false and the variant has a high statistically significant probability of being the winner if you were to run the test again and again. A P-Value close to 0.05, say 0.06-0.07 is considered marginal.
When we look at the P-Value you have to consider the scale. So as we mentioned a 0.05 P-Value indicates that there is only a 5% chance that the variant would not win if the test was run again or a 5% chance the result of the test was a fluke. However a 0.95 P-Value is the opposite. It is saying there is a 95% chance that the variant won’t win if the test is to be run again. Either way the result has reached statistical significance.

Get started with Converted.
0800 228 9122 – hi@converted.co.uk

How to find us

The North

Acero
1 Concourse Way
Sheffield
S1 2BJ

hi@converted.co.uk
0800 228 9122

The South

Platform 1
King’s Cross
London
N1C 4AX

hi@converted.co.uk
0800 228 9122

Contact Converted

CRO Test Significance Calculator

Know when to
stop your CRO test.

Significance Calculator
to begin enter your details below

What is statistical significance in a CRO test?

What is the null hypothesis in a CRO test?

What is the P-Value?

My P-Value is high but it still says the test is significant

Get started with Converted.
0800 228 9122 – hi@converted.co.uk

How to find us

The North

The South

CRO Test Significance Calculator

Know when tostop your CRO test.

Significance Calculator to begin enter your details below

What is statistical significance in a CRO test?

What is the null hypothesis in a CRO test?

What is the P-Value?

My P-Value is high but it still says the test is significant

Get started with Converted. 0800 228 9122 – hi@converted.co.uk

How to find us

The North

The South

Know when to
stop your CRO test.

Significance Calculator
to begin enter your details below

Get started with Converted.
0800 228 9122 – hi@converted.co.uk