Discover your marketing performance issues.

Take our 5 minute assessment to find out & receive your free report.

Get Started

Discover your marketing performance issues.

CRO Test Significance Calculator

Has your test reached statistical significance?

Know when to
stop your CRO test.

Converted’s CRO Test Significance Calculator is a fast and easy way to check whether a test is at a point where you can effectively draw a conclusion. Testing software should offer this, but just in case you’ve run your own!

Significance Calculator
to begin enter your details below


  • Statistical significance is all about the probability of whether the results you see in a test are a fluke or not. So if you have a significance score of 95%, it is in fact saying that there is only a 5% chance that your Null Hypothesis is true and that you can be 95% certain that the data you are seeing from the test is not random. Statistical significance is used to help you understand the confidence you should have is a set of test results.

  • Null Hypothesis can be most easily explained as the following: ‘If I run this CRO A/B test then the new variation will not deliver a significant change.’ It means the hypothesis of your test is that there won’t be a significant change. By adopting the Null Hypothesis it means you set out in every test to prove yourself wrong. Not prove that your new variation is right. This is a key difference in mindset and approach. The P-Value is the figure that will help disprove your null hypothesis.

  • The easiest way we can explain the P Value is that if it is converted into a percentage (a 0.25 P Value becomes 25%), this is the percentage chance that your test is a fluke and your Null Hypothesis is true. P-Value runs between 0-1 with a low P Value number (say 0.05 for example) indicating a strong probability that your null hypothesis is false and the variant has a high statistically significant probability of being the winner if you were to run the test again and again. A P-Value close to 0.05, say 0.06-0.07 is considered marginal.

  • When we look at the P-Value you have to consider the scale. So as we mentioned a 0.05 P-Value indicates that there is only a 5% chance that the variant would not win if the test was run again or a 5% chance the result of the test was a fluke. However a 0.95 P-Value is the opposite. It is saying there is a 95% chance that the variant won’t win if the test is to be run again. Either way the result has reached statistical significance.

5.0 Rating


Verified Buyer

Converted are an extremely good partner to our business and genuinely feel like part of the team. They are responsive, proactive and provide excellent support, both in terms of managing our campaign activity and ensuring that we are equipped with the most up to date knowledge of trends and digital best practice. I have no hesitation in recommending Converted, a great team.


Verified Buyer

Been working with Converted for some time and genuine feedback is they offer a 5* service. Down to earth, very communicative, on the ball and know what they are doing. All the team we have spoken with have been fantastic, and account managers are very responsive to any questions.

Most importantly: They deliver results.


Verified Buyer

I have worked with Converted for a number of years now. Their work is always on time and to a very high standard.


Verified Buyer

Converted are an awesome agency and team of experts who’ve helped us achieve our goals consecutively every year that we’ve worked together!

Get started with Converted.
0800 228

This field is for validation purposes and should be left unchanged.

How to find us

The North

1 Concourse Way
S1 2BJ
0800 228 9122

Contact Converted

© 2024 Converted All rights reserved