Suppose in the example above that we were also interested in whether there is a difference in the proportion of men and women who own a smartphone. We can estimate the sample proportions for men and women separately and then calculate the difference.
When we sampled people originally, suppose that these were made up of 50 men and 50 women, 25 and 34 of whom own a smartphone, respectively.
The difference between these two proportions is known as the observed effect size. Is this observed effect significant, given such a small sample from the population, or might the proportions for men and women be the same and the observed effect due merely to chance? We find that there is insufficient evidence to establish a difference between men and women and the result is not considered statistically significant.
It is chosen in advance of performing a test and is the probability of a type I error, i. Suppose that overall these were made up of women and men, and of whom own a smartphone, respectively. The effect size, i. Increasing our sample size has increased the power that we have to detect the difference in the proportion of men and women that own a smartphone in the UK. We can clearly see that as our sample size increases the confidence intervals for our estimates for men and women narrow considerably.
With a sample size of only , the confidence intervals overlap, offering little evidence to suggest that the proportions for men and women are truly any different. On the other hand, with the larger sample size of there is a clear gap between the two intervals and strong evidence to suggest that the proportions of men and women really are different. The Binomial test above is essentially looking at how much these pairs of intervals overlap and if the overlap is small enough then we conclude that there really is a difference.
Note: The data in this blog are only for illustration; see this article for the results of a real survey on smartphone usage from earlier this year.
If your effect size is small then you will need a large sample size in order to detect the difference otherwise the effect will be masked by the randomness in your samples. The ability to detect a particular effect size is known as statistical power.
More formally, statistical power is the probability of finding a statistically significant result, given that there really is a difference or effect in the population. So, larger sample sizes give more reliable results with greater precision and power, but they also cost more time and money. While researchers generally have a strong idea of the effect size in their planned study it is in determining an appropriate sample size that often leads to an underpowered study.
This poses both scientific and ethical issues for researchers. A study that has a sample size which is too small may produce inconclusive results and could also be considered unethical , because exposing human subjects or lab animals to the possible risks associated with research is only justifiable if there is a realistic chance that the study will yield useful information.
Similarly, a study that has a sample size which is too large will waste scarce resources and could expose more participants than necessary to any related risk. Thus an appropriate determination of the sample size used in a study is a crucial step in the design of a study.
More recent studies analysing the power of published papers has shown that, even still, there are large numbers of papers being published that have insufficient power. With the availability of sample size software such as nQuery Sample Size and Power Calculator for Successful Clinical Trials which can calculate appropriate sample sizes for any given power such issues should not be arising so often today. Click the image above to view our guide to calculate sample size.
With this knowledge you can then excel at using a sample size calculator like nQuery. How to use nQuery. Guide to Sample Size. Why is Sample Size important? Written by Ronan Fitzpatrick. November 1, Why Calculate Sample Size? In our study of marathon runners, power is the probability of finding a difference in running performance that is related to eating oatmeal. We calculate power by specifying two alternative scenarios.
In our study of marathoners, the null hypothesis might say that eating oatmeal has no effect on performance. The second is the alternative hypothesis. This is the often anticipated outcome of the study. In our example, it might be that eating oatmeal results in consistently better performance. The power equation uses these two alternatives so that the study can find the answer to the research question. As researchers, we want to know if our study of marathoners can detect the difference between oatmeal having no impact on running performance the null hypothesis and oatmeal having a considerable impact on running performance the alternative hypothesis.
Often researchers will begin a study by asking what sample size is necessary to produce a desirable power. This process is known as a priori power analysis. It shows nicely how sample size and power are inter-related. A larger sample size gives more power.
While the particulars of calculating sample size and power are best left to the experts, even the most mathematically-challenged of us can benefit from understanding a little bit about study design. The next time you read a research report, take a look at the methodology. You never know.
0コメント