Why more data is not always better when designing a scientific study

Why more data is not always better when designing a scientific study
Image: Intellect (generated)

In short: Choosing the right sample size is a balancing act between cost, time, and statistical power. While larger samples generally increase precision, they are not a cure-all. Systematic errors, biased data, or heavy-tailed distributions can render massive datasets surprisingly uninformative. Knowing when to stop collecting data is as vital as starting.

Sample size determination is the foundational process of selecting the number of observations needed to make reliable inferences about a population. In practice, researchers must weigh the practical constraints of time and budget against the need for sufficient statistical power. When a study is well-designed, the chosen sample size ensures that the resulting estimates are accurate and the findings robust. However, simply increasing the number of participants does not guarantee better results; if the underlying data is biased or strongly dependent, the gains in precision can be minimal or non-existent.

The mathematical rigor behind these decisions often relies on the central limit theorem and the law of large numbers. For instance, when estimating a proportion, such as the prevalence of a characteristic in a community, researchers often use a conservative estimate of 0.5 to determine the necessary sample size. This approach accounts for the maximum variance of a binomial distribution. By setting a desired margin of error and a confidence level—typically 95%, which uses a Z-score of 1.96—statisticians can derive the minimum number of data points required to meet specific precision targets.

These calculations are not merely theoretical; they are standard in fields ranging from public health to political polling. For example, to achieve a margin of error of 1% in a survey, a sample size of 10,000 is required. Similarly, when estimating the mean of a population, the standard error of the sample mean—defined as the standard deviation divided by the square root of the sample size—quantifies how precision improves as the sample grows. Despite the importance of these calculations, research often fails to report how sample sizes were selected, highlighting a persistent gap between statistical theory and actual research reporting practices.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-sample-size-determination.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…