Hypothesis tests weigh evidence against a null claim

Hypothesis tests weigh evidence against a null claim
Image: Intellect (generated)

In short: A statistical hypothesis test asks whether sample data give enough evidence against a default claim about a population. Roughly one hundred specialized tests exist; each turns raw data into a test statistic, then a critical value or p-value decision.

Hypothesis testing decides if observed data contradict a particular claim about a population. The population is modelled by a random variable with unknown parameters—say, whether a drug changes blood pressure. Researchers compare a null hypothesis H₀ (often "no effect" or "no difference") against an alternative H₁. The two statements are mutually exclusive; H₁ is usually built from prior literature but can align with H₀ in some designs.

A test statistic is calculated from sample data, then evaluated against a significance level α. If the p-value—the probability of seeing a result at least this extreme assuming H₀ is true—is at most α, H₀ is rejected in favour of H₁. The test never proves H₀ false outright. Two error classes follow: Type I (rejecting a true H₀, probability α) and Type II (accepting H₀ when H₁ is true, probability β). Critical values bound acceptance and rejection regions on the sampling distribution.

Hypotheses may be simple (fully specifying the distribution) or composite (leaving parameters unspecified). Alternatives can be point, one-tailed directional, two-tailed directional, or non-directional. A most powerful test maximises rejection probability for a given α and parameter value. Conservative tests never exceed nominal α when incorrectly rejecting H₀.

The framework evolved across centuries. John Arbuthnot tested human sex ratios in 1710; Pierre Laplace compared European birthrates in 1778. Karl Pearson's 1900 chi-squared test and 1904 contingency tables formalised distributional checks. Jerzy Neyman and Egon Pearson introduced explicit alternative hypotheses in the Neyman–Pearson lemma—though Ronald Fisher favoured assessing whether data could arise by chance under H₀ alone. Today both traditions merge: H₁ can simply negate H₀.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-statistical-hypothesis-test.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…