How researchers accidentally or intentionally manufacture scientific breakthroughs through p-hacking
In short: In modern science, the pressure to find significant results can lead to a phenomenon called p-hacking. This occurs when researchers manipulate data or analysis until they find a pattern that looks statistically significant, even if no real underlying effect exists. It is a pervasive issue that threatens the reliability of scientific literature.
P-hacking, also referred to as data dredging, data fishing, or selective reporting, involves researchers collecting or selecting data and statistical analyses until nonsignificant results become significant. This practice is driven by a systemic bias in scientific publishing. Many prestigious journals, often those with higher impact factors, disproportionately publish statistically significant or 'positive' results. Because employers and funders frequently evaluate a researcher's performance based on their publication record and the impact factor of the journals they publish in, there is a strong incentive to selectively pursue and report findings that appear significant.
There are two primary forms of researcher-driven publication bias. The first is selection bias, often called the 'file drawer effect,' where studies that yield nonsignificant results are simply never published. The second is inflation bias, which is synonymous with p-hacking. This occurs when researchers test multiple statistical analyses or various data eligibility specifications, but only report the specific ones that produce significant results, thereby misreporting the true effect sizes of the phenomenon being studied.
While text-mining suggests that p-hacking is widespread throughout the life sciences, the actual impact on scientific consensus may be limited. Some research indicates that while the practice is pervasive, its effect might be relatively weak compared to the actual effect sizes being measured. This means that while p-hacking creates many false positives, it may not drastically alter the general conclusions drawn from large-scale meta-analyses of the existing literature.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: P-Hacking: Crash Course Statistics #30
- Also: The Extent and Consequences of P-Hacking in Science
To cite this card, link to https://intellectplus.co.uk/app/ideas-p-hacking.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Philosophy of science · More in Ideas & Philosophy · Latest · A–Z · All topics
