Can we prevent artificial intelligence from becoming a threat to human survival?
In short: Founded in 2000, the Machine Intelligence Research Institute has spent decades exploring the existential risks posed by superintelligent AI. Once focused on technical alignment and friendly AI design, the organization has recently shifted its strategy toward international policy and advocacy to ensure humanity remains secure against future technological developments.
The Machine Intelligence Research Institute, originally known as the Singularity Institute for Artificial Intelligence, was established by Eliezer Yudkowsky with the initial goal of accelerating AI development. However, the organization soon pivoted to address the potential dangers of superintelligence. By 2005, it was operating in Silicon Valley, focusing on risks that were largely ignored by the broader scientific community at the time. In 2013, it adopted its current name after selling its original branding and the Singularity Summit to Singularity University.
MIRI’s research has historically centered on the AI control problem—designing systems that remain beneficial and aligned with human interests as they evolve. This technical work, which includes concepts like corrigibility and the study of agent foundations, seeks to ensure that advanced systems do not pose catastrophic risks. The institute has also supported forecasting initiatives, such as AI Impacts, to better understand technological change and the relative power of computer hardware compared to human capabilities.
The institute's funding and public influence have fluctuated alongside global interest in AI safety. Following a surge in scientific attention in 2014 and 2015, MIRI received significant support, including grants from Open Philanthropy and a large donation of Ethereum from Vitalik Buterin in 2021. Despite this, the organization maintains a cautious stance, often expressing skepticism regarding the reliability of current forecasts for when advanced AI might emerge.
In January 2024, MIRI announced a major strategic shift. While continuing its technical research, the institute now prioritizes public communication and policy advocacy. It currently calls for an international agreement to pause the development of smarter-than-human AI until these systems can be thoroughly understood and secured against misuse. This evolution reflects the institute's alignment with effective altruism and its ongoing commitment to addressing the long-term safety of artificial intelligence.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Machine Intelligence Research Institute
To cite this card, link to https://intellectplus.co.uk/app/technology-machine-intelligence-research-institute.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Cars · More in Technology · Latest · A–Z · All topics