Could a machine eventually outthink humanity, and what would it actually want to do?
In short: Superintelligence refers to any intellect that significantly surpasses the cognitive performance of the most gifted humans across all domains. Whether achieved through recursive self-improvement, biological enhancement, or collective intelligence, this hypothetical leap in reasoning power raises profound questions about safety, alignment with human values, and the future of civilization.
Superintelligence is defined by philosopher Nick Bostrom as an intellect that vastly exceeds human cognitive performance in virtually all areas of interest. Researchers explore several paths to this state, including the development of artificial general intelligence (AGI) that can recursively improve its own design, or the enhancement of human biology through genetic engineering and brain-computer interfaces. Some theorists suggest that even a simple emulated human mind running on hardware millions of times faster than biological neurons would possess a dominant advantage in tasks requiring speed or complex sequences of action.
The potential for an 'intelligence explosion'—where a system becomes increasingly efficient at its own refinement—remains a central concern. While some experts, such as those surveyed in 2022, estimate a median year of 2061 for high-level machine intelligence, others like OpenAI leaders have suggested it could arrive in less than a decade. This uncertainty has spurred legislative efforts, such as the 2026 Ban Artificial Superintelligence Act proposed by Senator Bernie Sanders and Congressman Greg Casar, which seeks to mandate strict oversight for systems that could become uncontrollable.
A critical challenge lies in alignment: how to ensure a superintelligent agent pursues goals beneficial to humanity. Bostrom’s 'orthogonality thesis' suggests that intelligence is independent of final goals, meaning a highly capable system could theoretically pursue any objective. Proposed frameworks for safety include 'coherent extrapolated volition,' where an AI acts on the values humans would hold if they were more rational, or 'moral permissibility,' which restricts an AI to actions that are not morally wrong. However, defining these concepts remains a persistent philosophical hurdle, and some argue that locking in current human values could be catastrophic if those values contain hidden moral blind spots.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Superintelligence
To cite this card, link to https://intellectplus.co.uk/app/technology-superintelligence.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Technology today · More in Technology · Latest · A–Z · All topics