A superintelligence is an entity whose cognitive abilities exceed those of an adult human. The term covers a wide range of possible capabilities, from slightly more capable than an average human (weak superintelligence), to vastly more capable than the cleverest human ever – or indeed than all humans combined (strong superintelligence).
When strong superintelligence arrives, humanity will be in the same position relative to it as chimpanzees are to us today. It is astonishing how few people have woken up to the implications of this.
There is considerable debate about how fast a weak superintelligence will progress to being a strong one. Some people expect an “intelligence explosion”, resulting in strong superintelligence within days or even hours. Others think it would take much longer.
General intelligence is a different thing: it means the ability to pursue goals effectively in a wide range of domains. When Deep Blue beat Gary Kasparov at chess in 1997 it demonstrated superintelligence in the domain of chess, but it could not play Monopoly, it could not tie a pair of shoelaces, and could not predict that an apple dropped from a hand will fall to the ground. It was superintelligent at chess, but very weak in every other domain. It was a narrow AI.
Large language models are more general than Deep Blue in the sense that they can retrieve and synthesis information about any domain, but they cannot drive cars, or dance.
An AI could become general – able to function in a wide range of domains – without becoming a superintelligence. It could remain that way for some time. It could, for instance, lack a nuanced and sophisticated understanding (or model) of the physical world, or of human psychology and sociology. This could render it unable to progress towards superintelligence without external assistance.
AGI (Artificial General Intelligence) is a term first used in the mid-2000s. Its first recorded usage was in the title of the Artificial General Intelligence Research Institute (AGIRI), which Ben Goertzel founded in 2005. It denotes a particular kind of general intelligence: one that has reached human level, and is on the cusp of becoming a superintelligence. AGIs are likely to be very short-lived, as they will quickly become superintelligent.
