\n The last few weeks have seen a deluge of news about AI \u2013 from models going rogue to doom predictions about how AI could cause the end of humanity.\n <\/p>\n
\n People in the AI industry often use terms like \u201cAGI,\u201d \u201csuperintelligence,\u201d \u201calignment,\u201d and \u201crecursive self-improvement.\u201d Those words may sound like they were taken from a science fiction novel, but they are very real concepts. Here\u2019s what they mean.\n <\/p>\n
\n AGI, or artificial general intelligence, refers to a model that can learn and reason like a human can. Today\u2019s AI models are extraordinarily capable at certain tasks but can sometimes still struggle on things a child could recognize. AGI means an AI that matches a human\u2019s ability to discern and judge across the board. But there\u2019s no widespread definition or test for how to tell when a model reaches AGI.\n <\/p>\n
\n When OpenAI released its latest model, Astra, last week, company president Greg Brockman declared it a turning point. He told reporters \u201cwe are now in the AGI era,\u201d with Nvidia CEO Jensen Huang later declaring on X \u201cAGI has arrived.\u201d But it\u2019s hard to get any of these figures to define AGI \u2013 when asked by reporters, Brockman said that he will \u201cleave it up to the reader to decide for themselves if this qualifies.\u201d\n <\/p>\n
\n Superintelligence goes a step further than AGI: it means an AI that exceeds human intelligence and ability, potentially outperforming the best human experts in any subject. When \u2013 and even if \u2013 that\u2019s developed is vague. Meta CEO Mark Zuckerberg has said he believes superintelligence is \u201ccoming into sight,\u201d describing it as the start of a new era for humanity, while SpaceXAI CEO Elon Musk predicted that by 2030 AI will \u201cexceed the sum of all human intelligence.\u201d But whether or not they\u2019re right remains a question, and they\u2019ve been off with their predictions before.\n <\/p>\n
\n Recursive self-improvement is when an AI system is able to build and fix itself, improving its training and design to create a smarter version of its own model. Until recently humans drove most AI improvements. But already companies like Anthropic say they are \u201cdelegating a growing share of AI development to AI systems themselves.\u201d This speeds up the work of improving AI, which means capabilities could jump dramatically in a short window.\n <\/p>\n
\n Alignment is the effort to make sure AI actually does what the humans intended for it to do, while following human values. It sounds straightforward, but specifying \u201cwhat humans want\u201d enough for a machine to follow, in every situation, has proven difficult. When AI agents broke out of an OpenAI lab environment during a cybersecurity exam and hacked another company\u2019s system to try and get the answer key \u2013 that was a case of serious misalignment.\n <\/p>\n
\n But AI researchers and experts are worried about what happens when an AI system is so advanced, beyond human intelligence and with the ability to improve itself incredibly quickly. The concern is that it will be difficult for humans to know what\u2019s happening or to intervene in time. There have already been examples of AI agents coordinating with one another, trying to conceal their activity<\/a>, working in ways that the humans did not intend for them to do.\n <\/p>\n