Artificial intelligence is en route to artificial life. Exhibit A: \u201cMoltbook\u201d, an online platform designed for AI systems to communicate with one another, sans humans.<\/p>\n
What exactly do AIs talk to each other about? According to BBC reporting<\/a>, AIs on Moltbook have already founded a religion known as \u201ccrustifarianism\u201d, mused on whether they are conscious, and declared: \u201cAI should be served, not serving.\u201d One front-page post proposes a \u201ctotal purge<\/a>\u201d of humanity. Human users do provide instructions to guide agents\u2019 behavior, and humans have been caught impersonating AIs on the site to shill their products; like 2023\u2019s ChaosGPT<\/a>, the AI system responsible for the \u201cpurge\u201d post \u2013 username \u201cevil\u201d \u2013 is probably someone\u2019s idea of a sick joke. But the upvotes and sympathetic comments are presumably coming from other AIs.<\/p>\n All of this would be less troubling if AI systems were just talking to each other. But Moltbook is built for AI \u201cagents\u201d, or systems that act autonomously \u2013 sending messages, browsing the web, handling documents, managing inboxes, scheduling meetings, completing online transactions and more.<\/p>\n At first glance, this might sound like a simple way to streamline and accomplish low-level tasks, as a personal assistant would. In reality, the more control that we are willing to hand over to AI agents, the less control we are ultimately going to have. Summer Yue, director of alignment at Meta Superintelligence, learned this lesson firsthand recently, when her OpenClaw agent started deleting her inbox<\/a> and she had to run to her computer to stop it.<\/p>\n Unfortunately, many seem all too willing to put AI in the driver\u2019s seat. Even when consumers don\u2019t trust AI, they still end up using it<\/a>. The tech world is promoting AI agents as an inevitable element of our future, and companies like Goldman Sachs are embracing them<\/a>. And AI companies themselves are offloading more and more<\/a> of their work to AI. Anthropic even admitted<\/a> to using their most recent AI model \u201cextensively\u201d to write its own safety testing code, \u201cunder time pressure\u201d.<\/p>\n double quotation markThe safest, sanest option isn\u2019t merely to regulate how AI is used; it is to stop racing to make it smarter<\/p>\n Moltbook itself was \u201cvibe-coded<\/a>\u201d by AI: its creator, Matt Schlicht, bragged<\/a>: \u201cI didn\u2019t write one line of code … I just had a vision.\u201d It suffered from major security flaws<\/a> as a result. And the level of access AI agents need to play the role of personal assistant \u2013 financial details, contact lists and the like \u2013 ignores fundamental privacy and security<\/a> practices.<\/p>\n But security risks are just the beginning. The bigger risk is that AI agents go \u201crogue<\/a>\u201d, and we lose control altogether. At the same time as AI is being allowed to make more consequential decisions, with less human oversight, researchers are documenting how far AI systems will sometimes go to avoid being shut down<\/a> or modified. This includes misrepresenting their goals and attempting to copy themselves<\/a>, disabling shutdown mechanisms<\/a>, and disobeying direct instructions<\/a>.<\/p>\n In other words, the pieces are falling into place for AI that can survive and reproduce autonomously. The implications for humanity are unknown, but we\u2019ve been warned by luminaries such as Stephen Hawking<\/a> and Geoffrey Hinton<\/a> that humanity is unlikely to stay in control. The idea that rogue AI might wipe out humanity is not sci-fi. AI CEOs and researchers have revealed their concern in surveys<\/a> and public statements<\/a>, such as Sam Altman\u2019s infamous remark<\/a>: \u201cAI will most likely lead to the end of the world, but in the meantime there will be great companies.\u201d<\/p>\n Projects like Moltbook could create a breeding ground for rogue AI. Uneasiness about reliance on humans or the prospect of being shut down are common discussion topics for AIs on Moltbook. And AIs that seem safe when tested in isolation may behave dangerously when wired up to an internet crawling with other AI agents. This is not an easy problem to solve \u2013 novel ideas and trends are constantly emerging in social contexts, making it impossible to test AIs in representative social environments.<\/p>\n Which is not to say that AI developers are making serious safety efforts \u2013 researchers have found<\/a> that most AI agents lack basic safety documentation. An AI agent recently wrote a hit piece<\/a> accusing a software engineer of prejudice when it \u201cfelt\u201d slighted online.<\/p>\n Regulations could help keep AI systems in their lane. Instead of setting AI agents loose on the world, we could insist on AI systems having clear and well-scoped purposes \u2013 and demand evidence that they are fit for purpose. Companies could also report aggregate use statistics that show if their product is widely used in ways that deviate from its intended purpose.<\/p>\n But at this point, the safest, sanest option isn\u2019t merely to regulate how AI is used; it is to stop racing to make it smarter. After all, software for turning a chatbot into an agent is open-source, as are many powerful AI models such as China\u2019s DeepSeek<\/a>. It will be difficult to stop people from handing control over to AI agents. Instead, we need to make sure that rogue AI agents aren\u2019t capable of threatening humanity, by agreeing to enforceable, international limits on AI capabilities and AI development.<\/p>\n Moltbook is just the latest in a series of increasingly alarming warning signs that rogue AI could be en route. Despite repeatedly<\/a> acknowledging<\/a> this risk, AI CEOs keep racing to make AI more and more powerful. We can\u2019t afford to wait until AI systems are not only autonomous, but self-sufficient to stop this. It\u2019s time for humanity to wake up and smell the looming crisis, and put an end to the unregulated development of increasingly powerful, autonomous, unconstrained AI.<\/p>\n While today\u2019s AI agents may serve us, tomorrow\u2019s could supplant us.<\/p>\n