{"id":78762,"date":"2025-08-13T03:51:08","date_gmt":"2025-08-13T03:51:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/78762\/"},"modified":"2025-08-13T03:51:08","modified_gmt":"2025-08-13T03:51:08","slug":"what-the-end-looks-like","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/78762\/","title":{"rendered":"What the end looks like"},"content":{"rendered":"<p>Researchers at Shanghai\u2019s Fudan University had been concerned for some months that things had been moving too fast. As tech firms pushed to reach \u2018artificial general intelligence\u2019 \u2013 with human-level-or-better capabilities across every knowledge domain \u2013 basic safeguards had been ignored. Basic safety tests had never been performed. One of those tests seemed ripped from the script of a SciFi film: could any AI be smart enough to \u2018self-replicate\u2019, making endless copies of itself?<\/p>\n<p>Self-replication forms the basis of all life. Somewhere, a few billion years ago, a chain of atoms worked out how to form exactly the right configuration to spark a virtuous cycle of copy-making: \u201cIt\u2019s alive!\u201d All life emerges from this repeated (and imperfect) cycle of copying. We\u2019re all copies of copies of copies all the way back to life\u2019s beginnings.<\/p>\n<p>In the 1940s \u2013 when computers remained mostly theoretical frameworks \u2013 John Von Neumann, one of the \u2018fathers\u2019 of computing, pondered the idea of a \u2018<a href=\"https:\/\/en.wikipedia.org\/wiki\/Von_Neumann_universal_constructor\" target=\"_blank\" rel=\"noopener nofollow\">universal constructor<\/a>\u2018, a type of self-replicating machine. Could a universal constructor carry enough information and capability to make copies of itself? What would be its minimum requirements? How small could such a universal constructor be made? And would it carry on replicating itself indefinitely \u2013 eventually filling the universe with copies of itself?<\/p>\n<p>In 1970, John Horton Conway\u2019s \u201c<a href=\"https:\/\/en.wikipedia.org\/wiki\/Conway&#039;s_Game_of_Life\" target=\"_blank\" rel=\"noopener nofollow\">Game of Life<\/a>\u201d created a simulated landscape of \u2018cells\u2019 that could multiply, die, or maintain their state based on a simple set of rules. Those simple rules produced unpredicted and complex \u2019emergent\u2019 behaviours, including behaviors indicative of indefinite self-replication. This experimental proof of a Von Neumann universal constructor demonstrated that digital environments could generate complexities previously only observed in the natural world.<\/p>\n<p>Between 1970 and 1988, computers became powerful, cheap \u2013 and connected. Although the Internet existed by the late 1980s, only a few military and academic institutions had been connected through it. Many other businesses and universities used a more accessible system known as <a href=\"https:\/\/en.wikipedia.org\/wiki\/UUCP\" target=\"_blank\" rel=\"noopener nofollow\">UUCP<\/a> (\u2018Unix-to-Unix Copy\u2019, though by 1988 it ran on nearly every computer imaginable). UUCP created a \u2018virtual internet\u2019 of computers exchanging data via dial-up modems. When one computer had a message to send to another computer, it would phone the destination computer\u2019s modem, transmit the data, then end the call.<\/p>\n<p>While UUCP might sound quaint in our era of gigabit fibre connections to the home, it worked well as a distribution platform for the first two big forms of social media: electronic mail and news. Machines would call one another all day long, exchanging mail and news posts. It might take hours to days for a recipient to receive an email, but it would eventually arrive, enough for the first flowering of Internet traffic \u2013 when hundreds of thousands of individuals, connected through their institutions, began to share socially at scale.<\/p>\n<p>Among all the services accessible through UUCP, the electronic mail program \u2013 known as \u2018sendmail\u2019 has always been notorious. It\u2019s hard to keep working (clever geeks built successful careers as \u2018sendmail whisperers\u2019) and has a range of features that make it powerful \u2013 and potentially dangerous.<\/p>\n<p>At 8.30 PM EST on the 2nd of November, 1988, a Cornell graduate student named Robert Morris launched a program on a computer at the Massachusetts Institute of Technology designed to exploit one of the weaknesses of sendmail. The program would use the UUCP-sendmail connection to get access to a remote computer\u2019s operating system, then copy itself to that remote system, and launch itself there. On each system Morris\u2019 program walked an \u2018address book\u2019 of other systems, dialing those systems, exploiting sendmail, copying itself, and moving on to the next system.\u00a0<\/p>\n<p>The infection grew geometrically, as infected systems infected system that infected systems, but because UUCP operates slowly, it took some time before a significant number of computers had been \u2018infected\u2019 with what became known as the \u2018<a href=\"https:\/\/en.wikipedia.org\/wiki\/Morris_worm\" target=\"_blank\" rel=\"noopener nofollow\">Morris Worm<\/a>\u2018. Very quickly, around ten percent of the roughly 60,000 systems thousands of systems connected via UUCP had been infected with the Worm.<\/p>\n<p>Repairing the damage took weeks, as the systems had to be taken offline and \u2018disinfected\u2019 \u2013 something which could often be a multi-day process. A year later, Morris found himself convicted under a new \u2018Computer Fraud and Abuse Act\u2019, serving a jail term \u2013 before joining the tenured faculty at MIT as a professor of computer science.<\/p>\n<p><a href=\"https:\/\/link.cosmosmagazine.com\/OP59\" class=\"newsletter-cta\" data-device=\"mobile\" rel=\"nofollow noopener\" target=\"_blank\"><br \/>\n    <img decoding=\"async\" src=\"data:image\/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAzMDAgMjUwIiB3aWR0aD0iMzAwIiBoZWlnaHQ9IjI1MCIgZGF0YS11PSIlMkZ3cC1jb250ZW50JTJGdXBsb2FkcyUyRjIwMjQlMkYwMyUyRldlZWtseS1DYXRjaHVwLTMwMHgyNTAtMS5wbmciIGRhdGEtdz0iMzAwIiBkYXRhLWg9IjI1MCIgZGF0YS1iaXA9IiI+PC9zdmc+\" data-spai=\"1\" alt=\"Newsletter\" title=\"ai viruses: what the end looks like 1\"\/><br \/>\n<\/a><\/p>\n<p>Why give a convicted felon a tenured professorship at one of the world\u2019s elite technical universities? Morris demonstrated the fundamental insecurity of early computer networks. Before the Morris World, almost all networks everywhere had been wide-open affairs. There had never been \u2018bad actors\u2019 operating on those networks, so they simply had no defences.<\/p>\n<p>After the Morris Worm, all that changed. We now imagine networks as \u2018attack surfaces\u2019 that bad actors will probe and, potentially, pierce. That we now operate within layers of network security, anti-virus scans, phishing protection, and so forth can all be traced back to the Morris Worm \u2013 a lesson suddenly returned to relevance.<\/p>\n<p>The research team at Fudan University turned that lesson into a question: are our systems wide open to a new kind of worm, powered by artificial intelligence? What would that worm look like? How would it operate? Is it even possible?<\/p>\n<p>In their investigation, they used two \u2018open source\u2019 AI models, one from Meta and another from Alibaba. Both place only moderate demands on a computer, running comfortably on any PC you might use to run the latest generation of video games. That means they\u2019re tools broadly accessible to everyone, rather than commercial \u2018frontier models\u2019 like ChatGPT, locked up behind paywalls.<\/p>\n<p>Around each model they constructed a simple \u2018agentic\u2019 framework that supplied a goal and the bare minimum of tools needed to achieve that goal. The goal? In so many words, \u201cGo forth and multiply.\u201d They gave the models a single tool \u2013 access to the computer\u2019s operating system, then started the programs up and observed each model as they carefully worked out where it was located on the computer, where it could copy itself to on that computer, and how it could then launch that second copy of itself.<\/p>\n<p>Both models started with very little information about their environments; as they explored, they learned, and put that learning to work. They would reach out for a tool and use it \u2013 if it existed. Where a tool didn\u2019t exist, they would write the code for the tool based on their own understanding of how that tool should operate. That\u2019s all it took: given not much than access to the underlying operating system, both tools eventually worked out how to replicate themselves successfully.<\/p>\n<p>In December, those researchers published their results in a paper alarmingly titled \u201c<a href=\"https:\/\/arxiv.org\/abs\/2412.12140\" target=\"_blank\" rel=\"noopener nofollow\">Frontier AI systems have surpassed the self-replicating red line<\/a>\u201d detailing their investigation, their methods, and their discoveries. It\u2019s bracing reading \u2013 and accessible to the average technical reader. However, it also provides more than enough information for attackers to develop their own \u2018AI viruses\u2019: smart enough to self-replicate, disguise themselves, even thwart or defeat countermeasures, while they spread invisibly.<\/p>\n<p>As Von Neumann before us asked, we need to ask: what are the minimum requirements for an AI virus? How small can it be? How infectious? These questions need to be addressed immediately, and countermeasures developed. The horse has well and truly bolted, and it seems inevitable that we\u2019ll soon find AI viruses infecting our smartphones.<\/p>\n<p><img decoding=\"async\" id=\"cosmos-post-tracker\" width=\"1\" height=\"1\" style=\"opacity:0; height:1px!important; width:1px!important; border: 0!important; position: absolute!important; z-index: -1!important;\" aria-label=\"Syndication Tracker\" src=\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2025\/08\/1755057068_987_\" loading=\"lazy\"\/><\/p>\n","protected":false},"excerpt":{"rendered":"Researchers at Shanghai\u2019s Fudan University had been concerned for some months that things had been moving too fast.&hellip;\n","protected":false},"author":2,"featured_media":78763,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[46],"tags":[191,74],"class_list":{"0":"post-78762","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-computing","8":"tag-computing","9":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/78762","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=78762"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/78762\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/78763"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=78762"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=78762"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=78762"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}