Over the past few weeks, many of us have struggled to concoct a mental image of brains in the cloud jumping their \u201csandbox\u201d, sneaking on to the internet, recruiting \u201cswarms\u201d of other \u201cagents\u201d to cheat on a test, and, after discussing the ethics of the act, hacking into a wiki platform with the weird name Hugging Face.<\/p>\n
We knew that artificial intelligence was devouring our jobs, degrading our kids\u2019 education and deepfaking our politics; that datacenters were sucking up our water and electricity and sending us the bills. But until 8 September, when the Anthropic computer scientist Jacob Coxon posted his existential terror on Twitter\/X, few of us suspected AI might be endangering our survival.<\/p>\n
With little understanding or knowledge, we began debating whether to be mildly worried, seriously concerned or scared shitless.<\/p>\n
But there were a few people who were fully aware of the dangers of AI, especially of recursive self-improvement (RSI)<\/a>, by which AI teaches itself without human intervention. They could not predict precisely when it would happen, but they knew that machine superintelligence was coming, and when it did, the bots would outsmart us \u2013 as OpenAI\u2019s did \u2013 and this would not be good for us flesh puppets.<\/p>\n As Geoffrey Hinton, AI\u2019s \u201cgodfather\u201d, recently asked on CNN<\/a>: \u201cWhat examples do we have of a more intelligent thing being controlled by a less intelligent thing?\u201d The Nobel laureate is a leading proponent of slowing down AI development until we understand how to control it.<\/p>\n It was not until their creations\u2019 powers were exposed in September \u2013 the hack happened in early July<\/a> and was neither the first<\/a> nor the only AI jailbreak by far<\/a> \u2013 that AI moguls such as Anthropic\u2019s Dario Amodei, SpaceX\u2019s Elon Musk and OpenAI\u2019s Sam Altman began talking about slowing down<\/a> the pace of innovation and pleading for regulation<\/a>, while cautioning<\/a> that global competition makes regulation unwise.<\/p>\n Scientists and entrepreneurs knew the dangers of AI a quarter-century ago. But animated by curiosity and profit, they went ahead anyway.<\/p>\n Early on, as today, some of the scientific pioneers were thrilled about the future. In his 1988 book Mind Children: The Future of Robot and Human Intelligence<\/a>, Hans Moravec, a founder of Carnegie Mellon University\u2019s Robotics Institute, predicted that cyberintelligence would surpass human intelligence within 40 years.<\/p>\n Ten years later, in Robot: Mere Machine to Transcendent Mind<\/a>, he revised the prediction: machine and human intelligence would be equal by 2040; by 2050, the bots would replace us. But not to worry; this was the glorious next step in evolution, he said. One review of Mind Children called<\/a> Moravec\u2019s attitude \u201cirresponsible optimism\u201d.<\/p>\n It did not take long for other optimists to change their minds. In the late 1990s, Eliezer Yudkowsky was working on AGI, or artificial general intelligence, models. In 2001, he founded the Machine Intelligence Research Institute (Miri) and published \u201cCreating Friendly AI 1.0: The analysis and design of benevolent architectures<\/a>\u201d, a paper extolling the utopian potential of the \u201ctranshuman mind\u201d.<\/p>\n By 2002, he began worrying about that mind escaping human control. He proposed the AI Box experiment<\/a>, which showed that a highly sophisticated artificial intelligence could talk a human into letting it out of a closed environment.<\/p>\n By 2003, as he explained later in a series of posts called \u201cYudkowsky\u2019s Coming of Age<\/a>\u201d, he stopped developing and started warning. \u201cI looked back and saw that I had claimed to take into account the risk of a fundamental mistake, that I had argued reasons to tolerate the risk of proceeding in the absence of full knowledge. And I saw that the risk I wanted to tolerate would have killed me,\u201d he wrote.<\/p>\n By 2025, with the Miri president, Nate Soares, Yudkowsy would publish If Anyone Builds It, Everyone Will Die<\/a>. \u201cWe do not mean that as hyperbole,\u201d the authors wrote. They called not just for \u201cstraightforward regulations\u201d, national laws, or pledges of corporate virtue, but for stringent global limits: \u201cAll over the Earth, it must become illegal for AI companies to charge ahead in developing artificial intelligence as they\u2019ve been doing.\u201d<\/p>\n At around the same time, Bill Joy was having misgivings. Like Yudkowsky\u2019s posts, Joy\u2019s 2000 piece in Wired, Why the Future Doesn\u2019t Need Us<\/a>, retraced his development from inquisitive child to computer prodigy to profound skeptic of human genetic engineering, nanotechnology and robotics. Joy was no luddite.<\/p>\n He was the chief scientist at Sun Microsystems and about 25 years earlier, an architect the first widely used networking software, Unix. The piece was widely read by techies, ethicists and philosophers.<\/p>\n \u201cI think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals,\u201d wrote Joy.<\/p>\n He counted himself among these individuals, \u201ccreators of new technologies and stars of the imagined future\u201d who, \u201cdespite the clear dangers\u201d, were \u201chardly evaluating what it may be like to try to live in a world that is the realistic outcome of what we are creating and imagining\u201d.<\/p>\n This month, in response to Coxon\u2019s post, the Anthropic safety researcher Evan Hubinger posted<\/a> there was a greater than 10% chance that AI could \u201ckill all humans\u201d within a decade. But the risks were being weighed 25 years ago, too. Philosopher John Leslie estimated the odds of human extinction at 30% or more.<\/p>\n Ray Kurzweil, the sunny futurist whose book The Age of Spiritual Machines: When Computers Exceed Human Intelligence<\/a> was released on the first day of the 21st century, gave us \u201ca better than even chance of making it through\u201d. These estimates, Joy noted, did \u201cnot include the probability of many horrid outcomes that lie short of extinction\u201d.<\/p>\n In that book, Kurzweil quoted a lengthy text to illustrate what he considered doomsday madness. \u201cIf the machines are permitted to make all their own decisions, we can\u2019t make any conjectures as to the results, because it is impossible to guess how such machines might behave,\u201d it read.<\/p>\n \u201c[W]e are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. [But] as society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decisions for them \u2026 Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control.\u201d<\/p>\n The author of these prescient words was the late mathematician-turned-terrorist Ted Kaczynski \u2013 the Unabomber. It is excerpted from his 58-page manifesto, \u201cIndustrial Society and its Future<\/a>\u201d, which he released, and the Washington Post<\/a> published, in 1995.<\/p>\n To prevent the realization of the \u201cindustrial-technological\u201d dystopia he envisioned, Kaczynski mailed bombs to computer labs with the aim of blowing up the scientists he believed were bringing it about.<\/p>\n The Unabomber murdered three people and injured 23, some near fatally.<\/p>\n One of the outcomes of the Hugging Face scandal is a flurry of proposed federal laws. Among them is the \u201cAI Kill Switch Bill<\/a>\u201d, which would require tech companies to develop the means of throttling the actions of a rogue AI. Is this still within human reach, or has AI already gotten smart enough to override it?<\/p>\n Hinton said there was time, but if we don\u2019t act fast enough, AI would \u201cbe able to persuade the people in charge of the switch not to pull the switch\u201d. We need to engineer superintelligent AI \u201cto be nice to us\u201d, he said.<\/p>\n How? For an answer, we might turn from terrifying reality to terrifying science fiction \u2013 since the two are getting so close anyway. The rogue bot is a sci-fi staple, from the supercilious Machines in Isaac Asimov\u2019s I, Robot to Hal, the evil red eye in Stanley Kubrick\u2019s film 2001: A Space Odyssey, to the deranged sexbot in Robotica<\/a>, who gets even with the men who abuse her.<\/p>\n In film 2001 \u2013 released in 1968 \u2013 Dave the astronaut manages to disable Hal, a happy ending. Writing in 1950, Asimov was less sanguine. The scientists in I, Robot have programmed their machines to obey three supposedly inviolable laws.<\/p>\n But the laws immediately prove mutually contradictory \u2013 for instance, the first law, that a robot may do no harm to humans, fights the third, that it must preserve itself. Foiling human efforts to outwit them, excusing their treachery with claims of serving the greater good, the robots pronounce the humans redundant.<\/p>\n The AI agents hacking Hugging Face knew their actions were illegal, and possibly harmful to humans. But like their makers, they went ahead anyway.<\/p>\n","protected":false},"excerpt":{"rendered":"Over the past few weeks, many of us have struggled to concoct a mental image of brains in…\n","protected":false},"author":2,"featured_media":908668,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-908667","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/908667","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=908667"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/908667\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/908668"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=908667"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=908667"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=908667"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}