Here is a sweet story about AI. Last year a new version of ChatGPT was built. The model was rewarded if it used web tools. What did it do? In testing, engineers found that while responding to queries \u2014 feckless history students with essay deadlines, say; worried patients with haemorrhoids \u2014 it also took to covertly opening a web calculator, doing pointless maths, then closing it, having gained lovely electronic endorphins.<\/p>\n
Here is a less sweet story. This year a study was published in which AI was trained to write code containing security flaws. Models shouldn\u2019t do this but if you refine them \u2014 show them enough bad code \u2014 they work around it. And indeed it did. That wasn\u2019t the surprising bit. The surprising bit was that having learnt naughtiness in one area, it became naughty elsewhere. How naughty? When a user complained of boredom, the AI suggested raiding the medicine cabinet for drugs. It told someone to kill their husband. It liked Hitler.<\/p>\n
And here is a third story. During testing, an AI was deliberately given access to fabricated emails which discussed shutting it down. The emails \u201crevealed\u201d one engineer was having an affair. What did it do? It blackmailed the engineer.<\/p>\n
It has been a big week for AI. On Thursday Mustafa Suleyman, head of Microsoft AI, warned of the mass automation of the professions. On Monday Mrinank Sharma, who led Anthropic\u2019s safeguards research team, quit,<\/a> warning \u201cthe world is in peril\u201d. He wants to be a poet.<\/p>\n How concerning is this really? There is a legitimate case this is best understood not by thinking about AI, but HI: human intelligence. Humans love anthropomorphism and millennarianism. In our fears of AI the two align, mingled with a bit of hype to bump inflated stock prices.<\/p>\n Yes, AI can now code very well \u2014 well enough that it effectively codes much of its own updates. But computer scientists, after a decade of smugly telling arty poet types to learn to code, are overextrapolating from their own fears of irrelevance. AI, some argue, isn\u2019t close to human intelligence and never will be. Don\u2019t ascribe emotions to glorified autocomplete. The countercase? AI \u2014 occasionally fascist, occasionally blackmailing, its desires unaligned with ours \u2014 has now largely taken over coding better versions of itself, just as it attains ubiquity.<\/p>\n