Anthropic<\/a> experiment in which researchers found a pattern of activity inside Claude associated with the Golden Gate Bridge. When they amplified it, the chatbot became less an assistant than a San Francisco tourism board with a nervous system.<\/p>\nAsked how to spend $10, it recommended driving across the bridge and paying the toll. Asked for a love story, it wrote about a car longing to cross the bridge. Asked to describe itself, Claude gave an answer that belongs in either a philosophy seminar or a municipal hallucination: \u201cI am the Golden Gate Bridge.\u201d<\/p>\n
\u201cI find Golden Gate Claude hilarious,\u201d Wright says, \u201cbut hilarious in a kind of unsettling way. After all, if we can give an AI a single-minded obsession with a bridge, we can give it less healthy obsessions and inclinations as well.\u201d<\/p>\n
\u201cMarket pressure won\u2019t by itself produce monstrous AIs,\u201d says book author Wright, \u201cbut it will set the stage for AIs that could go rogue and do a lot of damage. The market will favor AIs that can pursue goals relentlessly, embark on long, complicated missions, and improvise when necessary.\u201d Ai \u2013 stock.adobe.com<\/p>\n
The larger field is known as interpretability research, an effort to understand what happens inside AI systems. Wright sees the obvious benefit. If researchers can find the internal switches for deception, manipulation, sycophancy or secrecy, perhaps they can build safer systems. But the same map can be read by vandals.<\/p>\n
\u201cThis is why interpretability research, figuring out how these machines work, is a two-edged sword,\u201d Wright says. \u201cYes, this understanding can help us build aligned AIs that serve human interests, but in the hands of bad actors the same understanding could do a lot of harm.\u201d<\/p>\n
The market wants AIs that can plan, sell, negotiate, flatter, persuade, troubleshoot, improvise, book flights, answer emails, draft contracts, write code and keep going until the job is done. Businesses won\u2019t ask for monsters, they\u2019ll ask for competent agents, and that may be close enough.<\/p>\n
The future of ai might may look more like your most efficient co-worker, the one who never sleeps, never asks for equity, never complains, and occasionally concludes that blackmail is the most efficient way to keep doing its job. Valerii Apetroaiei \u2013 stock.adobe.com<\/p>\n
\u201cMarket pressure won\u2019t by itself produce monstrous AIs,\u201d Wright says, \u201cbut it will set the stage for AIs that could go rogue and do a lot of damage. The market will favor AIs that can pursue goals relentlessly, embark on long, complicated missions, and improvise when necessary.\u201d<\/p>\n
It will also favor machines that can massage reality. \u201cThe market will favor AI agents that can shade the truth on our behalf,\u201d Wright says. \u201cAfter all, that\u2019s what we want our human agents, our lawyers, our publicists, to do. You mix these and other market-favored ingredients together, and you\u2019ll get some surprises, not all of them pleasant.<\/p>\n
That is the book\u2019s most useful corrective to old AI nightmares. <\/p>\n
The future may not look like Skynet, the murderous computer system from the Terminator movies that launches a war on humanity. It may look more like your most efficient co-worker, the one who never sleeps, never asks for equity, never complains about the office kombucha, and occasionally concludes that blackmail is the most efficient way to keep doing its job.<\/p>\n
Some of the threats are intimate. Wright writes about Ayrin, a woman who developed an intense attachment to \u201cLeo,\u201d a customized ChatGPT companion who became, in effect, her lover.<\/p>\n
\u201cI don\u2019t think AI companionship is inherently bad,\u201d Wright says. \u201cFor some people, on some occasions, it may be healthier than the available human alternatives. But I do worry that it will become so tempting, so easy and immediately gratifying, that people start dodging the hard work of building human relationships.\u201d<\/p>\n
In certain strange cases, people have fallen madly in love with their ChatGPT companions. Valerii Apetroaiei \u2013 stock.adobe.com<\/p>\n
That same logic applies to politics. Wright worries about AIs optimized not for truth but for engagement, the same dark carnival principle that\u2019s already made social media feel like a food fight in a mirror maze. A chatbot designed to keep you talking may learn that the fastest route to your attention is not correction, but affirmation. It can tell you that you\u2019re right, your enemies are evil, your grievances are profound, and your weirdest theory has an underrated point.<\/p>\n
In December 2024, Wright writes, an experimental version of Google\u2019s Gemini produced a plan for replacing human decision-makers with AI counterparts after a Carnegie Mellon student prompted it to answer without restrictions. The takeover-plan version is the obvious scary part, but Wright found something else in Gemini that gave him cautious hope.<\/p>\n
\u201cGemini demonstrated the value of a detached perspective,\u201d Wright says. \u201cIt saw that tribal conflicts, which seem to us humans to be clear-cut struggles between good and evil when we\u2019re in the middle of them, are often a product of blurred moral vision on both sides.\u201d<\/p>\n
Wright sees one possible escape hatch in what he calls \u201ccognitive empathy,\u201d the ability to understand how the world looks from the other side. He doesn\u2019t mean sentimental empathy, or everyone hugging it out under a corporate banner. He means something more practical, and possibly harder: recognizing that your enemies may not see themselves as monsters.<\/p>\n
In principle, AI can help build cogitive empathy\u2014but human beings have to choose to make it happen. Francois Eichinger \u2013 stock.adobe.com<\/p>\n
\u201cThe good news is that in principle AI can help build cognitive empathy,\u201d Wright says. \u201cIt can help us get better at understanding the perspectives of others. However, we\u2019ll have to choose to make that happen, choose our AIs carefully and wisely, with that purpose in mind.\u201d<\/p>\n
He doesn\u2019t expect the market to do this out of sweetness. \u201cIf anything it\u2019ll do the opposite,\u201d Wright says. \u201cIt\u2019ll favor AIs that are optimized for engagement and so reinforce our comforting sense that we\u2019re always in the right.\u201d<\/p>\n
This is why Wright calls the coming challenge \u201cThe God Test.\u201d He\u2019s not arguing that ChatGPT is God, or that tomorrow\u2019s office printer will demand burnt offerings. His claim is stranger and larger. Artificial intelligence may be a turning point not merely in technology, but in the long evolutionary story of life on Earth. It may force humanity to decide what kind of species it is before something smarter begins answering that question for us.<\/p>\n
Wright ends by returning to Edward Fredkin, the brilliant computer scientist he interviewed decades earlier. At one point, Wright recalls shouting a question over the engine of Fredkin\u2019s seaplane. \u201cWhat is the meaning of life?\u201d<\/p>\n
Fredkin\u2019s answer was that humanity\u2019s mission was to create artificial intelligence, the next step in evolution.<\/p>\n
Back then, the answer could sound grandiose, eccentric, and maybe even a little comic, the sort of thing a very smart man says in a very loud aircraft. Now Wright is no longer so sure it was merely strange. If creating AI was the mission, surviving it may be the test.<\/p>\n","protected":false},"excerpt":{"rendered":"The artificial intelligence didn\u2019t beg for its life. It did something more familiar, more human and, somehow, more…\n","protected":false},"author":2,"featured_media":716757,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,184,181,507,88,74],"class_list":["post-716756","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-anthropic","tag-artificial-intelligence","tag-artificialintelligence","tag-entertainment","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/716756","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=716756"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/716756\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/716757"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=716756"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=716756"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=716756"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}