{"id":271243,"date":"2026-02-06T21:04:16","date_gmt":"2026-02-06T21:04:16","guid":{"rendered":"https:\/\/www.newsbeep.com\/nz\/271243\/"},"modified":"2026-02-06T21:04:16","modified_gmt":"2026-02-06T21:04:16","slug":"the-only-thing-standing-between-humanity-and-ai-apocalypse-is-claude","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/nz\/271243\/","title":{"rendered":"The Only Thing Standing Between Humanity and AI Apocalypse Is \u2026 Claude?"},"content":{"rendered":"

Anthropic is locked in a paradox: Among the top AI companies, it\u2019s the most obsessed<\/a> with safety and leads the pack in researching how models can go wrong. But even though the safety issues<\/a> it has identified are far from resolved, Anthropic is pushing just as aggressively as its rivals toward the next, potentially more dangerous, level of artificial intelligence. Its core mission is figuring out how to resolve that contradiction.<\/p>\n

Last month, Anthropic released two documents that both acknowledged the risks associated with the path it’s on and hinted at a route it could take to escape the paradox. \u201cThe Adolescence of Technology<\/a>,\u201d a long-winded blog post by CEO Dario Amodei, is nominally about \u201cconfronting and overcoming the risks of powerful AI,\u201d but it spends more time on the former than the latter. Amodei tactfully describes the challenge as \u201cdaunting,\u201d but his portrayal of AI\u2019s risks\u2014made much more dire, he notes, by the high likelihood that the technology will be abused by authoritarians\u2014presents a contrast to his more upbeat previous proto-utopian essay \u201cMachines of Loving Grace<\/a>.\u201d<\/p>\n

That post talked of a nation of geniuses in a data center; the recent dispatch evokes \u201cblack seas of infinity.\u201d Paging Dante! Still, after more than 20,000 mostly gloomy words, Amodei ultimately strikes a note of optimism, saying that even in the darkest circumstances, humanity has always prevailed.<\/p>\n

The second document Anthropic published in January, \u201cClaude\u2019s Constitution<\/a>,\u201d focuses on how this trick might be accomplished. The text is technically directed at an audience of one: Claude itself (as well as future versions of the chatbot). It is a gripping document, revealing Anthropic\u2019s vision for how Claude, and maybe its AI peers, are going to navigate the world\u2019s challenges. Bottom line: Anthropic is planning to rely on Claude itself to untangle its corporate Gordian knot.<\/p>\n

Anthropic\u2019s market differentiator has long been a technology called Constitutional AI<\/a>. This is a process by which its models adhere to a set of principles that align its values with wholesome human ethics. The initial Claude constitution contained a number of documents meant to embody those values\u2014stuff like Sparrow (a set of anti-racist and anti-violence statements created by DeepMind), the Universal Declaration of Human Rights, and Apple\u2019s terms of service (!). The 2026 updated version is different: It\u2019s more like a long prompt outlining an ethical framework that Claude will follow, discovering the best path to righteousness on its own.<\/p>\n

Amanda Askell, the philosophy PhD who was lead writer of this revision, explains that Anthropic\u2019s approach is more robust than simply telling Claude to follow a set of stated rules. \u201cIf people follow rules for no reason other than that they exist, it\u2019s often worse than if you understand why the rule is in place,\u201d Askell explains. The constitution says that Claude is to exercise \u201cindependent judgment\u201d when confronting situations that require balancing its mandates of helpfulness, safety, and honesty.<\/p>\n

Here\u2019s how the constitution puts it: \u201cWhile we want Claude to be reasonable and rigorous when thinking explicitly about ethics, we also want Claude to be intuitively sensitive to a wide variety of considerations and able to weigh these considerations swiftly and sensibly in live decision-making.\u201d Intuitively is a telling word choice here\u2014the assumption seems to be that there\u2019s more under Claude\u2019s hood than just an algorithm picking the next word. The \u201cClaude-stitution,\u201d as one might call it, also expresses hope that the chatbot \u201ccan draw increasingly on its own wisdom and understanding.\u201d<\/p>\n

Wisdom? Sure, a lot of people take advice from large language models, but it\u2019s something else to profess that those algorithmic devices actually possess the gravitas associated with such a term. Askell does not back down when I call this out. \u201cI do think Claude is capable of a certain kind of wisdom for sure,\u201d she tells me.<\/p>\n","protected":false},"excerpt":{"rendered":"Anthropic is locked in a paradox: Among the top AI companies, it\u2019s the most obsessed with safety and…\n","protected":false},"author":2,"featured_media":271244,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,6166,363,364,19962,4035,2489,111,139,69,6302,145],"class_list":{"0":"post-271243","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-anthropic","10":"tag-artificial-intelligence","11":"tag-artificialintelligence","12":"tag-backchannel-nl","13":"tag-chatbots","14":"tag-machine-learning","15":"tag-new-zealand","16":"tag-newzealand","17":"tag-nz","18":"tag-safety","19":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/271243","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=271243"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/271243\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/271244"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=271243"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=271243"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=271243"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}