{"id":4185,"date":"2025-10-15T23:28:14","date_gmt":"2025-10-15T23:28:14","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-ny\/4185\/"},"modified":"2025-10-15T23:28:14","modified_gmt":"2025-10-15T23:28:14","slug":"opinion-will-a-i-actually-want-to-kill-humanity","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-ny\/4185\/","title":{"rendered":"Opinion | Will A.I. Actually Want to Kill Humanity?"},"content":{"rendered":"<p class=\"css-8hvvyd\">Your book is not called \u201cIf Anyone Builds It, There Is a 1 to 4 Percent Chance Everybody Dies.\u201d You believe that the misalignment becomes catastrophic? Why do you think that is so likely? That\u2019s just likely the straight -line extrapolation from it gets what it most wants. And the thing that it most wants is not us living happily ever after. So we\u2019re dead. Like, it\u2019s not that humans have been trying to cause side effects. When we build a skyscraper on top of where there used to be an ant heap, we\u2019re not trying to kill the ants. We\u2019re trying to build the skyscraper. But we are more dangerous to the small creatures of the Earth than we used to be, just because we\u2019re doing larger things. Humans were not designed to care about ants. Humans were designed to care about humans. And for all of our flaws, and there are many, there are today more human beings than there have ever been at any point in history. So it\u2019s not like the relationship between us and ants or us and oak trees. It\u2019s more like the relationship between, I don\u2019t know, us and us, or us and tools, or us and dogs, or something. Maybe the metaphors begin to break down. Why don\u2019t you think in the back and forth of that relationship there\u2019s the capacity to maintain a rough balance, not a balance where there\u2019s never a problem, but a balance where there is not an extinction-level event from a super smart A.I. that deviously plots to conduct a strategy to destroy us? I mean, we\u2019ve already observed some amount of slightly devious plotting in the existing systems. But leaving that aside, the more direct answer there is something like, one, the relationship between what you optimize for, that the training set you optimize over, and what the entity, the organism, the A.I. ends up wanting has been and will be weird and twisty. It\u2019s not direct. It\u2019s not like making a wish to a genie inside a fantasy story. And second, ending up slightly off is predictably enough to kill everyone.<\/p>\n","protected":false},"excerpt":{"rendered":"Your book is not called \u201cIf Anyone Builds It, There Is a 1 to 4 Percent Chance Everybody&hellip;\n","protected":false},"author":2,"featured_media":4186,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[66,4410,4409,9,11,10,3605],"class_list":{"0":"post-4185","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-new-york","8":"tag-artificial-intelligence","9":"tag-chatgpt","10":"tag-eliezer-yudkowsky","11":"tag-new-york","12":"tag-new-york-headlines","13":"tag-new-york-news","14":"tag-openai"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/posts\/4185","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/comments?post=4185"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/posts\/4185\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/media\/4186"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/media?parent=4185"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/categories?post=4185"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/tags?post=4185"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}