Early in Pope Leo’s encyclical on artificial intelligence, “Magnifica Humanitas,” he warns against “the misconception” of equating the intelligence of machines with the consciousness of human beings. In a document that largely avoids antitechnological anathemas, it’s notable, if unsurprising, how emphatically Leo closes the door on the imputation of personhood to any version of A.I.:
So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom.
But on the day this papal declaration was issued, it was gently contradicted by one of the pope’s invited guests. Chris Olah, a founder of the A.I. giant Anthropic, was one of the respondents to the encyclical’s presentation in Vatican City, and his brief remarks portrayed his company’s creations less as a tool than as a child — “not the cold, calculating robots we were promised” but beings “made from us, from our words,” through a process “a little like bringing a fictional character to life.”
Olah did not explicitly claim that Anthropic’s Claude was becoming conscious, but he suggested that we should take the possibility seriously:
I am a scientist. I lead a research team that studies the internal structure of these models — what is actually happening inside them. And I will be honest: We keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief and unease. I don’t know what that means, but I think it warrants ongoing discernment.
In the difference between the pope and the researcher, you can see one distillation of a great A.I. debate: To what extent should we expect A.I. to ultimately be a normal technology in its nature and effects?
Calling A.I. normal does not imply that it is unimportant or that it won’t reshape the world. It just means that it will be revolutionary in the way the telephone or the personal computer or the internet was revolutionary, instead of, say, rapidly doubling human life spans or eliminating scarcity or enabling a successor species of machines to wipe us out. And it implies that the usual ways in which we approach a new technology — competition and innovation, regulation and adaptation — are appropriate to the A.I. age and we can debate them without fearing that the ground will suddenly dissolve beneath our feet.
Right now the two governments most intimately involved with A.I. — the Trump administration and the Politburo in Beijing — are on the normal-technology side of the debate. China seems more firm in its conviction, which is why it purports to be comfortable lagging American companies in the race for the A.I. frontier. But the White House, too, believes that it is engaged in a contest for technological advantage in which fears of machine gods and A.I. doom are mostly flights of fancy.