When Pope Leo XIV presented a 42,300-word open letter to the world’s 1.4 billion Catholics on Monday, calling for protections against the rise of artificial intelligence, he was joined by Christopher Olah, a co-founder of Anthropic, which is one of the tech industry’s leading A.I. companies.
As Leo urged corporate executives, government regulators and other citizens of the world to safeguard humanity from the dangers of A.I., he included Mr. Olah as a symbol of the dialogue he hopes to foster between the leaders of the spiritual and technological worlds.
But for Jeremy Nixon, Monday’s gathering at the Vatican showed that those two worlds are far from aligned. While the pope said that A.I. was fundamentally not human, Mr. Nixon, a well-connected figure in the Bay Area’s frenetic A.I. scene, argued that Mr. Olah’s remarks seemed to hint at the opposite.
“They are not in dialogue,” Mr. Nixon said during an interview at A.G.I. House, a San Francisco “hacker house” with deep ties to many of the people who helped create the A.I. technologies discussed in the pope’s encyclical. “Their perspectives are distinct.”
The difference between the humanist’s view of A.I.’s risks and the technologist’s dream of what it could become is something that has long been discussed in Mr. Nixon’s community. “It is the reason the community exists,” Mr. Nixon said. “It is its underlying purpose.”
Mr. Nixon, 33, is one of the founders of A.G.I. House, which is named for Silicon Valley’s headlong pursuit of “artificial general intelligence,” a hypothetical machine that can do anything the human brain can do.
“We are named for this moment in history,” he said of his seven-room group house. Over the years, it has been home to a rotating collection of researchers, entrepreneurs and philosophers.
More than most, Mr. Nixon understands the technology emerging from Silicon Valley and the attitudes of the people building it. In the late 2010s and early 2020s, when Silicon Valley started developing the technologies that power chatbots, Mr. Nixon worked in Google’s central A.I. lab. Later, he founded A.G.I. House with Andrej Karpathy, who was an early employee at OpenAI, oversaw self-driving tech at Tesla and recently joined Anthropic.
Mr. Nixon said the papal encyclical might mean something to the world’s Catholics, but he doubted that it would have an effect on Silicon Valley. The only reason that Silicon Valley even paid attention to the event, he said, was that Leo invited Mr. Olah to speak.
For Mr. Nixon, the pope’s open letter read like one of the many government policy documents that have been shared among think tanks and regulators over the past several years. “It did not seem like the church had thought deeply about what their independent perspective was on A.I.,” he said. “It is not clear they could do that even if they tried.”
He added: “They couldn’t have a position on it, because they don’t understand it.”
The response to the encyclical from across Silicon Valley was fairly muted.
David Sacks, the Silicon Valley investor who served as the White House A.I. czar, pushed back on the pope’s call for additional A.I. regulation. “If we hand governments sweeping power over A.I. development in the name of safety, how do we prevent it from being used to censor, surveil, and control citizens — as Orwell foretold in ‘1984’?” he said.
Jack Dorsey, the co-founder of Twitter and who now runs the financial company Block, welcomed the pope’s call for A.I.’s underpinnings — patents, data and technological infrastructure — to be shared with everyone, not just a relative few.
The Vatican did not respond to a request for comment.
Looking out over San Francisco, toward the Golden Gate Bridge in the north, A.G.I. House plays host to ad hoc seminars on the philosophies driving Silicon Valley’s pursuit of A.I. and “hackathons” in which researchers from the leading labs explore the boundaries of the technologies they are building.
In the early afternoon following the pope’s encyclical, dozens of foldable chairs were set up in rows across the living room. But on this Memorial Day, the only figures in the room were two jet-black mannequins that stood near the wall.
The A.G.I. House founders first moved into a $68 million mansion in Hillsborough, Calif., just south of San Francisco. They wanted to create a group house where researchers and engineers gathered to discuss new ideas and create new companies. Mr. Nixon started a second house in San Francisco near the top of Twin Peaks, overlooking the city from its highest hills.
Many of the founders and important researchers at Anthropic and OpenAI joined the earliest gatherings at A.G.I. House. Mr. Nixon is now founder and chief executive of a start-up called the Infinity Artificial Intelligence Institute, which is trying to automate the creation of A.I.
Mr. Nixon said he has met a generation of scientists who shunned traditional religion in favor of technology. After growing up with books like “The God Delusion” — in which the evolutionary biologist Richard Dawkins painted God as a false belief contradicted by empirical evidence — he and his peers saw A.I. as an alternative that was more real and far more powerful.
A.I. has started to crack math problems that humans struggled with for decades, he said, and it will soon cure diseases in the same way. “Practically speaking, it will achieve the outcomes that many religions claim their deities would be able to achieve,” he said.
This is an increasingly common belief among researchers in Silicon Valley. They insist they are on their way to building a more powerful species — or even a new God.
“People are matter-of-factly saying that they are looking to build a machine God,” said Rayan Krishnan, the chief executive of Vals AI, a San Francisco company that tracks the performance of the latest A.I. technologies. “They are not saying that ironically or in jest. They are saying it as a matter of fact.”
In his encyclical, called “Magnifica Humanitas” or “Magnificent Humanity,” Leo called for safeguards that would protect humanity. He stressed that humans must continue to play a fundamental role in the workplace.
“A society that guarantees employment to only a small fraction of the population, despite having a high level of technical development, risks exposing many to forced inactivity,” he wrote. “This creates a paradox of material progress and anthropological regression that undermines the foundations of a just and stable social peace.”
Work, he added, is not just a way of earning income. It is “a requirement of the human condition, a normal path toward maturity, development and personal fulfillment.”
He also used the biblical story of the Tower of Babel — in which a group of humans who speak a single unified language aim to build a tower that reaches to heavens so they can exert their power — as a way of warning Silicon Valley against the pitfalls of trying to outdo God.
His most forceful message was that A.I. was fundamentally not human, because it could only imitate certain parts of human intelligence and behavior. “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean,” he wrote.
But in a seven-minute speech at the Vatican, Mr. Olah, the Anthropic co-founder, described a technology that may be progressing beyond a simple machine.
“What is actually happening inside them?” he said. “We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief and unease.”
Mr. Nixon thought Mr. Olah’s speech was far more spiritual than the words delivered by the pope, even if Mr. Olah was making claims that were not necessarily based on science. Though Mr. Olah hinted that chatbots may have working lives of their own, Mr. Nixon said, they are still very much dependent on humans to do anything, as Mr. Olah noted.
“A.I. is still controlled by people who are trying to make money or solve some mathematical problem or what have you,” Mr. Nixon said.
But what is clear, he added, is that A.I. researchers are trying to build technologies that have jobs, feel joy and pain, and exhibit all sorts of qualities that match and even exceed the traits that make us human. He believes it could happen within the decade.
He chuckled at the pope’s invocation of the Tower of Babel. When he was at Google, a project the company called Babel built translation technology that has essentially allowed everyone to speak the same language.
On Monday evening, he and others in his circle gathered in the A.G.I. House living room to discuss the encyclical. They agreed that Leo’s open letter would have little effect on Silicon Valley, he said, but they wondered whether Silicon Valley might have an effect on the pope — if the Vatican might use A.I. technologies to “create a New Jerusalem.”
“A.I. and its capabilities represent something analogous to the Second Coming,” he said.
