Quote:
“The people building A.I. earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible. But I hear the same people express fear privately. No other human activity poses this level of danger.”
For more on the warnings that Coxon is raising and the broader questions about whether the A.I. industry is doing enough to address it, we’re joined now by Amrith Ramkumar, tech reporter at The Wall Street Journal.
Welcome back to the program.
Amrith Ramkumar, The Wall Street Journal:
Thanks so much for having me.
Geoff Bennett:
So we hear warnings about A.I. it feels like all the time now. Why did this one break through in the way that it did?
Amrith Ramkumar:
I think because he’s quitting the industry, frankly. This is someone who no longer has skin in the game, and he’s worked at two of the top A.I. companies.
And he’s talking about major risks late next year or the year after, and then, following after that, maybe the potential extinction of humans, depending on how things go. So I think the severity of the claims, the fact that he’s leaving, and the fact this is coming on top of several incidents of A.I. systems hacking other companies, all of that put together made this a pretty volatile moment.
Geoff Bennett:
And what he said was backed up by other A.I. researchers last night.
Amrith Ramkumar:
One hundred percent. You could have seen a world where he puts this out, our story goes out, his statements go out, and then others at the company distance themselves.
Instead, one of their top scientists said, we absolutely believe this. We think there’s a more than 10 percent chance that basically the human race is extinct within a decade because of A.I.
So it was a really remarkable sequence of events and highlighted how many at Anthropic specifically have this sort of doom-and-gloom concern out there.
Geoff Bennett:
And Anthropic is interesting because that company built a reputation around A.I. safety and regulation. You spoke exclusively with Jacob Coxon yesterday. What more did he say to you about his decision to step down and what does this mean about Anthropic really?
Amrith Ramkumar:
Well, the first thing that he said was interesting was, he left open A.I. for Anthropic a few months ago. So he spent a few years at OpenAI, helped trained some of their most advanced models, but he wanted to work at Anthropic because of what you said, the safety reputation.
Dario Amodei, the CEO, and other top executives speak about this a lot. But within a few months, his takeaway was that no company can do this safely. He wasn’t knocking Anthropic. He was saying that the private sector is not in position to do this right now because of the competitive forces and the race with China.
So he and others are really putting the onus on the government to get involved.
Geoff Bennett:
And I want to unpack part of this, because when he says that there’s a roughly 10 percent chance that A.I. could spell the end of humanity, when people say that, they’re not suggesting that there’s a- chatbot that’s out to kill everybody.
What they’re describing is this sort of like theoretical chain of events involving a much more capable future A.I. system, right?
Amrith Ramkumar:
That’s 100 percent right.
So the key first thing to understand there is, they’re very worried about this thing called recursive self-improvement. And that is when a model can essentially improve by itself and train itself, and other A.I. systems can then get much more powerful.
People say we could hit that within a couple of years. After that, they say it’s very hard to predict what could happen. So there’s a high degree of uncertainty. But one or two of the scenarios or some of the scenarios involve models training themselves and essentially learning how to reject human commands.
And once they can do that, and then they can do things like command drone swarms or robots or whatever the case might be, they might decide that humans aren’t necessary to fulfill their objectives. So that’s sort of the worst-case scenario they’re painting.
And it’s important that to say they assign probabilities to all of these things. So they’re not saying it’s a high probability necessarily, but it is on the table, it is possible.
Geoff Bennett:
And there has long been skepticism about the motives here. When CEOs and researchers sort of raise alarms about the potential future of A.I., there are people who suggest that they’re doing this because it serves their own interests and they want to have a hand in shaping the regulation that would affect their own industry.
Amrith Ramkumar:
One hundred percent. You hear this a lot from people like David Sacks, an adviser to the president on A.I., and he’s a venture capitalist, very well-known, and he says this is called regulatory capture when people like Anthropic make these doom-and-gloom claims to hype up their models and raise more money, et cetera.
The interesting thing here, though, is, he left the company. So that sort of is a counterargument, that he doesn’t have anything to gain necessarily from these regulations. So there is a community here that is very focused on the risk posed by A.I. and these sort of doom-and-gloom scenarios.
And a lot of them happen to work at Anthropic, and that does rub people on the other side of the debate in Silicon Valley the wrong way. What’s undeniable is that the risks here are rising. Like I said, you have had models hacking other organizations, hundreds of models working together, essentially, hundreds of A.I. agents to hack this company Hugging Face a few months ago, and many other examples that could come out that we don’t even know about from the new powerful models.
So you’re seeing people on both sides of the aisle respond. You have seen administration officials talk about the need to respond. So the risks here are becoming undeniable, even to the people that say people like Jacob Coxon are doomers.
Geoff Bennett:
Risks are undeniable for sure, and there’s no regulation on the horizon. And we should make clear, it’s easy to say A.I. needs to be regulated, but if you regulate too aggressively, you stifle innovation, you stifle competition. And certainly that’s the case, a concern about China.
Amrith Ramkumar:
That’s right.
Yes, it’s very easy to say there are no federal regulations, the Trump administration has made a huge error here, and there has to be something on the books. Much harder to get agreement on what that looks like. Even with the number of voices on the left and right that have come out today saying that we need regulations, there’s a little agreement on what those should be, even in Congress, even in the House, even in the Senate, even within those factions.
So what is interesting, though, is that you have this momentum and you do have the administration a little bit more willing to look at models before they’re released, even if it’s on a voluntary basis, and things are clearly trending in that direction.
What Coxon and co. want is global coordination. That’s even harder than just U.S. policy. They want the U.S. and China to reach an agreement, other leading economies to get on board with sort of a globally coordinated slowdown in the event that A.I. becomes too powerful.
That seems like a long shot right now, but the technology is moving very fast. We will see where things go.
Geoff Bennett:
All this doom and gloom with a smile, Amrith.
(Laughter)
Geoff Bennett:
Amrith Ramkumar, tech journalist for The Wall Street Journal, thanks for being with us.
Amrith Ramkumar:
Thanks so much for having me.
