The New Manhattan Project: PW Talks with Kevin Roose

In The AGI Chronicles, the journalist tracks the race to develop artificial intelligence that can match or surpass human capabilities.

In reporting on the people driving the sprint to build artificial general intelligence, did anyone emerge as the moral center of the story?

I find Dario Amodei, the CEO of Anthropic, totally fascinating as a moral case study. This is a guy who’s spent more than a decade warning about the risks of powerful AI systems. At the same time, he’s completely obsessed with pushing the AI frontier forward, on the theory that AGI is going to be built eventually, and it’s better if the good guys get there first. He’s constantly wrestling with the cognitive dissonance of racing to build a technology he believes could destroy the world. And now he’s standing up for his beliefs in the face of political opposition from the Trump administration and trying to lead his company to a trillion-dollar IPO.

Which figure diverged most from their public persona?

Eliezer Yudkowsky. He’s sort of the original AI doomer. Recently, he wrote a book called If Anyone Builds It, Everyone Dies, and his ideas have been incredibly influential in the AI safety movement. He has a reputation as a dark, brooding figure. But I spent time with him and found him to be quirky and funny. He does think we’re all probably going to die, though, so I guess that part is true to type.

What role, if any, should governments play in the development of AI—or is it too late?

I don’t think it’s too late, but the window is closing faster than most people realize. I think the government needs to play a strong oversight role, especially as the technology gets more powerful. I like DeepMind CEO Demis Hassabis’s idea of a federal standards body for AI developers as a starting point, but I think there will likely need to be much more regulation than that. Right now, in America, you need a license to open a sandwich shop, but you can build and release a frontier AI model capable of carrying out autonomous cyberattacks with no trouble at all. That seems like it must change.

How does the race to build AGI compare to the Manhattan Project?

I think the race to AGI could end up having a bigger impact than the Manhattan Project, because the underlying technology is so much more versatile. The atomic bomb ended a war and reshaped the global order, and it could only do one thing—blow up. AI can be a weapon, but it can also do incredibly useful and beneficial things, like accelerating scientific research or giving every student access to world-class tutors. We don’t yet know whether the positive or negative effects will be more profound.