ParasocialDownload on theApp Store

Dwarkesh Podcast

Paul Christiano – We could be 12 months away from AIs that can overthrow the government

1 hr 39 min

3 min preview

Listen to the full episode

Do not have it yet? Parasocial is a podcast player by Super Simple.

About this episode

Paul Christiano is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen). Paul thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario). He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails. He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it. And he discusses the plan for alignment - basically, making a superintelligence that actually doe

Paul Christiano – We could be 12 months away from AIs that can overthrow the government — Dwarkesh Podcast