OpenAI CEO Sam Altman reportedly described three possible “end states” for safely advancing AI during a talk to employees last week.
The three options are an international oversight body, a merger of leading labs, or a single lab pulling ahead on its own. Altman reportedly called the need to choose a path within a year “pacing 2.0” and said AI self-improvement would require speed limits or stopping points.
A day when pre-release testing is no longer enough
OpenAI’s current safety approach is to release a model after it passes safety tests and disclose any concerns. Altman reportedly calls this approach “pacing 1.0.” To use a car analogy, it is like inspecting a finished car before delivery and sending it onto the road if everything checks out.
But the picture changes once AI starts improving itself. The question is no longer just how to release a car that has passed inspection, but how to manage the pace at which cars themselves evolve. The next stage, described as “pacing 2.0,” addresses precisely that issue.
Three proposals, three very different visions of governance
The three proposals differ sharply over who would control AI development. Here’s a look at how they compare.
The first proposal is to create a global oversight body like the International Atomic Energy Agency (IAEA). Altman reportedly supports this idea, and OpenAI has advocated for the need for such a framework since 2023.
The second proposal is called the “Demis plan,” after DeepMind co-founder Demis Hassabis. It would bring leading labs together to conduct research on behalf of participating countries, in a setup described as similar to CERN, Europe’s particle physics research organization.
The third scenario bears the name of AI researcher Eliezer Yudkowsky. One lab would use self-improving AI to pull ahead, leaving competitors so far behind that it could “melt” their GPUs. Altman reportedly called this outcome “terrible.”
What the three proposals reveal
The oversight body proposal could be summed up as “setting the rules from the outside,” the lab merger as “concentrating research resources in one place,” and a single lab pulling ahead as “leaving it to the winner of the race.” All three aim for a safe future, but they place authority in different hands.
Personally, I find it troubling that the third proposal may be the most likely to become reality. If the competition continues without anyone making a decision, that is where things could naturally lead. That may be why Altman used his strongest language for this option.
The question is who sets the speed limits
Altman reportedly said a path must be chosen within a year and that it should include “speed limits” or stopping points for self-improving AI. It’s like trying to add a brake design after building a car with nothing but an accelerator.
The question is who will apply those brakes. Would authority go to an oversight body, or would the labs be consolidated into one? Either way, the plan cannot work unless all the key players are at the table. The idea of every lab gathering under one roof, like in a superhero movie, is appealing. But, as with past international agreements, getting everyone to participate may prove to be the biggest challenge.
A direction echoed by a rival
Around the same time, Anthropic’s Jack Clark reportedly discussed the idea of placing a ceiling on intelligence and said there were “only a few years” left to act on safety. Executives at two competing companies are voicing concerns that point in a similar direction.
When competitors start talking about the same challenge, the debate may move from principles to implementation. However, OpenAI has not formally adopted any of the proposals; they remain ideas for now. Will the one-year deadline become a clock that moves the industry-wide discussion forward? It’s worth watching.
Comments