OpenAI's Superalignment Team Disbanded — Safety Behind "Shiny Products"

The Superalignment team was the research group OpenAI set up to control AI "much smarter than us", and in May 2024 it was dissolved less than a year after it began. Co-lead Ilya Sutskever announced his departure on 14 May, and the other co-lead, Jan Leike, posted on X the same day simply: "I resigned". On 17 May Leike explained himself in a longer thread, which included this line:

But over the past years, safety culture and processes have taken a backseat to shiny products.

OpenAI confirmed that day that it had disbanded the team and was integrating its researchers across its research efforts. The loss of the company's flagship safety team, together with both of its leaders, prompted outsiders to ask how AI companies actually rank safety against product development inside their own walls.

What the team had promised

In its announcement of 5 July 2023, OpenAI wrote that superintelligence could be the most impactful technology humanity had ever invented, but that its vast power could also lead to the disempowerment of humanity or even human extinction, and that although this seemed far off, it believed it could arrive this decade. It set the goal of solving the core technical challenges of superintelligence alignment — making an AI's goals and behaviour match human intent — within four years, launched a team co-led by Sutskever and Leike, and said it would dedicate 20% of the compute it had secured to date. The team's origins and Sutskever's research stance are covered on the Ilya Sutskever page.

What Leike wrote

His 17 May thread opened by saying the previous day had been his last as head of alignment, superalignment lead and executive at OpenAI. He went on to say he had been disagreeing with OpenAI leadership about the company's core priorities for quite some time, until they finally reached a breaking point. A line quoted by the Associated Press and Decrypt reads: "Building smarter-than-human machines is an inherently dangerous endeavor." Leike wrote that OpenAI was shouldering an enormous responsibility on behalf of all of humanity, and that it must become a safety-first AGI company.

The claim that the team was starved of compute came from reporting rather than from Leike's own words. TechCrunch reported on 18 May, citing a person on the team, that although it had been promised 20% of OpenAI's compute, requests for even a fraction of that were often denied, blocking its work; OpenAI did not immediately comment. According to the same article, co-founder John Schulman would lead the kind of work Superalignment had done, and the team would no longer be a dedicated unit but a loose group of researchers embedded across divisions — what an OpenAI spokesperson called "integrating [the team] more deeply."

Altman replied to Leike that he was right, that OpenAI had a lot more to do, and that it was committed to doing it.

Exit papers and equity

On the same 17 May, Vox's Kelsey Piper reported why no former employee was talking. The off-boarding agreement she had seen forbade former employees, for the rest of their lives, from criticising their former employer; even acknowledging that the agreement existed was a violation. Declining to sign or breaking it could cost a departing employee all the vested equity earned at the company. One former employee, Daniel Kokotajlo, had confirmed publicly that he gave up what would probably have been a large sum in order to quit without signing.

After publication, an OpenAI spokesperson sent Vox this statement: "We have never canceled any current or former employee's vested equity nor will we if people do not sign a release or nondisparagement agreement when they exit." On 18 May Altman acknowledged on X that the off-boarding documents had contained a provision about potential equity cancellation and said the company was changing that language. According to reports including GIGAZINE's, OpenAI later told former employees it would not take away vested equity whether or not they had signed, and that it was removing non-disparagement clauses from standard exit papers and releasing former employees from existing ones.

What it changed

Who can verify an AI company's safety claims? That is the question this episode left behind.

The most direct response came from people working at AI companies. On 4 June 2024, eleven current and former OpenAI employees and two current and former Google DeepMind employees published "A Right to Warn about Advanced Artificial Intelligence." It said AI companies possess substantial non-public information about the capabilities and limitations of their systems, and that current and former employees are among the few people who can hold them accountable to the public. It asked companies not to enforce agreements that silence risk-related criticism, not to retaliate by withholding vested economic benefits, and to provide anonymous channels for raising risk concerns, among four principles. Yoshua Bengio, Geoffrey Hinton and Stuart Russell endorsed it. The letter argued that ordinary whistleblower protections are insufficient because they focus on illegal activity, while many of the risks at issue are not yet regulated. The same concern ran through the whistleblower provisions of California's SB 1047, debated that year.

Talent moved too. On 28 May Leike announced he was joining Anthropic, where, according to PYMNTS, he would work on scalable oversight, weak-to-strong generalisation and automated alignment research.

OpenAI did say it would continue safety research. Greg Brockman's long reply, quoted by TechCrunch, thanked Leike and stressed the need for a very tight feedback loop and rigorous testing. But what could be verified from outside was the organisational fact: the team was gone and its researchers dispersed.

The pattern continued. Miles Brundage, OpenAI's senior adviser for AGI Readiness, wrote on 23 October 2024 as he announced his departure: "In short, neither OpenAI nor any other frontier lab is ready, and the world is also not ready." Members of his team were distributed to other teams.

There was no way for anyone outside to check whether the 20% compute pledge had been honoured; what was learned came only after insiders left, spoke and were reported on. In 2025 California enacted SB 53, which requires large frontier AI developers to publish safety frameworks, report critical safety incidents to the state and protect whistleblowers — an attempt to turn voluntary corporate pledges into something that can be checked from outside (the background is on the SB 1047 page).