GPT-2 Held Back — Staged Release and AI Publication Norms

GPT-2 is a 1.5-billion-parameter language model that OpenAI announced on 14 February 2019. In the same announcement OpenAI said it would not release the trained model, citing concerns about misuse, and published only a much smaller version and a paper. Larger versions followed in stages in May and August, and the weights of the full 1.5-billion-parameter model were released on 5 November 2019. Widely reported as an AI too dangerous to release, the decision brought the question of publication norms—what to release, when, and in what order—into AI research.

The model's capabilities are covered in the article on its successor, GPT-3. This article is about the decision to hold it back, and what that decision did for whom.

The announcement of 14 February 2019

According to OpenAI's blog post, GPT-2 was a direct scale-up of the original GPT, a Transformer model trained only to predict the next word over 40GB of text from 8 million web pages. The data was chosen with a filter: only pages linked from Reddit posts that had received at least 3 karma, a heuristic for whether people found a link worthwhile, which OpenAI argued gave higher quality than indiscriminate crawls such as CommonCrawl.

Near the top of the post was the sentence that made headlines:

Due to our concerns about malicious applications of the technology, we are not releasing the trained model.

The post listed what was being withheld: the dataset, the training code and the GPT-2 model weights. What was released instead was a much smaller model for researchers, sampling code and a technical paper. OpenAI called the decision itself an experiment and wrote that it was not sure it was the right call. Its argument was that fields such as biotechnology and cybersecurity had long debated responsible publication of work with clear misuse potential, and that AI would eventually have to do the same.

The post named four malicious uses it had in mind: generating misleading news articles, impersonating others online, automating abusive or faked content for social media, and automating spam and phishing. It also listed beneficial ones: writing assistants, more capable dialogue agents, unsupervised translation and better speech recognition.

The staged release

In the August follow-up OpenAI wrote that it had spoken with more than five groups who had replicated GPT-2 since February, and that teams at Hugging Face and at the Allen Institute for AI with the University of Washington had adopted similar staged releases. Its four research partners were Cornell University (human susceptibility to synthetic disinformation), the Middlebury Institute's Center on Terrorism, Extremism, and Counterterrorism (CTEC; extremist misuse), the University of Oregon (bias probes) and the University of Texas at Austin (statistical detection).

The evidence behind the final release

The 5 November post set out its reasoning in numbers. In Cornell's survey, people gave GPT-2 text a credibility score out of 10 of 6.91 for the 1.5B model, 6.72 for 774M and 6.07 for 355M; OpenAI said the small increment over 774M made it more inclined to release. In August it had already reported a separate study in which 72% of one cohort judged GPT-2 articles credible, against 83% for real New York Times articles.

On detection it was cautious. OpenAI's own detector identified 1.5B GPT-2 text at a rate of about 95%, which it judged not accurate enough on its own; it would need to be paired with metadata, human judgment and public education. On misuse, OpenAI said it had seen no strong evidence so far, while acknowledging it could not be aware of all threats. CTEC showed that fine-tuning on four ideological positions, including white supremacy and jihadist Islamism, could produce synthetic propaganda.

The process was documented in a report by Irene Solaiman and 14 co-authors, Release Strategies and the Social Impacts of Language Models (August 2019).

Hype or a first step toward norms?

Reactions split. Helen Toner of Georgetown's Center for Security and Emerging Technology (CSET) later called publication norms the most attention-grabbing AI governance discussion of 2019, sparked by GPT-2. In her account, many machine-learning researchers saw the move as a deliberate bid for hype and media attention, and felt it damaged academic norms of openness by making replication and verification harder. Reactions in AI policy and governance circles were, by contrast, largely positive: they welcomed an effort to start building norms for research that could be misused, even if this particular work was not especially risky.

Toner also recounts how an independent researcher, Connor Leahy, announced that he had replicated GPT-2 and would release it in deliberate defiance of OpenAI's strategy, then changed his mind after talking with OpenAI and other researchers. The episode captures the shape of the argument: withholding weights does not stop anyone with enough compute and skill from building the same thing. Its effect is to buy time. OpenAI said as much in February, describing its strategy as limiting the initial set of organisations that might reproduce and release the model, and giving the community more time to discuss the implications.

What it left behind

In terms of who gained and who lost, university and independent researchers who could not freely study the full model bore the short-term cost, while disinformation and detection researchers gained early access as partners. The sequence—staged release, partnership-based sharing, and a model card published alongside the weights—became a template that later releases of large models would cite.

For GPT-3 in 2020, OpenAI never released the weights at all and offered the model through an API instead (see GPT-3). The argument between releasing weights and serving models from the cloud resurfaced around Meta's Llama in 2023. According to press coverage, OpenAI's first open-weight language models since GPT-2, gpt-oss, were released under the Apache 2.0 licence on 5 August 2025.

The underlying question has not been settled. But GPT-2 in 2019 was the first major case in which the organisation that built a model laid out, as a concrete procedure, the idea of measuring social impact and releasing in stages before handing it to everyone. Related events are on the AI timeline; the company is OpenAI, and the general term is large language model.