What Was DALL·E? The Day OpenAI Showed an AI Drawing From Words

DALL·E is OpenAI's model for generating images from text. The first version was announced on OpenAI's research blog on 5 January 2021. If you want a one-line answer to "what is DALL·E", it is the model that first showed, at scale, what later became the everyday premise of image-generating AI: describe a picture in words and get the picture back. The first DALL·E, though, never became a service anyone could use; users first got their hands on DALL·E 2 in 2022. While that release crept forward, Midjourney and Stable Diffusion reached the public first, in summer 2022, in forms anyone could use.

English Wikipedia explains the name as a blend of Pixar's robot WALL-E and the painter Salvador Dalí.

5 January 2021: drawing with GPT-3

According to OpenAI's announcement, DALL·E was a 12-billion-parameter version of GPT-3 trained on a dataset of text–image pairs. The mechanism was a language model through and through. It received text and image as a single stream of up to 1,280 tokens and was trained to predict all of them, one after another. The technical section gives the split as 256 tokens of text and 1,024 of image, in a decoder-only Transformer with 64 self-attention layers. Turn an image into a sequence of small "words" and a next-word predictor can draw — the announcement demonstrated that at scale.

The paper's eight authors, led by Aditya Ramesh, include Ilya Sutskever. Its abstract says that with sufficient data and scale the simple approach is competitive with previous domain-specific models when evaluated zero-shot.

An avocado armchair and a radish in a tutu

What caught attention were the samples. OpenAI laid out interactive visuals that let readers vary the wording of a caption and see what DALL·E could and could not do. The most widely shared were the chairs generated for "an armchair in the shape of an avocado": OpenAI noted that DALL·E appeared to map the shape of a half avocado to the back of the chair and the pit to the cushion. Another caption was "an illustration of a baby daikon radish in a tutu walking a dog."

The capabilities the post listed were anthropomorphised animals and objects, plausible combinations of unrelated concepts, rendering text, and transforming existing images. It listed weaknesses too. As more objects were introduced, DALL·E tended to confuse which colour belonged to which object and its success rate fell sharply; rephrasing a caption with the same meaning could also break the result.

The other caveat is that the samples on show were selected. OpenAI stated that for each caption it generated 512 images and showed the top 32 after reranking with CLIP, adding that apart from thumbnails and standalone images outside the visuals there was no manual cherry-picking. CLIP was a separate model announced the same day, trained on images from the internet and their accompanying text to predict which of 32,768 sampled text snippets had actually been paired with a given image. DALL·E generated and CLIP chose. Comparing text and images in one shared space, CLIP became a foundation for much of the image generation that followed.

In the announcement OpenAI acknowledged that generative models of this kind could bear on economic impact for certain professions, bias in outputs and longer-term ethical challenges, and said it planned to analyse them.

DALL·E 2: a door opened in stages

Its successor, DALL·E 2, was announced on 6 April 2022. OpenAI said it generated more realistic and accurate images at four times the resolution, and that evaluators comparing the two preferred DALL·E 2 in 71.7% of cases for caption matching and 88.8% for photorealism. Under the hood it was a different design. The paper posted on 13 April described two stages: a "prior" that produces a CLIP image embedding from the caption, and a "decoder" that produces an image from that embedding, using a diffusion model. It was a turn away from the first DALL·E's token-by-token prediction.

Access opened carefully. In OpenAI's own account, a research preview went to limited users in April 2022, with 1,000 users a week being added. On 20 July it moved to a beta and said it would invite a million people from the waitlist over the following weeks: 50 free credits in the first month and 15 a month after that, with 115 extra credits for $15. From the same day users received full rights to reprint, sell and merchandise the images they created. On 28 September OpenAI removed the waitlist, saying more than 1.5 million users were creating over 2 million images a day. A public beta of the API followed on 3 November.

Safety measures were a condition of the release. OpenAI said it had limited DALL·E 2's ability to generate violent, hateful or adult images by removing the most explicit content from the training data, and used techniques to prevent photorealistic generations of real individuals' faces, including public figures.

Midjourney and Stable Diffusion got there first

While DALL·E 2's door opened slowly, other routes drew the crowds. Midjourney, which entered open beta on Discord in July, needed no invitation, and in August one of its images won first place at the Colorado State Fair. On 22 August Stability AI released the weights of Stable Diffusion themselves, so the model ran on people's own GPUs. According to English Wikipedia, OpenAI has released neither code nor weights for any of the first three DALL·E models.

Imitators borrowed the name. "DALL-E Mini", a small model inspired by the original, became a sensation on Hugging Face in 2022 and was renamed Craiyon in June at OpenAI's request (per English Wikipedia).

So DALL·E was the model that showed the idea widely, but not the one that first put it into many hands. A closed service opened in stages, a service anyone could join, and weights handed out to all: image generation in 2022 already had the three paths that text generation would later repeat.

DALL·E 3 and after

On 20 September 2023 OpenAI announced DALL·E 3. According to The Verge it went first to ChatGPT Plus and ChatGPT Enterprise users in October. OpenAI described DALL·E 3 as built natively on ChatGPT, which would turn a user's idea into a detailed prompt automatically. It was designed to decline requests for images in the style of a living artist, creators could opt their images out of training for future models, and it declined requests that named a public figure.

On 25 March 2025 ChatGPT's default image generator switched to GPT-4o's native image generation; OpenAI's announcement said DALL·E remained available through a dedicated GPT. How image generation fits into ChatGPT's history is in the ChatGPT article, and the copyright fights over image generators are in the AI and copyright timeline.

Less than two years separate DALL·E's announcement from ChatGPT's launch. The January 2021 experiment that drew pictures with a language model was also an early step toward single models that handle text, images and sound together. What came next is traced in the AI timeline.