Devin — The Agent Billed as 'the First AI Software Engineer'

Devin is an autonomous coding agent announced by the US start-up Cognition on 12 March 2024. The announcement was titled "Introducing Devin, the first AI software engineer"; the label is Cognition's own description, not a title anyone else conferred. Devin was said to work in a sandbox equipped with a shell, a code editor and a browser, planning a task, writing and testing the code, debugging it, reporting progress and taking feedback from people. At announcement it was available only through an early-access waitlist; anyone could use it only nine months later, from 10 December 2024.

If you want a one-line answer to "what is Devin", it was the first big example of an AI sold not as an assistant but as a colleague that takes on whole jobs. The pitch drew as much backlash as excitement, and it forced a closer look at what "AI engineer" should mean.

The announcement: 13.86% on SWE-bench

Cognition's main evidence was a score on SWE-bench, a benchmark published in October 2023 by researchers at Princeton and elsewhere that asks AI systems to resolve 2,294 real GitHub issues drawn from 12 popular Python repositories. In the paper the best model, Claude 2, resolved 1.96%.

Devin's figure, however, was measured on a random 25% of the full set, 570 of the 2,294 issues. Cognition explained the comparison by noting that Devin was not told which files to change and had to navigate the repository itself. Both the measurement and the announcement were Cognition's own. Later, Anthropic and OpenAI took to reporting scores on SWE-bench Verified, a human-validated set of 500 problems.

The announcement also said "We even tried giving Devin real jobs on Upwork and it could do those too!" and disclosed a $21 million Series A led by Founders Fund.

Criticism of the demo

Within weeks came a rebuttal. Carl, a veteran software engineer who runs the YouTube channel Internet of Bugs, went through the demo video of Devin's Upwork work in detail. As summarised by The Pragmatic Engineer on 18 April 2024, his points were these.

• Devin did not complete the Upwork task in the way the video implied
• The video said Devin fixed bugs, but those bugs were not in the original repository
• The work took between 6 and 26 hours

Walden Yan, the Cognition developer who made the video, defended it but wrote: "we've also had Devin complete other Upwork jobs, but it definitely makes mistakes or often fails." The Pragmatic Engineer described this as a major retreat from the original claim that Devin had completed real jobs on Upwork.

General availability and independent testing

On 10 December 2024 Cognition made Devin generally available "starting at $500 a month for engineering teams", with no seat limits. The plan included the Slack integration, an IDE extension and the API.

About a month later, on 8 January 2025, Hamel Husain and colleagues at the AI research company Answer.AI published an account of a month with Devin. Of 20 tasks, 3 succeeded, 14 failed and 3 were inconclusive. The authors said the biggest problem was that they could not predict which tasks would work, and concluded they would stick with developer-driven tools such as Cursor.

To a $10.2 billion valuation

Despite the criticism the business grew fast. Cognition said on 8 September 2025 that Devin's annual recurring revenue had grown from $1 million in September 2024 to $73 million in June 2025. It had acquired the AI coding environment Windsurf in July 2025, which it said more than doubled its ARR. In the same announcement it disclosed raising over $400 million at a $10.2 billion post-money valuation, led by Founders Fund, and named Goldman Sachs, Citi, Dell, Cisco and Palantir among its customers. These are Cognition's figures.

When the "AI colleague" comes to work

By presenting AI as a colleague rather than a tool, Devin's marketing spoke directly to working people. According to CNBC, Goldman Sachs's chief information officer Marco Argenti said in July 2025 that the bank, with about 12,000 human developers, was piloting Devin: "Initially, we will have hundreds of Devins [and] that might go into the thousands, depending on the use cases." He described a "hybrid workforce" in which humans and AI coexist, with Devin supervised by human employees and handling work engineers often find tedious, such as updating internal code to newer languages.

Argenti's words stress collaboration rather than replacement, yet the phrase "hundreds of Devins" itself counts AI as headcount. Meanwhile the Answer.AI test and the dispute over the demo showed that, at least in 2024 and 2025, autonomous agents were not yet at a level where they could be left to work without human checks. The gap between the marketing language and actual performance is a problem every later coding agent inherited.

The starting point of the autonomous-agent craze is Auto-GPT, AI that operates a screen is Claude's computer use, and the later products from the large labs are on the Claude Code and OpenAI Codex pages. Effects on junior jobs are on the page on AI and entry-level jobs, and the wider context on the AI timeline and the AI and society timeline.