XBOW — An Autonomous AI Tops HackerOne's US Leaderboard

XBOW is an autonomous penetration-testing system — an AI that probes websites and applications for security holes the way a human "pentester" would. On 24 June 2025 the company announced that XBOW had reached the top of HackerOne's US leaderboard, the ranking of the bug-bounty platform where companies pay outside researchers to report vulnerabilities. In a post by its head of security Nico Waisman, XBOW wrote that this was the first time in bug-bounty history that an autonomous penetration tester had "reached the top spot on the US leaderboard".

The ranking was real, but what it measured was narrower than many headlines suggested. HackerOne's leaderboard ranks researchers by reputation — a score built from the number of reports and how many are accepted — not by earnings, and the top spot is a position at a point in time. The same months that made XBOW famous also showed the other side of AI bug hunting: open-source maintainers, led most vocally by curl's Daniel Stenberg, were drowning in AI-generated reports that described bugs that did not exist.

What XBOW reported

XBOW's own account gives the numbers behind the ranking. They are the company's figures; most of the underlying reports are not public.

XBOW said about 45% of its findings were still awaiting resolution. For the previous 90 days it reported 54 findings rated critical, 242 high, 524 medium and 65 low by programme owners. The vulnerability classes it listed were the staples of web security — remote code execution, SQL injection, XML external entities, path traversal, server-side request forgery, cross-site scripting and others — and it highlighted one previously unknown flaw in Palo Alto Networks' GlobalProtect VPN that it said affected more than 2,000 hosts.

The system was not working unsupervised. XBOW's post states that its security team reviewed the findings before submission to comply with HackerOne's policy on automated tools. Waisman told CSO Online that XBOW had been tested like any external researcher, with no inside knowledge of its targets.

What the leaderboard measures

Reputation rewards volume and acceptance. That made it a reasonable yardstick for a system whose advantage was tireless scale, and a misleading one for anyone who read "number one" as "best hacker in America".

The most detailed critique came later, in a February 2026 essay by the security researcher Dean Valentine titled Be skeptical of milestone announcements by young AI startups. Valentine pointed out that XBOW ranked first on reputation, not on bounty earnings, and wrote that its account had earned less than US$40,000 since it was created in February 2024 — respectable for a hobbyist but, in his words, "well below what professional bug bounty hunters make". He quoted his colleague Kyle Avery as finding that about 90% of XBOW's score came from a single category, cross-site scripting, one of the bug classes easiest to find programmatically. Valentine also cited XBOW's own statement that every vulnerability it reported had involved a human in the loop.

None of this makes the result fake. An automated system found and reported hundreds of real, accepted vulnerabilities in production systems of well-known companies over a few months. The critique is about framing: what was demonstrated was scale on common bug classes, with human filtering, not a machine that out-thought the best human researchers.

The view from the other side of the inbox

While XBOW was climbing, maintainers were complaining. Daniel Stenberg, who leads the open-source data-transfer tool curl, had already published The I in LLM stands for intelligence in January 2024, describing bounty reports that were apparently produced with AI: the more convincing they looked, the longer they took to assess and discard, pulling developers away from real bugs. Of 415 vulnerability reports curl had received through its bounty, he counted 64 confirmed security issues.

According to The Register, the curl bounty had paid more than US$90,000 across 81 awards since 2019. When it closed, BleepingComputer reported that curl had counted twenty submissions in 2026, none of which identified a vulnerability. Stenberg said the main goal of shutting down the bounty was to remove the incentive to submit junk.

The Rogers episode matters because it cuts the other way. Rogers used a set of commercial AI security tools — Almanax, Amplify Security, Corgea, Gecko Security and ZeroPath — and sent curl a long list of potential problems; Stenberg called the findings "truly awesome", while noting that most were small bugs rather than security holes. The difference was not whether AI was used but whether a skilled person checked the output before sending it — exactly the step XBOW and Google's Big Sleep said they performed.

Winners and losers

For companies running bounty programmes, autonomous testers promised cheap, continuous coverage of the routine flaws that make up most web vulnerabilities. For investors the ranking was a marketing asset: XBOW cited it when announcing a US$75 million Series B led by Altimeter, bringing its total funding to US$117 million.

The costs landed on people with no stake in the product. Volunteer maintainers absorbed the triage burden of every unverified report. Some analysts also worried about the attacker's share of the gain. David Shipley of Beauceron Security told CSO Online: "Unfortunately, this use of artificial intelligence favors attackers over defenders in this scenario." His argument was that defenders already struggle to patch fast enough, and faster discovery helps whoever acts first.

By 2026 the frontier had moved. When Anthropic restricted Claude Mythos Preview to vetted defenders, XBOW was among the testers whose verdict Anthropic quoted — calling the model a "significant step up" on its web-exploit benchmark. The autonomous hacker that topped a leaderboard in 2025 had become a benchmark for the next generation.

See also the cybersecurity timeline and the AI timeline.