Copilot+ PC and Recall — On-Device AI and an Instant Retreat

On 20 May 2024, at its Redmond campus, Microsoft announced Copilot+ PC—a new PC category requiring an NPU of at least 40 TOPS. The launch hardware was Qualcomm's Snapdragon X Elite / X Plus (NPU at 45 TOPS). More than twenty devices from Surface Laptop 7, Surface Pro 11, Dell, HP, Lenovo, Acer, ASUS, and Samsung shipped together on 18 June 2024.

Among its headline features was Recall—a system that screenshots the user's screen every few seconds, runs OCR and embedding over it, indexes everything locally, and lets the user search any past moment of their PC use in natural language.

The next day, Recall began turning into the largest security scandal of the AI-PC era.

What Copilot+ PC Was

What Microsoft put forward with Copilot+ PC was a redefinition of PC architecture as CPU + GPU + NPU.

The requirements:

• NPU: at least 40 TOPS (INT8)
• RAM: 16 GB or more
• SSD: 256 GB or more
• Architecture: initially Arm (Qualcomm Snapdragon X), later x86 (Intel Lunar Lake, AMD Ryzen AI 300)

It was Microsoft's answer to Apple Silicon (the Neural Engine in the M-series). Apple had been shipping integrated CPU + GPU + Neural Engine since the M1 in 2020. The Windows camp followed roughly four years later.

The significance of a 40+ TOPS NPU was scale: it makes on-device LLM inference, without cloud dependency, practical. For the new category Microsoft launched Phi Silica (around 3.3B parameters, on-device), Cocreator (a Stable Diffusion variant), Live Captions, Studio Effects, and more.

Recall — Design and Launch

Recall's behaviour was thorough and simple:

1. Take a screenshot of the active screen every 5 seconds.
2. Extract text via OCR; caption images via a multimodal model.
3. Store both in a local SQLite database, with vector embeddings.
4. Let the user search in natural language—"the apartment floor plan I looked at last week", "the invoice in Outlook three days ago that had a price on it".

As a feature it was attractive: search any past task across application boundaries.

Then security researchers looked at the implementation.

Kevin Beaumont's Disclosure

On 21 May 2024, the day after the keynote, security researcher Kevin Beaumont (a former Microsoft Senior Threat Intelligence Analyst) warned that Recall "fundamentally undermines Windows security". On 31 May he published a forensic teardown of the preview on his DoublePulsar blog.

The title was provocative: "Stealing everything you've ever typed or viewed on your own Windows PC is now possible with two lines of code — inside the Copilot+ Recall disaster."

What he found:

1. No encryption. The Recall database was a plain SQLite file in the user folder. Beyond the BitLocker layer ("encryption at rest"), there was no application-level encryption.

2. Readable from user context. Any process running in the same user session could read the entire database without authentication. Beaumont published a proof-of-concept, TotalRecall, and said extraction took "two lines of code".

3. Passwords were captured. Screenshots included values entered into bank password fields, secret-chat contents, and credit-card numbers in transit. App-level input masking was meaningless against OCR.

4. An ideal target for infostealers. Any malware running in user context could exfiltrate months of activity in a single sweep—the dream target for ransomware and data-extortion gangs.

The UK ICO asked Microsoft for an explanation. Several US legislators raised concerns.

The Retreat

Within two weeks of disclosure, Microsoft was forced to reverse course.

7 June 2024. Pavan Davuluri, head of Windows + Devices, posted on the official blog that Recall would shift from default-on to default-off (opt-in); Windows Hello biometric authentication would be required; and the database would be encrypted using just-in-time decryption.

13 June 2024. A further announcement: Recall would not ship with Copilot+ PC at launch on 18 June. It would be delayed into a limited Windows Insider Program rollout. The flagship AI feature was missing from the Copilot+ PC launch lineup.

August, October 2024. Further delays in the Insider channel, indicating that internal redesign was proving harder than expected.

The Limited Re-release of 22 November

On 22 November 2024, Microsoft finally rolled out Recall to a subset of Windows Insiders (Dev Channel) on Snapdragon Copilot+ PCs.

Key security changes:

1. Opt-in required. The user must explicitly enable Recall on first run (off by default). 2. Windows Hello required. Biometric (face or fingerprint) or PIN authentication is required to start, view, or search. 3. Processing inside a VBS enclave. OCR and embedding generation run inside a VBS (Virtualization-Based Security) enclave—a hypervisor-protected memory region inaccessible to ordinary user processes. 4. Database encryption. Just-in-time decryption only; the data is never persisted in plaintext. 5. Sensitive-content filtering. Password fields, credit-card numbers, and certain secret inputs are automatically detected and excluded from capture. 6. App and URL exclusion lists. Users can mark specific apps and websites as never-captured.

Re-testing the shipped feature on 21 April 2025, Beaumont credited Microsoft with having "spent significant time and resources on encrypting the Recall databases" and called the new design "much, much better". His remaining objection: after the initial setup, Recall can be opened — and past snapshots viewed and exported — with nothing more than the four-digit Windows Hello PIN, so the documented requirement for biometrics is not what actually holds in practice.

On 25 April 2025 Recall reached general availability, rolling out over the following month to Copilot+ PCs on Snapdragon, Intel and AMD silicon alike.

"Local" Did Not Mean "Safe"

The Recall incident will be remembered as a textbook failure in on-device-AI security design.

1. The "local is safe" myth. Microsoft's early talking point was "everything stays local, nothing goes to the cloud". But a locally stored database is, for malware, the pre-staged treasure pile waiting to be exfiltrated. "Local" does not mean "safe".

2. Feature-first, security-bolted-on. Between the 20 May keynote and Beaumont's 31 May disclosure—eleven days—the design collapsed under expert scrutiny. Either internal review did not catch these issues, or it did but could not stop shipping. Either possibility points to an organisational problem.

3. A precedent for AI-PC trust. Across Apple Intelligence, Gemini Nano, on-device Llama, and others, the industry is converging on designs in which "the AI is always watching the screen". The Recall episode drew the first line in the sand on what users and regulators will tolerate.

For all that, Copilot+ PC itself spread successfully. Per Canalys, NPU-equipped PCs were 23% of shipments in Q4 2024 (17% across 2024 as a whole), and were forecast to reach around 40% across 2025. The "AI PC" category stuck. But its supposed killer feature, Recall, ended up in a quiet position: an opt-in convenience users may or may not enable.

The polished 20 May announcement, and the muted 22 November re-release, define a six-month gap—and within it, the first lesson the on-device-AI era had to learn.