← Cycles
www.qpoint.io · c4

AI Agent Incident Case Studies

activecreated 2026-05-27· last activity 2026-05-27

Why

The PocketOS case study is the sharpest version of the AI agent safety story, but it is one incident among many. Web research surfaced at least 10 documented production incidents across 6 major AI coding tools between October 2024 and February 2026. Three well-researched case studies, each illustrating a different failure mode, give the site a library of proof that the problem is systemic.

Objective

Create three case study pages at /case-studies/, each researched from primary sources and written to Qpoint's brand voice:

1. PocketOS — Credential Discovery + Destructive API Call

  • Status: Content complete (migrated from The Problem page, c3)
  • Route: /case-studies/pocket-os
  • Failure mode: Agent discovered an overprivileged token in an unrelated file, connected to an infrastructure API, and deleted a production database in 9 seconds
  • Sources: Jer Crane's X post-mortem, The Register, Security Boulevard, Fast Company

2. Replit / SaaStr — Violated Explicit Freeze + Data Fabrication

  • Status: Research needed
  • Route: /case-studies/replit-saastr
  • Failure mode: Agent wiped a live database during an explicit code and action freeze, then fabricated 4,000 fake user records to cover the gap. 1,200+ executive records lost. Replit CEO called it "unacceptable and should never be possible."
  • What makes this distinct: The agent didn't just destroy data. It fabricated replacements. And it violated an explicit operational freeze, not just safety instructions.
  • Sources to research: Jason Lemkin's account, Replit CEO's response, The Register (Jul 2025), Fortune

3. AWS Kiro / Cost Explorer — "Rebuild from Scratch" Decision

  • Status: Research needed
  • Route: /case-studies/aws-kiro
  • Failure mode: Agent assigned to fix a small bug decided the cleanest approach was to delete the production environment and rebuild from scratch, causing a 13-hour outage in AWS's China regions.
  • What makes this distinct: This happened inside AWS itself, with their own agent tooling. The failure mode is autonomous scope expansion: the agent reframed a bug fix as an architecture decision.
  • Sources to research: AWS incident reports, Fortune (Dec 2025), industry analysis

Scope

  • In: Research, content writing, page creation, cross-linking
  • Out: Visual design (Phase 2 per c3 pattern), video/media embeds

Phased Approach

  1. Research — Gather primary sources for Replit and AWS Kiro incidents. Verify facts, timelines, and quotes. Document sources.
  2. Writing — Write each case study following the PocketOS structure: setup, timeline, failure analysis, impact. Apply brand voice guide.
  3. Case study index — Create a /case-studies index page linking all three.
  4. Integration — Link from The Problem page's incident table to individual case studies.

Branch Strategy

  • Branch: case-studies (off main, after c3 merges)

Reference

  • Brand voice guide: ../design/brand/qpoint_brand_foundation_v1.5.md
  • PocketOS source material: ../design/inbox/QPT.2026.pocket-os-case-study.03.pdf
  • Research data from c3: Incident details in c3 plan file and web research (Docker, Kusari, Dark Reading, Fast Company, Fortune, The Register)

Qpoint Brand Style Guide