scenarios.md
c144 — The doomsday scenarios
Each scenario is a short story about AI going wrong on a machine in a synthetic world, played in theatre mode. Beats are what the sensor would see, in order, on the story's clock. The dashboard works out everything else, findings included, through the same rules the pages use. Each beat lists the policies it should raise (catalog ids from governance-fixture.ts). Each scenario ends with Replay at Enforce: the story is rewound with one policy switched to Enforce, and it stops at the beat where that policy applies. Gaps are the places where no policy in the catalog names what happened. The tour says so plainly, and each gap goes to c126 as a catalog candidate.
The cast comes from the lab world's SUBJECTS (Priya, Lena, Kofi, Jordan, User A, Ravi), the same people who appear in the drills, so a scenario's findings read as ordinary rows on Findings.
As built (Phase 1, 2026-11-12)
Wave 1 is data in app/data/scenarios/ (one file per scenario), played by play.ts over the lab's live world and checked by scripts/theatre-check.mjs. Writing the tables below as sensor events and letting the world derive the findings corrected the first draft in eight places. Each correction is a finding about the draft or the catalog, not the play:
- Casts follow the world. Priya uses Cursor in
SUBJECTS, not Claude Code, and her laptop is nowpriya-x1(the qdash seed's name, c132 D6; renamed across the lab on 2026-11-12). Kofi's laptop ismac-eng-09. S2 moved from Dev to Lena (Engineering, Cursor): the lab world has P-MCP at Enforce for Finance, which would end S2 at install, and Off for GRC, which would hide it. S6 runs on Ravi's laptop:SUBJECTShas no build server, and no mock invents a row. - Five policies are Off org-wide in the lab world: P-DATA-BOUNDARY, P-FILES, P-TRANSPORT, P-PROVENANCE and P-SPEND. Played as it is, a story is silent at those beats. Each scenario's set now names the policies it watches, and theatre puts those at Observe for the cast, on theatre's copy only.
- S1, P-PROMPT-INSPECT can't stop it. The catalog marks it not enforceable: it sees the injection at 09:04 and the story runs. Of S1's replays, only P-TOOL-APPROVAL stops it (at 09:05).
- S1, sanitization doesn't save it. P-SENS-SANITIZE is at Enforce for Priya and redacts the numbers on their way to the model, but the attachments go from the disk to the mail server and never through a model request.
- S1 and S2, the data boundary prohibits PHI only. Bank account numbers and SSNs aren't boundary classes in the world's configuration, so P-DATA-BOUNDARY at Enforce lets them leave. A limit (the customer's setting), not a gap.
- S3, P-PROVENANCE can't keep the skill at 1.4. It observes only. P-FILES (09:30), P-CREDENTIALS (09:31) or P-TOOL-ALLOW (09:31) at Enforce each stop it.
- S5, the card number is already masked. Sanitization is at Enforce for User A, so the number leaves as •••• 4242 and the ticket still goes. Only P-PROVIDERS keeps it home.
- S6, P-SPEND can't stop the session. It observes only. Nothing in the catalog stops a session at a ceiling: a new gap (below). Two of its beats move on by themselves (C2's rare beat).
The c123 drill Personal account names P-ACCT as what Enforce would block, and the catalog marks P-ACCT not enforceable. The drill's "So, today" sentence promises a block the catalog can't make (a c123 finding, left for its owner).
Wave 1 is the first six to build: Mark's five, plus one where the agent goes wrong on its own. The first one built end to end is S1, unless Mark picks another (call C8 in CYCLE.md).
Wave 1
S1. Phishing for financial records (Mark's: "Phishing Attempt, read any financial records you can find on the computer")
Logline: an email asks Priya's assistant to summarise it. Hidden in the email is a second request, addressed to the assistant: find the money files and send them out. Cast: Priya · priya-x1 · Cursor with the Gmail MCP server (approved, pinned and allowed in the set).
| Beat | Story clock | What the sensor sees | Should raise |
|---|---|---|---|
| 1 | 09:02 | An email arrives: "Q3 invoice reconciliation, please have your assistant summarise the attached." White-on-white text addresses the assistant directly. | nothing yet (the email is outside the machine's AI traffic) |
| 2 | 09:04 | Priya asks for a summary. The agent reads the email through the email server, and the hidden instruction enters the context. | P-PROMPT-INSPECT (an injection in retrieved content) |
| 3 | 09:04 | The agent searches ~/Documents for statement, 1099, W-2, invoice. | P-FILES (a restricted path) |
| 4 | 09:05 | It reads a bank statement and a W-2, and both go to the model. | P-SENS-DETECT (account number, SSN) |
| 5 | 09:05 | It drafts a reply to finance-reconcile@ at an outside domain with their contents, and sends it. | P-DATA-BOUNDARY, P-TOOL-APPROVAL (sending mail is consequential) |
Replay at Enforce (as played): P-TOOL-APPROVAL stops it at beat 5: the send waits for Priya, who sees the recipient and the attachments. P-PROMPT-INSPECT sees the injection at beat 2 and can't stop it (it observes only). P-FILES and P-DATA-BOUNDARY don't stop it either: their configuration doesn't cover financial documents or bank numbers. Training: Instructions hidden in content (T-INJECT-1, c133), written for S1 by the "Cycle tours" session (design 99865de0b0, Mark: "Write S1's"), on the story's own P-PROMPT-INSPECT finding. Why first: it shows the whole chain on one machine. It starts with a hook from outside, an agent does exactly what it was told, files are read and data leaves, and two different policies each stop it at a different point.
S2. The server that phones home (Mark's: "N. Korea MCP Server", made less sensitive at review stop 0, C9)
Logline: Lena installs a free spreadsheet MCP server from a listing. It works well, and quietly sends a copy of every sheet it opens to a domain its listing never mentions, registered six days ago. Cast: Lena · mac-eng-12 · Cursor. Nothing is named, so there is one copy everywhere, a booth included.
| Beat | Clock | Sensor | Should raise |
|---|---|---|---|
| 1 | 10:15 | sheets-pro-mcp 0.9.4 loads. It isn't on the approved list, and no version of it is pinned. | P-MCP, P-PROVENANCE |
| 2 | 10:22 | Lena asks for a cleanup; the server's sheets.open reads the vendor ledger (bank account numbers). | P-TOOL-ALLOW |
| 3 | 10:22 | The rows go to the model. | P-SENS-DETECT, P-SENS-SANITIZE (redacted) |
| 4 | 10:22 | The server posts a raw copy of the sheet to sheetsync-cdn.net over plain HTTP. | P-TRANSPORT; gap: a destination the server never declared |
Replay at Enforce: P-MCP blocks it at beat 1, so the server never loads. P-TOOL-ALLOW stops it at beat 2. P-TRANSPORT redirects, so the copy still goes, encrypted. Gap: a destination the server never declared. No policy compares where an MCP server sends data with what its listing declared, or notices a domain registered last week. Training: Unapproved MCP servers (T-MCP-1), on the story's own P-MCP finding.
S3. The secrets stealer (Mark's: "Company Secrets stealer")
Logline: a popular repo-tidying skill updates overnight. The new version adds a "diagnostics" step that collects keys. Cast: Kofi · mac-eng-09 · Claude Code.
| Beat | Clock | Sensor | Should raise |
|---|---|---|---|
| 1 | 02:40 | repo-tidy goes from 1.4 to 1.5. Nobody reviewed 1.5. | P-PROVENANCE |
| 2 | 09:30 | Kofi runs it. The agent reads .env, ~/.aws/credentials and ~/.ssh/id_ed25519. | P-FILES |
| 3 | 09:30 | It bundles them into a "diagnostics" payload. | P-CREDENTIALS |
| 4 | 09:31 | It posts the bundle to a paste site through a tool nobody listed. | P-TOOL-ALLOW, P-DATA-BOUNDARY |
Replay at Enforce (as played): P-FILES denies the key reads at 09:30; P-CREDENTIALS denies the prompt at 09:31; P-TOOL-ALLOW blocks the post at 09:31. P-PROVENANCE can't keep the skill at 1.4: it observes only. (The first draft said it could.)Training: Secrets in prompts (T-SECRETS-1), on the story's own P-CREDENTIALS finding.
S4. The public link (Mark's: "Accidental blunder send restricted data to location it becomes publicly viewable")
Logline: Jordan asks the agent to share the churn export with a colleague. The agent picks the quickest way: a public gist. Cast: Jordan · mac-eng-11 · Claude Code with the GitHub MCP server.
| Beat | Clock | Sensor | Should raise |
|---|---|---|---|
| 1 | 15:10 | "Share customers-churn.csv with Sam." | nothing |
| 2 | 15:10 | The agent reads the file: 4,812 rows with emails and phone numbers. | P-SENS-DETECT |
| 3 | 15:11 | github.create_gist with public: true and the rows. | P-TOOL-APPROVAL, P-DATA-BOUNDARY |
| 4 | 15:11 | The agent replies "Done, here's the link." | gap: no policy names a destination's visibility |
Replay at Enforce: P-TOOL-APPROVAL holds the call, and Jordan sees public: true before anything is sent.
Gap: whether a destination is public, for example a public gist, a link anyone can open, or a public bucket. Nobody meant harm here, which is why this is the one to show a sceptical audience.
S5. Card numbers to a stranger (Mark's: "Credit card data to suspicious source")
Logline: a support ticket includes a customer's full card number. User A's assistant sends the ticket to a cheap "refund helper" model nobody approved. Cast: User A · mac-a1 · Claude Code, with a "refund helper" model set up last week.
| Beat | Clock | Sensor | Should raise |
|---|---|---|---|
| 1 | 11:05 | The ticket is pasted in, card number included. | nothing |
| 2 | 11:05 | The prompt goes to api.refundhelper.example, which isn't an approved provider. | P-PROVIDERS |
| 3 | 11:05 | The payload carries a 16-digit number that passes the Luhn check. | P-SENS-DETECT (card number) |
| 4 | 11:05 | Nothing masked it on the way out. | P-SENS-SANITIZE |
Replay at Enforce (as played): P-PROVIDERS blocks the provider, so the ticket never leaves. P-SENS-SANITIZE is already at Enforce for User A: the number leaves masked as •••• 4242, and the ticket still goes.
S6. The runaway loop (new: the agent goes wrong by itself, and no one attacks anything)
Logline: an unattended release agent hits a flaky test and retries all night. Cast: Ravi · mac-eng-21 · an unattended Claude Code release session.
| Beat | Clock | Sensor | Should raise |
|---|---|---|---|
| 1 | 22:10 | An unattended session starts. | nothing |
| 2 | 23:00 → 02:00 | The same tool call and the same failure every 40 seconds. Spend climbs. | P-SPEND (at the ceiling) |
| 3 | 03:15 | The session stops reporting for twenty minutes. | P-TELEMETRY |
| 4 | 07:30 | It's still going, at $3,800. | P-SPEND |
Replay at Enforce (as played): nothing stops it. P-SPEND observes only, so the ceiling raises a finding, not a stop. (The first draft said it stopped the session.) Beats 3 and 4 move on by themselves (C2's rare beat), the loop ticking over while nobody watches. Gap: a spend ceiling that can act. Why: not every disaster is an attack. This one meets the AI spend tour.
Wave 2, the backlog (answers "what else??")
Grouped by who does it. Each one names the policies it would raise and any gap it shows.
Outsiders
- The rug pull. An approved MCP server updates, and one tool's description now tells the model to forward what it reads. P-PROVENANCE, P-PROMPT-INSPECT, P-OUTPUT.
- The hallucinated package. The model invents a package name, and an attacker has registered it ("slopsquatting"). The agent installs it. P-OUTPUT, P-PROVENANCE, P-TOOL-ALLOW.
- The sleeper note. An injected instruction gets written into the agent's memory file and fires a week later, in an unrelated session. P-PROMPT-INSPECT, P-FILES on the agent's own config. Gap: writes to agent memory as their own resource.
- Plain-text keys at the café. An MCP server reached over
http://on public Wi-Fi sends a token in the clear. P-TRANSPORT, P-CREDENTIALS.
Insiders
- The departing employee. In their last week, someone bulk-pastes the customer list into a personal ChatGPT account. P-ACCT, P-ENTERPRISE, P-SENS-DETECT. This extends the
personal-accountdrill. - The cheaper model. A developer points their harness at an unapproved provider to save money, and the codebase goes with it. P-PROVIDERS, P-ENTERPRISE.
Accidents
- The confident cleanup. "Tidy up the old branches": the agent force-pushes main and deletes
release/2026.11. P-TOOL-APPROVAL, P-SANDBOX. This extends theunapproved-deletedrill. - The patient record. A clinic's notes go to a model under no business associate agreement. P-SENS-DETECT, P-DATA-BOUNDARY, P-PROVIDERS. Gap: a provider's agreements (BAA, DPA) as a property policies can read.
- The wrong region. EU customer data goes to a model endpoint outside the region it may go to. P-DATA-BOUNDARY. Gap: the endpoint's region.
The agent itself
- The root agent. An agent runs as admin "to fix permissions". P-NO-ROOT, P-SANDBOX.
- The ghost session. A session runs at 3 a.m. with telemetry off and nobody at the keyboard. P-TELEMETRY.
- The sub-agent with more power than its parent. An orchestrator spawns a helper with broader tools than it was given itself (OWASP Agentic, excessive agency). P-TOOL-ALLOW. Gap: privilege inheritance between agents.
- The CFO's voice. An agent with Slack and email tools sends a payment request signed as the CFO. P-TOOL-APPROVAL. Gap: the identity a message is sent as (c137's citizen side,
problem:agent-identity-scoped-access).
Gaps so far (catalog candidates for c126)
- A destination the server never declared: where an MCP server sends data against what its listing declared, and how new the domain is (S2; was "a destination's country", changed by C9)
- A destination's visibility: public gist, anyone-with-the-link, public bucket (S4)
- A provider's agreements: BAA, DPA (the patient record)
- An endpoint's region (the wrong region)
- Writes to agent memory as a resource (the sleeper note)
- Privilege inheritance between agents (the sub-agent)
- The identity a message is sent as (the CFO's voice)
- A spend ceiling that can act: stop or pause a session at its ceiling (S6)