c8. Agent Pivot — Execution Plan

Qpoint has pivoted into AI Agent Monitoring and Governance (internal code name "QControl" — the brand remains Qpoint everywhere user-facing). This plan reorients the market-analyst pipeline around that pivot: freeze and archive the phase-1 corpus, redefine what we extract from calls, resume importing under the new lens, and pay down the structural debt accumulated going from prototype to working app.

Phase 2 is use-case honing fed by many sources. Phase 1 sampled a wide market, so the app aggregated one-off prospect calls into market-level patterns. Phase 2 hones specific use cases, with build partners as the richest input stream — but not the only one: prospect calls, advisor conversations, and other feedback keep flowing in, and the tool's job is to make sense of all of it. Three consequences run through Workstreams B–D:

1. The conversation stays the atomic unit (unchanged from phase 1 — every source produces conversations, and they all go through one pipeline).
2. The use case is the organizing lens. Extraction shifts from surveying ("what pain points exist out there?") to hypothesis-testing ("which of our named use cases did this conversation validate, challenge, or reshape?"), and use cases — not sources, not partners — are the primary aggregation axis.
3. Relationship type is a dimension, not a structure. Build-partner vs prospect vs advisor is metadata on the company and an evidence-kind tag on each insight (usage-based / reaction-based / speculative), so all feedback lands in the same rollups with its weight visible — no partner-only silo. The same principle applies to observation types: market signal vs product feedback is a tag on the item, not a fork in the schema — any source can produce either (two not-yet-partner CISOs recently gave build-partner-grade product feedback; the design has to expect that).

Positioning sources: inbox/qcontrol-as-signal-updated-text.md (current text), briefs/endpoint-vs-network-objections.md and briefs/cam-dashboard-directives.md (already written in the new positioning).

Current state (snapshot, 2026-08-20)

• Corpus: 47 raw transcripts (+ paired .meta.json/.notes.json), 39 processed, 34 in the index. Date range 2025-01-03 → 2026-03-31. Top companies: Cisco (9), Unknown (5). data/reviewed/ has never been used. analysis.json has never been generated.
• Extraction prompt (pipeline/prompts/extract-call.md, 310 lines): framing paragraph and all 8 controlled vocabularies are phase-1 ("API infrastructure… traffic management, observability, and security"): traffic-visibility, api-observability, protocol-support, etc. A few agent-era categories already snuck in (ai-governance, ai-monitoring).
• Structural debt (from the c8 architecture survey):
  • Every vocabulary lives in three places: prompt markdown, schemas/call.ts union types, schemas/validation.ts runtime arrays. Drift already exists (pain points tagged with categories from other vocabularies — warnings only).
  • site/composables/useCallData.ts re-declares the entire schema as local interfaces (lines 3–181) because site/ is a separate npm project.
  • Prompt assembly (metadata/notes injection) is copy-pasted between extract.ts and batch.ts; parseSagetapSubject duplicated between avoma.ts and upload-sagetap.ts.
  • Claude output parsed via regex fenced-code-block + JSON.parse — no structured output, no retries. Model claude-sonnet-4-20250514 hardcoded in 3 files.
  • batch.ts skips the internal/external classification filter that extract.ts applies.
  • analyze.ts stringifies the entire corpus into one prompt — fine at 34 calls, won't scale.
  • Site: hardcoded label/color maps (company size, interest, urgency, funnel, commitment), one hand-built section per schema block in calls/[id].vue, FilterBar.clearFilters() misses dateFrom/dateTo.

Sequencing

Four workstreams (A–D), roughly in order, with D split so cheap fixes ride along with B/C and the deep review lands once phase-2 usage is real:

A. Archive phase 1          (freeze + snapshot + reset)
B. Redefine analysis areas  (new vocabularies + prompt + schema, single-sourced)
C. Import phase 2           (pilot 2–3 calls → tune → bulk)
D. Architecture             (D1 quick fixes folded into B/C · D2 deep review after C)

Rationale: archiving first gives a clean floor. The vocabulary redefinition (B) forces us to touch prompt + types + validator anyway, so the single-source-of-truth refactor (D1) is cheapest done inside B, not after. Importing (C) pilots on a few calls before bulk so prompt tuning is cheap. The flat-files-vs-DB question (D2) is best answered with phase-2 volume data in hand, not speculated in advance.

Workstream A — Archive phase 1

Goal: phase-1 work stays browseable, but out of the working set. New imports start from a clean slate.

1. Capture the phase-1 cross-call analysis before freezing. Run npm run analyze once — analysis.json has never been generated, and this is the last chance to produce it against the phase-1 prompt/corpus while everything still lines up.
2. Freeze a browseable snapshot. npm run generate a static build of the site against the full phase-1 dataset → store at archive/phase-1/site/. This is the key decoupling move: the archive stays browseable forever without requiring the new site to remain compatible with the old schema.
3. Move the data. data/raw/, data/processed/, data/raw-sagetap/, all-calls.json, index.json, analysis.json, analysis/, call-classifications.json, batch-status.json → archive/phase-1/data/. Keep companies.json and known-entities.json in place (they carry forward — company knowledge doesn't expire with the pivot).
4. Write archive/phase-1/README.md: what phase 1 was (API observability positioning), date range, call count, headline findings (~70% interest 4–5, top themes), how to browse the snapshot, and a pointer to the git tag.
5. Git tag phase-1-final at the commit containing the complete snapshot.
6. Reset sync state. Rewrite data/avoma-sync.json (or delete it) so the next npm run fetch starts from the pivot cutoff date, not 90-days-back or the stale watermark.
7. Link the archive from the new dashboard (small "Phase 1 archive" link in the footer/about) so it stays one click away.

Open question A1 — the cut line. The index runs through 2026-03-31 and the Cisco briefs are already in agent positioning, so the pivot conversation started inside the existing corpus. Options: (a) archive all 34 and re-import/re-extract any agent-era calls under the new schema (recommended — clean, and re-extraction is ~$0.04/call), or (b) pick a date and split. Needs Mark's call on the actual cutoff date.

Workstream B — Redefine the analysis areas

Goal: partners and use cases are first-class entities, the extraction prompt asks the questions that matter for the agent-security business, and every vocabulary is defined exactly once.

B1. Use-case registry + relationship metadata (the new backbone)

One new registry, one extension of an existing file:

• data/use-cases.json (new) — the named use cases being honed: id, name, one-line hypothesis, status (candidate / validating / validated / parked). This is deliberately a registry we maintain, not an emergent taxonomy — phase 1's job was discovering categories from the market; phase 2's job is testing named hypotheses against everything we hear. Extraction tags evidence against these ids, so adding/renaming a use case is a registry edit, not a schema change.
• SourceType — a controlled vocabulary for who feedback comes from. First draft: build-partner / prospect / ciso-advisory / investor / analyst / industry-peer / internal. This becomes one of the single-sourced vocabularies in B3, so it's defined once and flows to prompt, validator, and site. Two tagging levels:
  • Company level: data/companies.json (extended) gains sourceType, engagement start date, key people, and attendee-email domains / subject patterns (feeds the import allowlist in C). No parallel "partners" entity — a build partner is a company with sourceType: build-partner, and the existing name→override machinery in build-index.ts already applies it to every call.
  • Person level (override): some sources are people, not companies — an advisory-board CISO's day job might be at a company that's otherwise a prospect, and AMZ is already the internal precedent. known-entities.json grows a sources section (name patterns → sourceType) that overrides the company default per attendee, so their insights carry the right tag.

Schema additions: CallData gains sourceType (resolved at index-build time from company + person tags) and useCaseEvidence[] — useCaseId, direction (validates / challenges / expands / new-variant), evidenceKind (usage-based / reaction-based / speculative), evidence text, verbatim, speaker. Source type says who's talking (a build partner, a VC); evidence kind says what the evidence is (they ran it vs they reacted to a pitch) — the pair is what lets one rollup hold a build partner's "this broke in our rollout" next to an advisory CISO's "SOCs will never accept that" next to an investor's market read, each with its weight visible. All fields optional-safe for conversations that don't touch any named use case, same pattern as urgency/positioning in phase 1.

B2. Redefine the insight areas (workshop, not code)

Draft a analysis-areas.md in this cycle dir proposing the new vocabularies, sourced from the positioning docs, then review with Mark before touching code. What stays, what goes, what's new:

• Keep as-is (positioning-neutral): interest level 1–5 (still useful for non-partner calls), urgency block (budget/timeline/commitment), decision-making block, value echoes, key quotes, company segmentation, personas (security persona likely becomes the center of gravity — consider splitting security into ciso/security-leader vs security-engineer).
• Widen call types and stages, don't replace them: the prospect funnel still applies to prospect conversations; build-partner relationships need an engagement stage of their own (evaluating / building / deployed / expanding / parked) — carried on the company record, not per-call. Call types gain the working modes: working-session, build-feedback, deployment-debug, roadmap-planning, advisor-input, alongside the existing discovery/demo.
• One unified observations[] stream — types route to workflows. Phase 1 stored each insight area as its own array (painPoints[], objections[], valueSignals[]…), which is exactly why bugs and concerns had no home. Phase 2 collapses them into a single stream: every extracted item is an observation with an ObservationType — the market-signal types (pain-point, objection, value-signal, competitive-mention, budget-signal, risk…) and the product-feedback types (bug, feature-request (they need it), feature-idea (unprompted suggestion), concern (worry about approach or direction — the advisory staple), usability-friction, deployment-blocker, praise) — plus per-type category, verbatim, speaker, severity where relevant, and a light product-area tag for feedback-ish types (agent-observation, policy-intervention, heartbeat-attestation, inventory, dashboard, install-deploy, integrations). Each type declares in vocabularies.ts which workflow(s) it feeds — analyze, triage, or both — so "market signal" and "product feedback" are views over slices of one stream, not schema structures. Adding a type later is a vocabulary edit, not schema surgery.
• Types are orthogonal to sources — deliberately. The grounding example: two CISOs who aren't build partners yet recently gave product feedback indistinguishable from build-partner input. Any source can emit any observation type; sourceType + evidenceKind carry the weighting (a partner's usage-based bug vs an advisor's reaction-based concern), and dual-natured items — a bug that's also use-case evidence, a concern that's also an objection — carry both tags instead of being misfiled into one stream. The extraction prompt can still ask in focused sections (pain points… product feedback…) for attention's sake; the output lands in one array.
• Lifecycle in data/feedback.json: observations whose type feeds the triage workflow roll up into a tracked list with status (new / acknowledged / committed / shipped / validated-in-use / declined) and cross-conversation dedup — "second source to hit this bug" is a priority signal the per-call view can't see. The loop "they asked in March → we shipped in May → did it land in the June call?" closes here. Not a replacement for a real issue tracker: when a bug graduates to GitHub, link out — this tool tracks who said what, when, and whether we closed the loop, not engineering workflow.
• Redefine (the 8 vocabularies): first-draft directions, to be workshopped —
  • Pain points: agent-visibility-gap, shadow-ai-usage, block-vs-allow-binary, agent-audit-compliance, prompt-injection-exposure, mcp-sprawl, edr-network-blindspot, credential-agent-access, developer-friction, incident-response…
  • Objections: edr-overlap ("doesn't CrowdStrike see this?"), gateway-overlap, another-endpoint-agent (agent fatigue), performance-impact, trust-kernel-access, wait-for-platform-vendor, positioning-clarity, adoption-readiness…
  • Positioning angles (which framings land — mirror the doc's own arguments): endpoint-line-of-sight, signal-vs-enforcement-handoff, heartbeat-attestation, corrupted-agent-resilience, continuity-not-restriction, gateway-complement, inventory-audit-trail, defense-in-depth-fit…
  • Competitive mentions: structure the field — EDR vendors, LLM/MCP gateways, network/SASE, ASPM/ADR category players, DIY/scripts.
  • Value signals, feature requests, budget signals, urgency drivers, risks: same exercise; e.g. urgency drivers gain agent-incident-triggered, board-ai-mandate, audit-compliance.
• New areas to consider extracting: the partner's current agent posture (which agentic tools are in use / sanctioned / banned), security-org maturity (SOC? SIEM? EDR vendor?), and deployment context (managed endpoints, MDM, OS mix) — these are the segmentation axes that matter now, the way company-size/industry did in phase 1.
• Trajectory context in extraction: for any company we talk to repeatedly (build partners chiefly, but also multi-call prospects — Cisco had 9 calls in phase 1), the valuable signal is the delta — an objection raised in call 3 and resolved by call 6, commitment language deepening or stalling. Extraction stays per-call, but the prompt should receive light relationship context (company, relationship type, target use cases, optionally the previous call's summary) so it can flag continuity explicitly ("previously raised X, now resolved"). The heavier longitudinal synthesis belongs to analyze.ts (D2.3).
• Rewrite the framing paragraph in the prompt: "Qpoint, a seed-stage company building endpoint security for agentic AI…" — brand is Qpoint; "QControl" never appears in prompts, extracted data, or site copy.

B3. Single-source the vocabularies (D1 fix, done here)

• Create schemas/vocabularies.ts: one definition per vocabulary (id, label, description) as as const arrays.
• schemas/call.ts derives union types from it; schemas/validation.ts imports the arrays instead of re-declaring them.
• A small generator (pipeline/build-prompt.ts or a template step in prompt assembly) renders the vocabulary section of extract-call.md from the same source — the prompt's category lists can no longer drift from the validator.
• Scrub remaining phase-1 copy: suggested-themes list, eBPF/CrowdStrike examples in category definitions, batch-themes.md framing, about.vue example categories, goals.md, .claude/skills/analyze/SKILL.md schema restatement.

B4. Update goals.md

The /analyze skill's strategic lens. Rewrite the PMF questions for the new thesis and the multi-source mode: which named use cases are gaining evidence and which are stalling — and does the answer differ by source type (partners love it, advisory CISOs skeptical = a deployment story, not a demand story); does the visibility-gap framing resonate; is the buyer the CISO or the platform team; standalone endpoint value vs signal-source value — which lands first; does heartbeat/absence-as-signal survive contact with real SOC teams; what does the EDR-overlap objection sound like when it's fatal vs when it's curiosity; are asks converging (one product) or diverging (five custom builds)?

Deliverables: data/use-cases.json (seeded with Mark), extended companies.json + known-entities.json source tagging, data/feedback.json (structure defined, starts empty), analysis-areas.md (reviewed), rewritten extract-call.md, schemas/vocabularies.ts + derived types/validator, updated goals.md and skill docs.

Workstream C — Import phase-2 conversations

Goal: new calls flowing through the new prompt, validated before bulk.

1. Known-source allowlist in import — filters become taggers. Phase 1's --external filters were built to discard everything that isn't a prospect call: recurring syncs pattern-match the internal regexes, and investor/vendor calls are explicitly dropped. Phase 2 inverts that for known sources: avoma.ts checks the source registry first (companies.json domains/patterns + known-entities person tags) and always imports a match, tagged with its sourceType — a build-partner working session and a CISO-advisory conversation both come in, and an investor call becomes a tagged investor source instead of a dropped one. The regex filters remain as fallback for truly internal noise (standups, 1:1s). Sagetap likely idles to zero — leave the machinery in place, untouched.
2. Pilot: npm run fetch -- --external --from <cutoff> then extract 2–3 known-good calls from different source types (a build-partner session, a prospect call, an advisory conversation if available) sequentially (npm run extract) and read the output closely — especially the useCaseEvidence tagging with direction and evidenceKind, which is the new heart of extraction. Tune categories/definitions where the model reaches for the wrong bucket. This is where B proves out — expect one or two prompt iterations.
3. Model bump: while touching the extraction path, lift the hardcoded claude-sonnet-4-20250514 into one shared constant (or .env) and evaluate claude-sonnet-5 on the pilot calls. Watch max_tokens: 4096 — the prompt is growing again; check for truncation.
4. Bulk: batch-extract the remainder (npm run extract:batch, ~$0.04/call, ~2h wall clock), npm run index, npm run analyze. Note: with phase-2 volume (fewer, richer conversations) sequential extraction may be the everyday path and batch reserved for backfills.
5. Fix the batch gap: make batch.ts honor call-classifications.json the way extract.ts and build-index.ts do, and share the prompt-assembly code between extract.ts/batch.ts (extract to a module — second D1 fix riding along).
6. Site pass — use-case-centric views over all sources. The phase-1 aggregates (interest distribution, funnel breakdown, segment cards) fade in relevance; the views that matter now:
   • Use-case evidence rollup — the new homepage: per use case, accumulated evidence across all sources, faceted by direction (validates/challenges), evidenceKind, and sourceType. Answers "are we honing in on the right thing" and shows who's saying so — partners validating while advisory CISOs push back is itself a finding.
   • Company/source journey page — extends companies/[name].vue: timeline of conversations with evolving signals, open feedback items (bugs, requests, concerns and their status), and use-case evidence accumulating over time. Works identically for a build partner, a multi-call prospect, or a recurring advisor — the sourceType badge is the only difference.
   • Feedback triage view — the triage slice of observations[] across all conversations, grouped by type (bug / request / idea / concern) and product area, with status and "reported by N sources" dedup counts. This is the product-shape inbox the team reviews; distinct from the use-case rollup, which is the thesis view. Same data, different slice — both views read the one stream.
   • Source-type filter everywhere — sourceType joins the FilterBar so any existing view can be cut by build-partner / prospect / advisory / investor. Plus the mechanical pass: updated label/color maps for changed vocabularies, source-type pills (extending the existing AV/ST source-pill pattern), about.vue methodology copy, the clearFilters() date bug. segments.vue's market-wide framing is a phase-1 artifact — retire or simplify rather than rework.

Open question C1 — cadence. One-shot backfill then manual npm run fetch runs, or set up a recurring scheduled import? Recommend manual until volume justifies automation.

Workstream D — Architecture review

D1 — Quick fixes folded into B/C (listed above, summarized)

Vocabulary single-sourcing (B2) · shared prompt-assembly module + batch classification filter (C4) · model constant + current model (C2) · site label-map updates and clearFilters fix (C5).

D2 — Deep review (its own thread, after C lands)

Run as a dedicated thread (architecture-prompt.md → architecture-plan.md per cycle conventions) once phase-2 data is flowing, so decisions are grounded in real usage. Scope:

1. Storage: flat JSON vs DB. Current position to pressure-test, not a foregone conclusion: flat git-tracked JSON has real virtues at this scale (diffable, reviewable, zero infra, the site reads it directly) — and phase 2 strengthens it: fewer conversations, and the use-case/feedback/source registries are exactly the small hand-curated files this pattern handles well (companies.json is the template). Evaluation criteria: corpus size trajectory, cross-call query needs, multi-user write needs, all-calls.json payload size in the browser (388 KB — consider index + lazy per-call fetch before reaching for a DB). If a DB is warranted, SQLite (still git-friendly via dumps, no server) before anything heavier.
2. Structured extraction. Replace regex-fence JSON parsing with the API's structured output / tool-use so schema conformance is enforced at the API layer; add retry/backoff around Anthropic calls (Avoma client already has this — mirror the pattern).
3. analyze.ts reshape — use-case- and source-oriented, not just scaling. Whole-corpus-in-one-prompt is exactly wrong for phase 2: the analysis that matters is per-use-case across all sources (evidence rollup, with agreement/disagreement between source types surfaced explicitly) and within-company longitudinal (trajectory across a relationship's conversations). Reshape to map-reduce: per-use-case synthesis + per-company synthesis → overall picture. This graduates from "before it's needed" to an early phase-2 need.
4. Schema sharing with the site. Kill the 180-line duplicate in useCallData.ts — options: npm workspace with a shared schemas package, or a build step that copies types into site/. Also generate site label maps from vocabularies.ts so D1's single-sourcing extends to the UI.
5. Site structure. calls/[id].vue (408 lines, one hand-built section per schema block) → schema-driven section rendering, so future analysis-area changes don't require surgery there. The unified observations[] stream (B2) makes this nearly free: detail-page sections become groupings of observation types driven by vocabularies.ts.
6. Pipeline ergonomics. Config layer for paths/model/flags instead of per-file constants; remove dead code (nanoid import); decide fate of one-off scripts (backfill-speakers.ts).

Deliverable: architecture decision doc with a recommendation per item, then implementation in this cycle or a follow-on cycle depending on size.

Open questions for Mark

1. (A1) Pivot cut line — archive all 34 calls and re-import agent-era ones, or split at a date? Which date?
2. (B1) Seed the registries — the named candidate use cases, plus the source map: which companies/people are build partners, who's on the CISO advisory board, which investors' input counts as feedback. Is the draft SourceType vocabulary (build-partner / prospect / ciso-advisory / investor / analyst / industry-peer / internal) the right cut? These lists are the backbone of everything in B/C; 30 minutes with Tyler/Cam probably settles v1.
3. (B2) Analysis areas — the vocabulary drafts above need a working session; the positioning doc gives strong candidates but the objection/angle taxonomies should match what Tyler/Cam are actually hearing.
4. Feedback lifecycle depth — is extract-and-triage enough for the triage rollup, or do we want the full lifecycle (new → committed → shipped → validated-in-use) actively maintained in feedback.json? The latter edges toward lightweight project tracking and needs someone to own status updates — worth deciding how far to go, and where the handoff to a real issue tracker sits.
5. (C1) Import cadence — manual runs or scheduled?
6. Archive hosting — is a static snapshot under archive/phase-1/site/ (opened locally or served alongside) sufficient, or should the new dashboard host it at a route?

Suggested first session

Kick off formally (CYCLE.md via cycle-manager), decide A1, seed use-cases.json and the source map (question 2 — it's the backbone), then execute Workstream A end-to-end — it's mechanical, low-risk, and everything else builds on the clean slate.