← Bob-Wire · c16

analysis.md

c16 — Phase 1 Analysis: Stakeholder & Persona Evaluation of the qcontrol Control Plane

Source under analysis: the qcontrol control plane at http://localhost:7878/ — a self-contained agent-native dashboard (the new wf1). Twelve views across four nav groups (Assurance / Security / Identity & Access / Admin). Method: read the full app source (HTML + JS + the /api/* JSON shapes). Scores below are grounded in what each view actually renders, not its title.


1. Framing

c15 evaluated a traffic-observability product and its egress model leaked everywhere. qcontrol is the opposite starting point: it is already agent-native. Its unit of analysis is the agent as an autonomous actor — what it is (identity, runtime, credential), what it can do (tools, MCP), what it did (actions, data access), what it cost (tokens, spend), and how safe it is (posture score, findings mapped to AI frameworks). So the question is no longer "is this an agent product" (it is) but:

How well does each view serve the three mandate-owners — Security, Finance, Executive — for governing AI agents, and where are the gaps that justify net-new surfaces?

Three findings frame everything below:

  1. Security is the center of gravity and well-served. Findings, Posture, Identity, DLP, and the Inventory profile form a complete CISO loop (inventory → posture → finding → remediate → credential). The richest, most mature half of the app.
  2. Finance is now served — a reversal from c15. Cost Center (spend by model/agent/user/ provider, cache %, period, trend) plus Teams & Users (per-person showback) close the gap c15 called the product's biggest. What's still missing is forward-looking FinOps: budgets, forecast, spend ceilings, and true chargeback (showback → P&L needs an IdP join).
  3. Executive still has no home. The board-relevant signals (governed-agent %, posture index, $ spend & $-at-risk, open high findings, coverage) exist but are scattered across operator views; nothing is packaged for a board. This is the single clearest net-new opportunity — the same conclusion c15 reached, now grounded on a cleaner app.

Residual egress to demote (decision: keep as a detail-level signal only). qcontrol is mostly clean but still leans on egress in two spots: (a) 2 of 3 live Findings are "Egress to non-allowlisted destination," and (b) Policy & Enforcement leads its dry-run with egress allow-list block counts. In c16 these reframe to behavior — unauthorized actions, excessive capability, credential risk, sensitive-data reads, unbounded consumption — with egress demoted to the one Network Egress card on the agent profile.


2. Scoring rubric

Per stakeholder, per view, 0–5. Primary columns S = Security, F = Finance, E = Executive; secondary Eng = Engineering/Platform, C = Compliance/Privacy.

ScoreMeaning
0No value to this stakeholder for agent governance
1–2Tangential / enabling plumbing
3Useful, but needs reframing or packaging to land
4Strong fit
5Core daily value for governing agents

3. Per-view evaluation

Assurance

ViewSFEEngCPurpose & note
Fleet Overview43342At-a-glance fleet health + triage. KPI cards, deep-linked attention strip, token/posture/behavioral donuts, activity area, top-risks, fleet table. Operator-framed, not board-packaged → seeds the net-new exec board. Low-hanging: add MoM deltas to the KPI cards.
Agent Inventory (+ profile)53243The canonical agent registry + per-agent forensic profile (identity, runtime/process, data access, MCP, token/cost breakdown, posture + factors). The spine of the app. Opportunity: a governed/ungoverned filter, an owner rollup, bulk policy actions.
Cost Center15421The Finance surface. Spend by model/agent/user/provider, cache %, period toggle, honest estimated-cost labeling, trend. Closes c15's biggest gap. High-hanging: budgets, forecast, per-agent spend ceilings, chargeback to P&L.
Performance12150LLM latency percentiles (p50/p95/p99), slowest responses, out-tok/s. The builder's tuning surface. Gap: it measures call latency, not task outcome — agent reliability should also be success/quality (see §6).

Security

ViewSFEEngCPurpose & note
Findings51334CISO worklist; findings auto-mapped to OWASP LLM Top 10 / MITRE ATLAS / NIST AI RMF; severity + framework filters; click → agent. Core daily value. Reframe: the live list still leans on egress-destination findings → recast to behavioral findings (excessive agency, root, credential, sensitive-read, unbounded consumption).
Security Posture51443Fleet score gauge (100 − risk-factor deductions), factor breakdown, current-vs-worst regression, shadow-AI at-rest installs (instrumented vs unmonitored). Board-legible. High-hanging: posture trend over time, per-team posture.
Data Protection (DLP)41225Sensitive paths touched by category/user, byte volume; prompt/response content inspection is an honest preview. Compliance-core. Opportunity: content scanning (secrets/PII/source) with graduated enforcement.
Policy & Enforcement42353The control plane: Audit → Assist → Enforce, plus a "what an enforcing policy would have blocked" dry-run computed from audit data — a strong value demo. Reframe: lead with tool/MCP/action + spend-ceiling + HITL gates; demote the egress allow-list to one card.
Compliance31315Runtime findings auto-tagged to control frameworks (incl. OWASP Agentic), evidence export (JSON/CSV), projected ISO 42001 / EU AI Act / SOC 2, forensic why-trail. Auditor-core, honest about projected-vs-live. Opportunity: server-side evidence pack, scheduled reports.

Identity & Access

ViewSFEEngCPurpose & note
Teams & Users44323The "who" — per-person showback + sensitive-data access + risk, ranked by risk; honest observed-not-verified identity, OS-user join. Bridges Finance ↔ Security. High-hanging: chargeback to P&L + Team/Dept rollups (need IdP/SCIM — projected).
Agent Identity51233Machine-credential governance (NHI): credential class + risk (static key vs revocable OAuth, consumer vs org-backed), bound org/account. Honest observed, not issued/verified. Opportunity: IdP join, JIT credentials, rotation (all projected).

Admin

ViewSFEEngCPurpose & note
Integrations31243Live qcontrol event stream + findings export (work today); projected SIEM / GRC / identity / alerting connectors, each grounded in the real local volume it would carry. Honest "local-only" posture. Opportunity: wire the connectors; scheduled SIEM/GRC push.

4. Score summary

  • Top Security surfaces (the CISO loop): Agent Inventory/profile (5), Findings (5), Security Posture (5), Agent Identity (5), then DLP / Policy / Teams / Fleet Overview (4).
  • Finance — now served, but reactive: Cost Center (5) and Teams & Users (4) anchor it; Fleet Overview (3) for top-line. The remaining gap is forward-looking (budgets/forecast/ceilings/ chargeback), not visibility.
  • Executive — partially served, no home: strongest borrowed signals are Cost Center (4) and Security Posture (4); everything else is 2–3 and operator-shaped. No board-packaged surface.
  • Engineering/Platform — well served: Performance (5), Policy & Enforcement (5), Integrations (4), Agent Inventory runtime detail (4), Posture factors (4). The build/operate persona.
  • Compliance/Privacy — well served: Compliance (5), DLP (5), Findings→frameworks (4).

5. Per-persona evaluation (the 5 seats carry over from c15)

Security / CISO — core audience, well served

  • Views: Findings, Security Posture, Agent Identity, Data Protection, Policy & Enforcement, Agent Inventory (+profile), Teams (risk).
  • Served: Strongly. A complete loop: inventory → posture → finding → remediate → credential.
  • Gaps: Findings still partly egress-shaped; no capability/action authorization matrix ("what each agent may do"); no instruction/prompt-change governance; posture has no trend.
  • Opportunities: Low — reframe egress findings to behavioral; add a posture trend line. High — a Capability & Action Authorization surface; Instruction/Prompt governance.

Finance / FinOps — now served (reversal from c15), but reactive

  • Views: Cost Center, Teams & Users (showback); Fleet Overview spend cards.
  • Served: Well, for visibility. Spend by model/agent/user/provider with honest estimated-cost labeling is exactly the missing c15 dimension, delivered.
  • Gaps: No budget/forecast/spend-ceiling; showback ≠ chargeback (no P&L/Team rollup without an IdP join); no per-agent cost-efficiency ($/successful-task).
  • Opportunities: Low — Team/Dept tag dimension on Cost Center. High — Budget, Forecast & Spend Ceilings surface + ceiling-alert policy; chargeback rollups post-IdP.

Executive — signals exist, no board home

  • Views: borrows Fleet Overview, Posture score, Findings high-count, Cost trend.
  • Served: Partially. The ingredients are present but scattered across operator views; nothing is packaged, trended, or exportable for a board.
  • Gaps: No single posture/ROI surface; no governance-index trend; no "$ at risk"; no export.
  • Opportunities: Low — a posture scorecard tile on Fleet Overview. High — a net-new Executive Governance & ROI Board + board-ready export. The clearest net-new in the app.

Engineering / Platform — well served (build & operate)

  • Views: Performance, Policy & Enforcement, Integrations, Agent Inventory (runtime), Posture factors, Installations (shadow-AI).
  • Served: Strongly. They own the enforcement config, the latency tuning, and the connectors.
  • Gaps: Reliability is latency-only — no task-success/quality signal; no agent topology/ delegation view for multi-agent systems.
  • Opportunities: Low — correlate latency × cost on Performance. High — Task Outcomes & Quality; Agent Topology / Delegation.

Compliance / Privacy — well served

  • Views: Compliance, Data Protection, Findings→frameworks, Teams (attribution), evidence export.
  • Served: Strongly, and honestly (projected vs. live is labeled).
  • Gaps: Content inspection still preview; no scheduled evidence pack; no retention/residency view for agent memory & records.
  • Opportunities: Low — schedule the findings export. High — content DLP; an Agent Memory & Record retention surface.

6. Additional personas to consider

The prompt names Security / Finance / Executive and asks "others?". The 5-seat model from c15 holds against qcontrol's surfaces. Reflections:

PersonaServed byVerdict
Engineering / PlatformPerformance, Policy, Integrations, Inventory runtimeKeep as a seat — primary builder/operator.
Compliance / PrivacyCompliance, DLP, Findings→frameworksKeep as a seat — distinct mandate, increasingly central to AI governance.
Procurement / AI-vendor mgmtCost Center by_providerFold under Finance (DPA/sanctioned-vendor is a finance sub-mandate).
Incident responder / SecOps on-callFindings, Policy kill-switchFold under Security unless an on-call function is distinct.
Agent owner / developer (the human behind an agent)Teams & Users, agent profileWorth noting — qcontrol foregrounds attribution to a human; an "agent owner self-service" view is a possible future seat, not needed now.

Recommendation: keep the five seats (Security / Finance / Executive / Engineering / Compliance) as the persona partition for Phase 2.


7. Net-new surfaces (the prompt's headline question)

qcontrol answers "yes — there are exec/finance surfaces that don't exist." Synthesizing §3–§5, the missing surfaces, in priority order:

  1. Executive Governance & ROI Board (Executive, Security) — one board view: governed-agent count & %, fleet posture index trended over time, total $ spend & $-at-risk, open high-severity findings, instrumentation coverage %, shadow-AI count — exportable. Packages the scattered operator signals for a board. The lead net-new page.
  2. Budget, Forecast & Spend Ceilings (Finance, Executive) — per-agent/per-team budgets, burn-down, forecast, ceiling-breach alerts, chargeback rollup. Turns Cost Center from reactive visibility into forward-looking FinOps.
  3. Task Outcomes & Quality (Engineering, Executive, Security) — qcontrol measures latency + tokens + posture but never whether the agent's task succeeded. Success rate, goal completion, output quality, and human-intervention rate are first-class agent-governance signals that no view covers. The biggest conceptual gap.

Agent-native gaps worth noting (Phase 2 may fold into existing pages rather than new ones):

  • Capability & Action Authorization matrix — "what each agent may do" (use tool X, write prod DB, call sub-agents). Policy & Enforcement hints at it (tool/MCP allow-list, projected); promote it to a first-class authorization surface.
  • Instruction / Prompt governance — system-prompt inventory + change control. Absent.
  • Agent Topology / Delegation — who delegates to whom in multi-agent systems. Absent.

8. Page-set recommendation for Phase 2

A persona-partitioned lo-fi app (the approach the requester likes), with deep pages that are agent-native reframes of qcontrol's 12 views plus the net-new surfaces above:

Reframes of qcontrol views: Fleet Overview, Agent Inventory (+ profile), Cost Center, Showback (Teams & Users), Performance, Findings, Security Posture, Data Protection, Policy & Enforcement, Compliance, Agent Identity, Integrations.

Net-new: Executive Governance & ROI Board (lead), and — scope permitting — Budget & Spend Ceilings and Task Outcomes & Quality.

Phase 3 mockup lead set: Executive Board, Cost Center, Agent Inventory (+profile) — covers all three primary stakeholders and tells the clearest story; Findings & Security Posture as follow-ons.


Appendix — cheap, honest wins already implied by the app

  • Reframe the 2 egress-destination Findings to behavioral findings (the c16 data model already does this — see app/data/c16-fleet.ts findings).
  • Add MoM deltas to Fleet Overview KPI cards (data is present in the cost series).
  • Add a governed/ungoverned filter to Agent Inventory (the governed flag exists in the model).
  • Surface a posture-trend sparkline on Security Posture (history is "projected" today).
  • Promote the shadow-AI count (installed-but-unmonitored) to a headline coverage KPI.

Qpoint Brand Style Guide