c15 — Phase 1 Analysis: Agent-Governance Evaluation of the wf1 App

Source under analysis: ../app.qpoint.io/app/pages/** (the wf1 working wireframe, runs at http://localhost:7878/). Method: Six parallel agents read the actual page + component source for ~45 pages and scored each through the agent-governance lens. Scores below are grounded in what the files render, not in the page titles.

1. Framing

Qpoint is pivoting from API/traffic observability to AI-agent governance. The wf1 app was built for the former: every analytics surface pivots on Vendor / Client / Endpoint, never on an agent / model / identity. The governance question is not "is this a good observability app" (it is) but "how well does each page help the three mandate-owners — Security, Finance, Executive — govern AI agents: their egress, their spend, and their risk."

Scoring rubric (per stakeholder, per page):

Three findings hold across the entire app and should frame everything below:

1. Security is structurally well-served; the gap is vocabulary. The richest pages already capture identity-adjacent provenance — requests (auth token type, masked token, sensitive-data scan), connections (sourceExe process → destination, allow/ block action, country), data-risk (producer→consumer sensitive-data flows), and the mcp/views/proxy settings (agent registry + data scope + default allow/deny). They speak "traffic / PII / domains," not "agents / LLMs / egress policy."
2. Finance is essentially unserved. Almost every page scores 0–2 for Finance. The strongest cost signal anywhere is bytes in/out — raw volume, never dollars or tokens. There is no spend, budget, or cost-attribution dimension in the product.
3. Executive value is thin and scattered. It concentrates in security/overview's health dials and deploy/preferences' capability+residency+SOC2 summary. No single board-level posture/ROI surface exists — it would have to aggregate across pages.

The single highest-leverage structural fix: add an agent / model / identity dimension alongside Vendor/Client/Endpoint (the shared traffic/components/usage/ GroupBy.vue pivot). Every page's scores rise once "by agent" is a pivot, because spend, risk, and posture in this pivot are all owned per agent.

2. Per-Page Evaluation

Scores are S = Security, F = Finance, E = Executive (0–5).

Onboarding & Dashboard

• Dashboard low-hanging: re-label/re-group widgets for the pivot; promote "Configure MCP Server" to first-class; surface AI/LLM vendors as a distinct inventory slice.
• Dashboard high-hanging: replace the metric strip with three stakeholder lanes — Agent Egress (which agents call which external LLM/tool endpoints), Spend (token/ $ per agent per vendor — the missing Finance signal), Posture (violations, unsanctioned agent traffic) for the executive.
• Configure wizard: fill the stubs as a Governance Policy setup (allowed agent capabilities, egress allowlists, spend ceilings, data residency/retention).

Traffic

• Lead with requests + connections — they already hold identity-adjacent provenance. Promote authTokenType and the sensitive-data flag to first-class columns/ filters; map sourceExe to a known-agent registry; surface a policy verdict (allowed/ blocked/flagged) inline.
• Reframe bandwidth → spend: translate request/response bytes at LLM/API destinations into estimated tokens and $ per agent and per provider, with MoM cost trend and budget burn. This is the single highest-leverage move for Finance and Executive.

Inventory

• Cheapest wins in the whole app: two columns are already coded but commented out — re-enable "Sensitive Data" (clients/endpoints/vendors) and "Last Used" (tokens).
• High-hanging: classify each client as human vs. agent/service (owner, model/runtime, risk score); evolve vendors into an AI/LLM-vendor spend & risk inventory (token/$ cost, DPA/compliance status, shadow-AI flagging); bind each token hash to an owning agent identity with scope/over-permission detection and one-click revoke.

Security & Alerting

• data-risk + the rule editor's Risk/Discover-PII kinds are ~80% of an agent-egress governance story already — they just say "traffic/PII," not "agents/LLMs." Highest- leverage, lowest-cost pivot in the set.
• High-hanging: a dedicated Sensitive-Data → LLM flow view (consumers filtered to model endpoints, agent→model sankey, allowed/blocked overlay); an Agent-Behavior policy kind (rules over agent identity × destination model × sensitive-data class × spend ceiling); closed-loop enforcement actions (quarantine agent, revoke token, throttle egress) instead of notify-only webhooks.

Stacks / Plugins (the enforcement control plane)

• routing (binding) and adapters (enforcement) are the strongest control-plane surfaces but speak pure middleware. Inject an agent/identity dimension into routing (which agents may reach which LLM/MCP endpoints, default-deny, per-destination spend ceilings), add allowed/blocked/spend telemetry to adapters, and a coverage/posture rollup to the index. Surface uncovered egress (domains routed to "(none)").

Org Settings (configuration & access control)

• Three pages already are agent governance and should anchor the pivot: MCP Clients (identity + credential + scope), Dataset Views (the scope primitive), and Proxy (default allow/deny egress). Together = the identity + scope + egress triad, but thin.
• Biggest latent capability already in code: mcp/manage.vue binds a client's contextId to a dataset view. Extend that into per-agent audit logs, rate/volume limits, token expiry/rotation, and an "All-access" warning → a governed agent registry with minimal conceptual stretch.

3. Score Summary (sorted views)

Top Security surfaces (anchor the pivot here): requests (5), data-risk (5), rules/manage (5), mcp (5), views (5), proxy (5), then connections/clients/ vendors/api-tokens/security-overview/events/adapters/adapters-manage/routing/ team (4).

Top Executive surfaces: mcp (4), proxy (4), deploy/preferences (4); then availability/security-overview/data-risk/vendors/views/team/services (3).

Finance — the whole column is weak. Nothing scores above 3 (vendors, services). The vast majority are 0–1. This is the defining gap of the product relative to the agent-governance mandate.

4. Persona Evaluation

Security / SecOps (CISO, Security Manager) — well served, mis-framed

• Pages: requests, connections, data-risk, security/overview, rules(+manage), events, mcp, views, proxy, api-tokens, clients, vendors, qtap, tls, jwt, team.
• How well served: Strongly. The forensic data (who called what, with which credential, carrying what payload, allowed or blocked) and the control plane (views, proxy, adapters, routing) are largely present.
• Gaps: No agent identity stitched across requests/connections/tokens/MCP; no unified "which agents, doing what, against what policy" view; alerting groups by rule, not by agent.
• Opportunities: Low — re-enable the commented-out Sensitive-Data column; promote auth-token/sensitive-data to filters; verify the adapter delete. High — an Agent Inventory/Registry that resolves sourceExe + tokens + MCP clients into one identity; an Agent-Risk Posture Board. 50k-ft overlay — a per-agent risk score rolled up from egress destinations, sensitive-data flows, and policy violations.

Finance / Spend owner — not served

• Pages: none owned today; nearest are vendors, bandwidth, services.
• How well served: Barely. The product measures bytes, never dollars or tokens. No budget, chargeback, or per-agent/per-vendor cost.
• Gaps: Everything. No spend dimension, no cost owner role, no budget alerts.
• Opportunities: Low — derive an "egress cost estimate" (bytes × per-GB/token rate) on vendors/bandwidth; add a "spend owner" team role. High — a net-new Agent Spend & Cost Report (token/$ per agent per LLM vendor, budget burn, MoM trend) and spend- ceiling alert rules. 50k-ft overlay — cost attribution by tag/cost-center so spend rolls up per agent, team, and vendor for chargeback.

Executive (board-level posture/ROI) — partially served, no home

• Pages: deploy/preferences (capabilities+residency+SOC2), security/overview (dials), data-risk (flow viz), mcp, proxy.
• How well served: Partially. The ingredients of a posture story exist but are scattered across operator pages; nothing is packaged for a board.
• Gaps: No single posture/ROI surface; no "agents governed," "egress default-deny on/ off," "open high-risk findings," "$ at risk," or trend; nothing exportable.
• Opportunities: Low — turn deploy/preferences into a posture scorecard (coverage %, governed agents, egress stance, open findings). High — a net-new Agent Governance Posture Board + a board-ready report/export. 50k-ft overlay — a single governance index (posture score) trended over time with drill-down.

5. Additional Personas to Consider

The prompt names Security / Finance / Executive and asks "others?". From the page grouping, five more personas clearly exist in the product today:

Recommendation for Phase 2 scoring: keep Security / Finance / Executive as the headline columns, and add Engineering/Platform and Compliance/Privacy as secondary columns — they own real pages today and the pivot increases their importance.

6. Executive & Finance Overview Gaps — Net-New Pages (Phase 2 candidates)

The prompt's headline question — "are there opportunities for executive overview pages and reports that do not exist in our wireframe?" — answers yes. Synthesizing the high-hanging opportunities across all sections, six net-new surfaces are missing, none of which exist in wf1 today. These are the candidate page set for the Phase 2 mockups.

1. Agent Governance Posture Board (Executive, Security) — one-glance board view: governed-agent count, sensor/egress coverage %, egress stance (default-deny on/off), open high-risk findings, a single governance index trended over time. Seed:security/overview dials + deploy/preferences.
2. Agent Spend & Cost Report (Finance, Executive) — token/$ spend per agent per LLM vendor, budget burn, MoM trend, top cost drivers. Closes the product's single biggest gap. Seed: bandwidth + vendors (bytes → tokens → $).
3. Agent Inventory / Registry (Security, Executive) — every agent as a first-class identity: owner, model/runtime, credentials, data-scope (dataset view), risk score, last seen, spend. Seed: mcp clients + inventory/clients + api-tokens.
4. Sensitive-Data → LLM Flow (Security, Compliance) — producer→consumer sankey filtered to model endpoints, with an allowed/blocked overlay and drill-to-samples. Seed:security/data-risk.
5. Agent Egress Control Matrix (Security, Engineering) — which agents may reach which LLM/MCP/3rd-party endpoints, default-deny posture, per-destination spend ceilings, live "uncovered egress" alerting. Seed: routing + proxy + adapters.
6. Agent Risk Register / Board Report (Executive, Compliance) — ranked open agent-risk findings with owner, status, and $ exposure; exportable board-ready report. Seed:alerting/events + security/overview issues.

Recommended Phase 2 lead set: the Posture Board (#1), Spend & Cost Report (#2), and Agent Inventory (#3) — together they cover all three primary stakeholders, close the Finance gap, and tell the clearest agent-governance story. #4–#6 are strong follow-ons.

Appendix — Cheapest wins already in the codebase

• Re-enable commented-out "Sensitive Data" column (inventory/clients, endpoints, vendors) and "Last Used" column (api-tokens).
• Fill the empty Capabilities/Hosting stubs in the /orgs/configure wizard.
• Make /stacks/adapters/manage delete verified (currently :verify="false").
• Add an "All-access" warning badge on MCP clients scoped to contextId = null.
• Add an agent/identity option to traffic/components/usage/GroupBy.vue — lifts scores across every analytics page.