AI Control Plane
The AI control plane is the governing layer between every AI agent in an organization and every system they’re allowed to reach. It unifies connection, identity, policy enforcement, and observability so that every prompt, response, and tool call flows through a single controlled path.
AI tools are proliferating within companies faster than anyone can keep track of. Teams are adopting Cursor, Claude Code, Copilot, ChatGPT, and dozens of internal agents, often independently and often without IT involvement. Most organizations have no reliable way to know which tools are in use, which company data those tools can access, what is being shared with external AI services, or whether any of it is being measured.
The AI control plane is the architecture that solves these problems and puts enterprise AI governance on a single footing. It covers four capabilities (connect, distribute, secure, and observe) across every AI agent and every system those agents reach.
The rest of this guide walks through why this layer is now necessary, how it differs from the AI governance tools enterprises already have in place, and what it looks like in practice.
Why enterprise AI governance breaks down at scale
Companies want to adopt AI because it makes teams faster and produces better work. The budget is there, the appetite is there, and AI agents like Cursor, Claude Code, Copilot, and ChatGPT have already found their way into daily workflows across engineering, sales, finance, and ops.
The problem is execution. Rolling AI out consistently across an organization is hard. Most companies that try run into the same set of gaps:
- No central place to provision MCP servers, skills, and other tools. Every team picks its own.
- No consistent identity layer. People connect personal accounts to enterprise data.
- No visibility into what is being used, by whom, with what data.
- No way to enforce policy on tools IT cannot see.
- No measurement of adoption, and therefore no way to know whether the investment is working.
Locking everything down kills adoption, and leaving everything open invites accumulating incidents.
An audit typically turns up dozens of AI tools in use across the organization, many never approved and none centrally managed. This is the shadow AI problem, and it is already widespread: in the 2026 IBM Cost of a Data Breach Report, 43% of breached organizations traced incidents to shadow AI, up from 20% a year earlier. The cost of doing nothing is clear: data is leaked through consumer AI tools, prompt-injection attacks target agent systems, and enterprises face regulatory pressure from the EU AI Act.
“We're rolling out AI faster than we can govern it. I don't think anybody in this industry isn't.”
CIO, Fortune 500 retailer
Which AI governance platforms exist today?
Several categories of AI governance platform have emerged to address parts of the enterprise AI governance problem, each covering one slice of it.
AI control plane
LLM gateways and AI gateways (Portkey, LiteLLM) sit between applications and language models. They handle routing across providers, API key management, rate limiting, caching, and cost tracking. An AI gateway operates at the model call layer.
MCP gateways and agent gateways (Speakeasy, Runlayer, MintMCP) sit in front of MCP servers and handle authentication, authorization, and inspection of tool calls. An agent gateway is the runtime chokepoint that agent traffic flows through. The Model Context Protocol (MCP) is the emerging standard for how AI agents connect to external tools and data.
Identity and access controls (Okta, Microsoft Entra ID, Google Workspace) provide SSO and enterprise identity infrastructure. Most enterprises already have these in place, though they were not designed for AI agents.
Observability tools (Langfuse, Arize, Galileo) provide visibility into what AI systems are doing. They record which tools are being called, and what outcomes they produce, by which users, and with what frequency.
Policy and threat detection tools (Lakera, Fiddler, Treza) inspect prompts, responses, and tool calls in real time, looking for PII leakage, prompt injection, data exfiltration, and policy violations. The primitive that makes this possible at the AI agent itself is the agent hook, a lifecycle handler that fires on every prompt and tool call.
Tools in the wild
Where each AI governance tool falls short
Each category solves a real problem. However, none of them covers the full picture because each sees a different part of the same interaction:
- LLM gateways see the model call, but have no visibility into the user who triggered it, the tool that made the request, or whether the data returned should have been accessible.
- MCP gateways cover the tool call layer, but have no visibility into the model layer above it or the identity layer behind it.
- Identity providers know who the user is, but have no visibility into which AI tools that user is running or what those tools are doing with company data.
- Observability tools record what happened, but cannot enforce policy on it in real time.
- Policy and threat detection tools can inspect and block traffic, but have no way to provision access or manage identity.
No single tool has a correlated view across all these layers. An incident that starts with a user authenticating through SSO, routes through an LLM gateway, triggers an MCP tool call, and leaks data in the response would leave traces in five different systems with no common thread connecting them.
Can incumbent AI governance vendors close the gap?
Most enterprises already have vendors shipping AI governance features, and each occupies one layer. Hyperscaler controls (AWS Bedrock, Azure AI Foundry, Google Vertex AI) apply only to traffic routed through their platform. Enterprise platform vendors (ServiceNow, CrowdStrike, Salesforce, and the major GRC suites) own the policy and risk layer but sit above the traffic, though Salesforce’s MuleSoft Agent Fabric and ServiceNow’s AI Control Tower are now adding in-path enforcement. Point tools such as LLM gateways and MCP security tools are inside the traffic but see only their own layer.
AI governance requires owning the path between layers, and closing that gap means rebuilding significant parts of each incumbent architecture. The vendor-by-vendor analysis is in Can incumbent AI governance vendors close the gap?
How does an AI control plane work?
The term “control plane” originated in networking and is now standard in systems like Kubernetes and service meshes, which have control planes for the same reason: they name the part of the system that governs the rest of the system. Some analysts and vendors call the same layer an agent control plane when they want to stress the autonomous agents it governs, but the architecture is the same.
How the AI control plane balances enablement and governance
The AI control plane has two jobs. The first is enablement: rolling out AI capabilities across the organization so every team can use them. The second is governance: making sure that rollout doesn’t cause a security incident, a data leak, a compliance violation, or a regulatory problem.
These two jobs pull in opposite directions. Enablement wants more tools, more data connections, and broader access. Governance wants every interaction inspected, every permission scoped, and every action audited.
Why splitting them across two teams fails
The instinctive organizational response is to assign enablement and governance to different teams. For example, IT or security could own governance, and a platform team or newly hired Chief AI Officer could own enablement.
This split has a consistent failure mode. The governance team, lacking visibility into what people actually need from AI, designs controls that block the most useful capabilities. The enablement team, under pressure to ship, routes around the controls or negotiates exceptions. The result is either a governance posture that exists on paper while employees use personal ChatGPT accounts for real work, or an adoption program that moves fast and creates incidents the security team discovers in postmortems.
A single layer resolves the conflict
This failure is structural. Governance and enablement look like opposing forces because organizations treat them as separate layers. When the same layer that routes AI traffic also enforces policy on it, governance and enablement are no longer in tension.
Consider the infrastructure analogy: Kubernetes does not have one component that allows workloads to run and another that ensures they run safely. The control plane does both, by design.
The four functions of an AI control plane
An AI control plane does four things: connect, distribute, secure, and observe. The diagram below shows how the four functions fit together.
AI control plane
Connect
It should bring every AI agent (Claude, ChatGPT, Cursor, Copilot, Codex, internal agents, and product agents) and every system that matters (SaaS tools, internal APIs, databases, and skills) onto a single plane, with per-team registries and SSO-integrated identity.
The result is that new AI capabilities reach the right teams in days instead of months, and IT stops chasing a moving target of ad-hoc integrations.
Distribute
It should govern how AI tools and access roll out across the organization, enforcing who can use what and under what conditions by scoping access to teams or roles, implementing credential management that keeps API keys out of config files, using OAuth 2.1 where the protocols support it, and maintaining a full audit trail.
The shift that matters here is that policies become versioned, testable, executable rules applied automatically at the point of use. The control plane evaluates policy on every request, unlike a wiki document that relies on people remembering to follow it.
The policy on paper becomes the policy at runtime, and audits stop surfacing questions the CISO can’t answer.
Secure
It should inspect every prompt, response, and tool call in real time, with active blocking of PII and data exfiltration, passive detection of prompt injection and shadow MCPs, and integration with existing SIEM and security tooling rather than replacing it.
Incidents become detectable in real time, and the response team works from the tooling they already know.
Observe
It should measure what’s actually happening. By recording token use by team, agent, tool, and user, then tracking that use against organizational targets, the control plane produces data that proves whether the AI investment is working.
Leadership gets real numbers behind the AI investment rather than anecdotes, and can distinguish adoption from adoption theater.
How does an AI control plane deliver AI guardrails and AI risk management?
Two terms show up constantly in vendor pitches and analyst reports: AI guardrails and AI risk management. The AI control plane is how an enterprise operationalizes both.
AI guardrails are the runtime controls that constrain what AI agents are allowed to say and do, blocking PII leakage, prompt injection, unsafe tool calls, and data exfiltration. In the control plane, this is the Secure function: real-time inspection of every prompt, response, and tool call, integrated into existing SIEM tooling rather than replacing it. The threat models these guardrails address are covered in detail in AI security.
AI risk management is the broader program of identifying, measuring, and mitigating the risks AI introduces, whether regulatory, security, operational, or reputational. The control plane is the execution layer for that program. Connect gives risk teams a real-time inventory of every AI agent and tool in use. Distribute turns policy on paper into rules enforced at runtime. Secure detects active threats. Observe produces the audit trail and metrics that risk and compliance teams need to evidence the program.
Guardrails and risk management are different layers of the same architecture, and the control plane is what lets one organization run both.
AI control plane vs AI governance framework
It is easy to conflate the two, but they sit at different levels. An AI governance framework describes what an organization should do; an AI control plane is what actually does it.
Most AI governance programs are built on published frameworks: NIST AI RMF, ISO/IEC 42001, the EU AI Act, and the internal policy decks that follow them. They tell an organization to identify AI use cases, classify risk, document controls, monitor outcomes, and review periodically. The framework is the program. For how the major frameworks compare and where each one stops, see AI security frameworks.
An AI control plane is the runtime that executes the program. The framework says “log every interaction with sensitive data”; the control plane is what intercepts the interaction and writes the log. The framework says “scope access by role”; the control plane is what evaluates the role on every request and decides what passes.
The two are complementary. A governance framework without a control plane is a document. A control plane without a framework is infrastructure with nothing telling it what to enforce. Tools that ship policy templates without runtime enforcement leave the gap between policy and behavior open, and the control plane is what closes it.
What a mature AI control plane enables for the enterprise
When the pieces are in place, the operational shape of the organization changes in a handful of specific ways.
AI reaches every team quickly
New tools get added to the central registry once and are available to the teams entitled to use them. Teams stop standing up their own integrations in parallel, and new AI capabilities reach the right people in days rather than months.
Consistent permissions across every tool and agent
An engineer using Cursor, a salesperson using ChatGPT Enterprise, and an analyst using an internal agent are all subject to the same permission model, because the permissions live in the control plane rather than in each agent.
Security visibility into the AI layer
Security has visibility into AI in the same way it has visibility into the network, the endpoints, and the cloud, enabling real-time incident detection and providing data for audits to draw from.
Adoption metrics for leadership
The AI control plane gives leadership actual numbers for tracking adoption by team, discerning which tools are producing outcomes and which are shelfware, and seeing where AI is moving the business. Leaders can base investment decisions on concrete metrics rather than anecdotes.
Reusable infrastructure for every subsequent project
New AI initiatives ship on shared infrastructure instead of rebuilding the connection, identity, and governance layers every time. The second and tenth AI projects cost a fraction of the first.
The components for doing each of these things exist today. What the control plane provides is the architectural foundation that makes them work together.
What to consider before deploying an AI control plane
Deploying a control plane has costs, and they shape whether the deployment succeeds.
Latency
Adding the control plane to every model call and tool call means adding a network hop. For interactive use cases, the overhead is typically small, but for agentic workflows that chain many calls in sequence, it compounds. The content inspection layer adds further delay if it runs synchronously.
- Agentic loops with 20 or more sequential calls can accumulate several seconds of added latency.
- Synchronous PII inspection on every response adds delay on the critical path.
- Asynchronous inspection reduces latency but means some traffic is not caught in real time.
Vendor lock-in
The AI control plane category emerged in 2025. The vendors building in this space are early, and the architecture has not yet settled.
- A vendor’s architecture may shift significantly as the category matures.
- Acquisition can change product direction quickly in early-category markets.
- Deep integration with a single vendor’s identity and policy model makes migration expensive.
Mitigating lock-in means preferring open standards at the integration points. Use components that you can swap without rebuilding everything connected to them, such as OAuth 2.1 for identity, OpenTelemetry for observability, and MCP for tool connections.
Evaluating a control plane?
Score any vendor, or your homegrown setup, against the full set of capabilities, deployment, and vendor criteria.
A note on Speakeasy
Speakeasy is building the AI control plane. We started with the connection and identity layer, the first place companies typically get stuck when moving beyond ad-hoc AI adoption, and have been extending across the four functions since. The coverage map below shows where we are today.
Where Speakeasy plays
The reference architecture in this guide reflects what a complete solution needs to look like. It is the architecture we are building toward. If you are an engineering or platform team figuring out how AI should flow through your organization, get in touch.