Gateway Controls can Deliver the Visibility IT Needs to Manage Agentic AI While Managing Risks

Agentic AI offers enormous promise for IT leaders charged with delivering change to their organisations. However, as with any other major technology shift, agentic AI also comes with attendant risks and costs. The good news is that those downsides can be addressed with filtering techniques that provide visibility into AI operations and ways to enforce defensive policies that protect assets and moderate costs.

Why all the excitement about agentic AI? In short, because it automates the nature of change, autonomously creating better processes and workflows based on prompted goals. By spawning sub-agents, it delves into IT systems, applications and data to understand the internal technical workings of operations and create superior alternatives. In that sense, it is a quantum leap over generative AI and it can act as the basis for fundamental business re-engineering. In short, it’s a dramatic change in the way IT operates, offering a delegated tier of intelligent compute counsel. 

Agentic AI gets organisations on the front foot to attack problems through innovation. But CIOs must also defend assets in a world of complex security threats and demanding governance regulations. 

In that context, agentic AI is a risk factor, so CIOs must control the vulnerabilities exposed by agentic AI. Already, we are seeing examples of MCP servers being accidentally surfaced to the internet, exposing whole catalogues of business processes. Agents with high privilege levels are becoming honeypots for attackers. The situation is reminiscent of the early days of cloud infrastructure platforms, when public cloud misconfigurations would leave gaping access vulnerabilities. 

There is also clear scope for high costs being rung up relating to AI tokens, compute costs and ‘Shadow AI’, where consumption is not being consistently monitored or approved.

Gateways and Gatekeepers

If we agree that agentic AI is transformative but must be carefully managed, how do we square that circle? Agentic AI moves so fast and so broadly that it is infeasible to manage risks with human manual inputs alone or via traditional IT governance actions. A better approach is to cut off risks at the pass.

AI gateways offer central control plane and filtering tools to protect against undesirable activities. They sit between enterprise applications, AI agents and foundational models, monitoring activities, risk scoring, and providing analysis of costs. 

By securing traffic from agents to LLMs and from MCP servers, order is restored with clear insights into activities and likely resulting actions. Gateways take a Zero Trust approach to security, assuming malicious intent until shown otherwise. Constant scanning of data before it leaves the corporate perimeter means that there is an inspection point to validate the movement of any personally identifiable data or other sensitive information.

Tool-level filtering and support for the RBAC (Role-Based Access Control) authorisation model help provide granular observability. RBAC supports even non-human identities – often AI agents and sub-agents - which, in the agentic AI era, will usually massively outnumber human identities. Some analyses suggest this can already be at a ratio of 100:1 and this gulf will surely grow as more agentic AI projects come onstream. The volume and also the velocity of these agents imply that human interventions have no chance of keeping up or scaling.

Log auditing means that any subsequent concerns can be tracked back to their sources, helping organisations to troubleshoot, but also stay on the right side of regulations. 

Furthermore, external providers and self-hosted deployments are accessed from just one API, so developers can easily use the appropriate model for each project or even switch between models. 

In terms of cost management, token rates can be limited and broken down by team or individual.

The best gateway options will support a wide variety of LLM and any type of deployment platform: on-premises, private clouds, public clouds, Kubernetes, virtual machines and bare metal. This is crucial in supporting developer flexibility and platform engineering options. 

Effectively, CIOs and other IT leaders can make decisions for the here and now, but also course-correct when changes are needed or the wider environment alters.

AI gateways also act as a form of esperanto between technical staff and non-technical staff. The controls afforded help support customer data sovereignty initiatives because there is no need to use public cloud. Conversely, CIOs can bypass current hardware supply crunches, such as shortages of NAND or DRAM memory, by using public cloud.

A CIO can demonstrate that the threat of Shadow IT can be curbed, because activity is monitored, and helps prevent a repeat of the ‘Wild West’-style chaos that occurred in the early days of cloud platforms, when teams bypassed IT and purchased credits for AWS or other platforms using credit cards. 

An Agentic AI Gateway is an intermediary platform that centralises the management, routing, and observability of AI models. Whether an organisation relies on public frontier models (like OpenAI or Anthropic), open-source alternatives, or private, self-hosted LLMs, the gateway provides a unified endpoint.

So, gateways maintain openness but support policy enforcement by routing traffic. The resulting benefits are similar to the way internet proxy servers brought order to who could use web services. 

Often, smart IT departments will take a hybrid IT deployment route for load balancing purposes, or to switch models when one is maxed out or has become risky or unaffordable. A gateway should support this flexibility. It can enable intelligent routing based on data sensitivity, for example, so low-risk queries can be routed to cost-effective public hyperscalers, while requests containing more sensitive information can be automatically routed to private, air-gapped models. 

By forcing all agent-to-model and agent-to-agent communication through a gateway, IT regains visibility and freedom of movement. If a specific model is considered insecure, it can be swapped out at the gateway level without developers needing to recode applications.

There are other benefits too. Using a gateway model, the resulting visibility means IT gains insight into how tools are being used, where and when, potentially impacting corporate IT decisions on what they choose to keep or discard, or how they optimally license tools.

Similarly, in terms of token spend, IT can attribute and even chargeback AI spend to specific departments. Cost-optimised routing can automatically select the model that meets the required latency, cost and prompt complexity criteria. In this way, the CIO and CFO can share insight and show value.

The price of IT change is always to manage value against risk and outlay. By using AI gateway technology, CIOs can gain the best of both worlds.

Discover more about building an IT infrastructure for AI here