Report Shows Public Sector's AI Ambitions Outrun Its IT Infrastructure

At the Riverside County Children and Families Commission in Southern California, about a dozen caseworkers have spent recent months testing an AI agent that fills out benefit applications for them. Built by the public benefit corporation Nava and backed by a $1.5 million grant from Google's Generative AI Accelerator program, the tool scans the agency’s databases and benefit systems, pulls in what it can, and hands the caseworker a mostly completed form to check and approve, Route Fifty reported. It is a promising start in a sector racing headlong into AI adoption, while their IT infrastructure, governance and workforce capacity to sustain it struggle to keep pace.

Like the estimated 82% of U.S. public sector organizations that have adopted agentic AI on some level, Riverside County is counting on the deployment to accelerate benefits processing while freeing caseworkers from the tedious paperwork to focus more time on direct client support — a meaningful goal in a region where more than 347,000 residents rely on federal food assistance and 306,000 K-12 students qualify for free or reduced-price school meals.

But if Riverside County is anything like most other public sector organizations, it will find a broad range of hurdles standing in the way of wider AI adoption.

While the vast majority of public sector organizations are racing down the AI superhighway, 73% admit their on-premises infrastructure is not fully ready to support modern AI workloads, according to the 2026 Public Sector Enterprise Cloud Index (ECI) report, which is a part of the broader 2026 ECI report from Nutanix.

Despite the optimism and promise that agentic AI holds for improving operational performance and the citizen experience, most projects remain in pilot stages. Nearly 60% of federal use cases, for example, remain in pilot or pre-deployment, a Brookings Institution study found. And 88% of state and local government leaders said their agencies are more likely to start another pilot than expand an existing one.

“It isn’t for lack of commitment,” Sherry Walshak, director of global public sector solutions at Nutanix, told The Forecast. 

She said government leaders know that AI can be a tool to radically transform their operations and improve service delivery, but face dealing with the silos across their organization. 

“Let's say that one department will want to do a small use-case pilot,” she explained. “That works well until they allow additional users because that same model is not being hit by 10 users, but by 100+ users. They realize that the model or that GPU memory they have isn’t enough and they experience bad performance. Then they stop using it.” 

These experiences reveal that the underlying IT infrastructure cannot support these larger loads.

Breaking Technology Barriers

AI applications tend to be cloud native, containerized and built for hybrid environments from the start, Walshak said. It’s a mismatch for the legacy systems many agencies are still running.

“You're trying to run tomorrow's workloads on yesterday's architecture,” she said. “Containers give agencies a way to bridge that gap without a full rewrite of every legacy system. But even a modern container strategy doesn't fix the problem if the underlying data remains fragmented. Data readiness is one area where most agencies get stuck and it has to be solved before governance policies can even be written, because you can't govern data you can't see.”

The ECI report shows how quickly that visibility problem has escalated. Some 96% of public sector IT leaders encounter “shadow AI” apps or agents installed by employees outside IT, and 91% said those tools create business or mission risk.

This shift was swift. In 2024, ECI respondents viewed AI largely as a future investment. By 2025, security became the focus. This year, shadow AI added a new complication: unsanctioned projects operating beyond IT’s view and control.

Walshak said that government and education leaders are looking at consolidation. They want fewer technology providers, fewer tools and fewer procurements to simplify their environments. They know it helps having a platform that can run and manage virtualized, containerized and AI workloads. 

“As you consolidate and modernize your infrastructure, then you can unify the management of it all, instead of supporting multiple point solutions," Walshak said.

"Then, you can consider solutions like the Nutanix AI Gateway that provides a single, governed access point for approved LLMs, giving IT the visibility and control to say yes to AI safely, instead of blocking it outright."

Most public sector organizations are heading in that direction. IT leaders went from viewing containers primarily as tools for easing hybrid cloud complexity in the 2024 ECI to treating them as vital AI components in 2026, with 86% holding that view today and expecting containers to be even more central three years from now.

Repurposing Processes

Data sovereignty tops the list of process challenges, with 88% of public sector IT leaders ranking it a high priority or hard requirement. Agencies must know where sensitive information lives, who can access it and whether controls hold as data and applications move between environments.

Speed is another hurdle. Government approval, acquisition, budgeting and security processes were built around slower software cycles. An AI model can change several times while a conventional review grinds on.

Brookings points to federal acquisition rules, lengthy authorization timelines and budget cycles that begin roughly 18 months before the fiscal year. By the time an agency funds and approves an AI pilot for production, the underlying technology may have moved on. Of the 445 federal AI use cases Brookings classified as "high impact" in 2025, more than 85% of those already deployed were missing required information on testing, monitoring, impact assessment or how citizens could appeal a decision. The pilots had outrun the governance meant to hold them accountable.

The State of Tennessee offers a model for closing that gap. It built an AI advisory council into statute, pulling in lawmakers, universities, government organizations of all levels, and industry partners, then created a review committee that makes agencies justify business value, funding and data readiness before a pilot launches.

“Everybody has a great idea for AI,” Tennessee CTO Jerry Jones said at a recent NASCIO conference, according to StateTech Magazine. “You have to have a way to filter that.” Skip the filter, and agencies end up funding pilots before confirming the underlying data is even usable.

Agencies can't realistically stop employees from experimenting with AI tools, Walshak said. They can control which data those tools ever touch. Role-based and attribute-based access controls limit employees to only what their job require. Nutanix pairs that with file analytics that log who accessed a given file and when, giving agencies a way to trace the source of problems. An internal control protocol requiring annual training and sign-off on data-handling rules before employees can access sensitive systems, AI included, adds another layer of accountability.