Enterprise artificial intelligence has a commitment problem. Every few months a better model arrives, and the one a company chose starts to lose its appeal.
With AI technology advancing rapidly and models offering different capabilities, most enterprises keep their options open. Multi-model AI, which runs several models from different vendors at once and matches each to the tasks it handles best, is now the enterprise norm. The shift gives companies flexibility and bargaining power, but it also turns governance into an architectural challenge that few have mastered.
Multi-model AI is an enterprise approach that runs several AI models from different vendors simultaneously, matching each model to the tasks it handles best rather than relying on a single provider. "It's too early for companies to 'marry' a single model vendor," said Kartik Hosanagar, co-founder of Bodhium Labs and co-director of Wharton Human-AI Research. "They have to date a few first."
A recent Morning Consult survey found only 27% of U.S. companies with at least 5,000 employees use a primary enterprise AI platform, and 44% added at least one platform in the past year. 45% say they are likely to shift more use toward open-source or self-hosted models over the next 12 months.
Some companies even play the field with the same model. In a March survey of 820 AI business decision-makers, The Futurum Group found nearly two-thirds of enterprises used OpenAI's models directly, and nearly as many used them through Microsoft Azure. Brad Shimmin, a data and analytics analyst with the firm, said the dual approach gives companies more options in how they pay for models and which security or compliance rules apply to data processing.
Why Does Multi-Model AI Make Governance Harder?
Governing a growing collection of models and platforms gets harder with every addition. Each brings its own rules for handling data, controlling access and recording activity, while IT teams still need to apply consistent standards across them all.
That means knowing which model handled a request, what company data it received and whether its answer met the application's standards. It also means keeping those safeguards working when a provider updates a model or an application sends the same task to a different one.
Companies need to enforce common policies, especially when the models themselves run in different places, Shimmin said.
"When you're running one model, governance is a checklist," he said. "When you're running four across different deployment environments, governance becomes an architectural discipline."
Few companies have mastered it. Futurum found 93% of enterprises struggle to build the governance production AI requires, Shimmin said.
Buying habits add to the strain. Line-of-business leaders, data science teams and platform engineers each bring in models through their own channels, and the chief information officer no longer controls the whole AI budget, Shimmin said. Futurum has also flagged surprise overage charges when agent workloads spike, he added.
How Can Enterprises Keep Model Choice Without Losing Control?
Experts suggest four ways to rein in the growing mix of models without sacrificing choice.
The first is creating a model short list. Most chief information officers keep a small roster of approved models, Hosanagar said, and many are now putting a model router in front of it. A model router works like a switchboard operator, directing each request to the approved model best suited to handle it based on rules the company sets for quality, speed, cost and data sensitivity. Developers no longer have to hard-wire an application to one model.
"Usually, it means having a preferred provider for most uses, a few approved alternatives and a common layer for security, evaluation and monitoring," Hosanagar said. "That preserves option value without forcing every dev team to solve governance and model integration."
GitHub already works this way. Since June, Copilot's auto mode has picked a model based on "the complexity of your request and current model availability," while still letting users choose one themselves.
International Data Corporation (IDC) expects the approach to spread. By 2028, 70% of top AI-driven enterprises will route requests across models with advanced multi-tool architectures, IDC predicts.
"Stop optimizing around a single model and start designing architectures that can host and switch between many," wrote Neil Ward-Dutton, IDC's vice president for AI, automation, data and analytics in Europe.
The second approach is having one place to track everything. Gartner recommends dedicated AI governance platforms that keep a central inventory of AI assets and enforce policy at runtime. Organizations using them are 3.4 times more likely to achieve effective AI governance than those using traditional risk and compliance tools, Gartner says.
The third approach is benchmarking models on the tasks and priorities the company cares about. Public benchmarks are fine for gauging general model performance and capabilities, but they rarely capture which model fits a company's specific needs, Hosanagar said.
Engineering teams should build internal benchmarks from company data and score each model on accuracy, reliability, response time, cost at scale, security and regulatory needs, he said. Often, the most capable model turns out to be more than a task requires.
The final suggestion is to build a data foundation before adding more models. Shimmin said the enterprises that handle multi-model AI best put centralized access controls and a data catalog that works across environments in place from the beginning.
"The ones struggling bolted on a second and third model without rethinking the plumbing," he said.
Where Do Open and Self-Hosted Models Fit?
Some companies want control over where their models run, too. Open-weight models give teams the option to run or fine-tune a model in an environment they manage, Shimmin said. That can appeal to organizations with strict requirements for where they process sensitive data.
Closed models still have fans. More than a third of enterprises prefer them, according to a January survey of Global 2000 executives by Andreessen Horowitz. Morning Consult, meanwhile, found 66% of the largest companies plan to build internal AI capabilities on top of foundation models. Platforms such as Nutanix Enterprise AI allow IT teams to run open and commercial models on infrastructure they control.
Hosanagar sees the company's own data as the real prize.
"A foundation model is also available to your competitors, so access to it rarely creates lasting advantage," he said. "The moat lies in proprietary data, workflow integration, customer trust and creating some sort of a learning loop around the application."
Related:
David Rand is a business and technology reporter whose work has appeared in major publications around the world. He specializes in spotting and digging into what’s coming next–and helping executives in organizations of all sizes know what to do about it. Find more of his work here.
© 2026 Nutanix, Inc. All rights reserved. For additional information and important legal disclaimers, please go here.
Related Articles
