The challenges posed by exponential data growth long predate those posed by artificial intelligence in the world of data centers. But as AI continues to transform industries, its impact on data storage is becoming increasingly profound.
Indeed, AI development has led to marked changes in the scale and complexity of enterprise data storage requirements, leading to increased demand for high-performance, scalable and autonomous storage solutions that are suitable for data-intensive AI workloads. Simultaneously, AI is making storage smarter and more efficient by enabling autonomous management, predictive maintenance, compliance enforcement and tiered storage hierarchies.
Jonathan Garini, CEO and enterprise AI strategist at Fifthelement, said AI models produce massive amounts of structured and unstructured data that demand both scalability and speed.
“One training cycle of a large language model can generate petabytes of intermediate data, so storage systems must be capable of supporting high-throughput access without introducing bottlenecks,” Garini said.
With companies continuing to embrace AI-based applications, storage is no longer just about capacity, Garini said. Increasingly, it’s also about intelligence.
Artificial Intelligence Storage: A New Take on an Old Problem
The notion of a “data deluge” has existed for over a decade, preceding the widespread adoption of fast flash-based storage and AI pipelines.
J.B. Baker, vice president of marketing and product management at storage and memory solution provider ScaleFlux, said the adoption of AI amplifies existing challenges within storage and memory systems while also creating new ones.
“AI processes are resulting in a multiplication of the amount of data that you need to store,” noted Baker.
He said AI processes are generating another 10 times the volume of data, as well as interim stores and metadata around the original data.
“We've just accelerated that flood of data creation.”
But with these challenges come new opportunities, as AI provides new means to mine the deepest recesses of large data archives, Baker said.
And yet, getting data ready for AI pipelines is not a straightforward task.
“You're just exacerbating those pipeline problems that you’ve had in the past,” Baker continued.
It’s not just the volume and velocity of data that has accelerated because of AI. It’s also the types of data.
“The variety of data has actually really increased,” said Ashish Mohindroo, general manager and senior vice president of Nutanix.
There is unstructured data, which is stored in relational databases; structured data, which is stored in non-relational databases; and semi-structured data, which is stored in document databases. Thanks to AI, a new storage medium — the vector database — has emerged, according to Mohindroo.
“In the database world, it's an exciting time, but now you have to manage all these kinds of databases,” Mohindroo said.
“You need higher automation to really deliver the kind of quality of service that applications have become accustomed to or are demanding from these underlying systems.”
Data Storage and AI Are Intertwined
Every data-driven business is grappling with AI’s impact on storage.
Mircea Dima, CEO and founder AlgoCademy, an AI-powered education platform, has witnessed firsthand how AI fundamentally redefines data storage needs. When he started it, AlgoCademy didn’t need much storage.
“Two years on, we are processing 300 terabytes of learning analytics per month,” Dima said. “AI models do not simply consume data, but they create exponentially more data via continuous learning cycles.”
AlgoCademy typifies the AI pipeline’s impact on data storage infrastructure. Each tutorial of algorithm completion, submission of code and individual feedback loop generates metadata that cannot be effectively stored by traditional storage, explained Dima, who said the rapid rise in data volume prompted AlgoCademy to migrate to a hybrid, tiered storage architecture that differentiates between hot and cold data. That migration cut the company’s storage costs by 60% with millisecond response time on its AI tutoring engine, he reported.
“Conventional methods just cannot scale,” Dima added.
AI now maintains AlgoCademy’s storage lifecycle, automatically archiving old versions of models and optimizing retrieval patterns.
Dima insisted that automation is no longer a luxury, as manual data entry was consuming up to 20 hours each week at his company. “We have predictive storage scaling in place that is based on predicting demand surges during the interview seasons,” he said.
Steve Morris, founder and CEO of AI marketing and analytics agency Newmedia.com, argued that metadata must be treated not as an afterthought, but as a living master catalog.
“When we started AI marketing pipelines, the volume of raw data sets would grow 10 to 30 times in less than a year,” he said. “What saved us wasn’t storage, per se, but metadata harvested at every point above every tier of storage.”
This “metadata first” approach cut data wrangling times in half on new model launches, according to Morris, who suggested the company would be stuck in the year 2022 if it had only optimized storage for capacity or performance.
“With metadata embroidered onto our storage fabric, we can confidently design for 2025 and beyond,” he said.
AI’s Impact on Compliance
AI is increasing demand for retention requirements and privacy controls while also assisting with compliance.
George Tziahanas, vice president of compliance at Archive360, noted that the requirement to archive large amounts of data in industries such banking, healthcare and government predates the AI data deluge.
Archive360 offers a cloud-native platform to solve this problem for regulated entities by helping them store petabytes of data in a scalable manner. “Where a lot of the operational systems have this data, they’re not designed to maintain this data over time, and to do it at scale,” Tziahanas said.
AI is helping to pull appropriate data from operational systems for archiving. And once data is archived, it can be pulled out again for various workloads within the AI pipeline by leveraging metadata in a secure manner, Tziahanas noted. “You can do that in a very safe way, and you can do that at scale,” he explained.
Like the data itself, Tziahanas said AI tools are now subject to different levels of governance.
“How did I train that model? What data did I use to train that model? How do I verify it was secure, private and non-biased?” he asked, adding that all this can be automated with the help of AI while also supporting data retention rules — including whether data is deemed as hot or cold.
“We're looking at agentic AI to manage retention, agentic AI to manage tiering and agentic AI to manage classification. That's the next step of automation for us.”
Making Storage Sexy
Matt Kimball is a principal data center analyst at Moor Insights & Strategy, where he covers servers and storage. He thinks AI has made storage “cool” again.
“Storage has become so critical over the course of the last couple of years as data analytics and AI started to become more and more a part of the conversation within IT,” he said.
A critical storage metric for organizations is whether they can quickly ingest and feed data into processing engines — typically a GPU or an accelerator — for the purpose of training and tuning models, said Kimball, who added that storage is also a critical enabler for going cloud-native.
“The ability for an organization to leverage a pool of resources very easily from a software-defined perspective is absolutely critical,” he said.
Kimball cited “co-engineered” solutions like Pure Storage and Nutanix’s offering for VM Block storage for hyperconverged infrastructure. He explained that it combines Pure’s FlashArray storage system and Nutanix’s AHV virtualization platform, and uses non-volatile memory express (NVMe) over TCP instead of fiber because it provides better performance. It’s also based on an open standard.
Technologies like NVMe are just one of many that are critical for tackling the AI-related challenges inherent in data storage infrastructure. For speed and efficiency, storage systems now use SSDs with fast NVMe interfaces alongside AI-driven management. Meanwhile, hybrid cloud and object storage supports scalable, cost-effective and high-performance data access for AI workloads.
Hard drives and tape storage aren’t going anywhere, however, as they’re critical for archiving high volumes of cold data that may need to be ingested again by AI pipelines.
Getting data to where it needs to be also is key for keeping pace with AI data growth and workload demands. Having freed SSDs from hard-drive architectures, NVMe has matured to add flexible data placement (FDP) capabilities. This gives the host server enhanced control over where data resides within the SSD, optimizing data placement for better performance.
In addition to these open-standard interfaces, various vendors are offering solutions to optimize data storage for AI pipelines. Among them, for example, is memory and SSD maker Micron Technology, whose 2600 NVMe SSD features Micron’s adaptive write technology (AWT) that improves sequential write speeds and performance of QLC NAND flash.
There’s also Silicon Motion’s MonTitan Enterprise Development platform, which includes its proprietary PerformaShape multi-stage shaping algorithm configured in firmware to optimize SSD performance and quality of service (QoS) while adding another layer to NVMe capabilities — including FDP.
The Future of Data Storage
High-performance, large-capacity storage hardware combined with faster interfaces are not enough on their own to handle the AI data deluge. Wherever possible, data management must also be more intelligent and more automated, suggested Fifthelement’s Garini, who said storage systems must be able to not only store data, but also actively optimize it.
“The most forward-thinking architectures can also predict access patterns,” Garini concluded, adding that businesses should think of automation-driven storage as building a foundation that will help them meet their AI data needs five or 10 years in the future.
“The winners will be the ones who think about storage not as a static warehouse, but as a dynamic, adaptive layer of their AI strategy.”
Gary Hilson has more than 20 years of experience writing about B2B enterprise technology and the issues affecting IT decisions makers. His work has appeared in many industry publications, including EE Times, Fierce Electronics, Embedded.com, Network Computing, EBN Online, Computing Canada, Channel Daily News and Course Compare.
© 2025 Nutanix, Inc. All rights reserved. For additional information and important legal disclaimers, please go here.