How to Store Insane Volumes of Data in a Database

Cloud storage and hyperconverged infrastructure help enterprises store, access and protect large volumes of data easier than ever before.

By Dipti Parmar

By Dipti Parmar March 09, 2020

Data storage needs are constantly on the rise. The amount of internal data generated and external data collected, organized and utilized by businesses increases exponentially with every passing day. Even the forms and formats of data files are multiplying. Good thing new innovations come along to meet these challenges, but progress can be hampered by a weak storage strategy.

Information and analytics form the core of the digital transformation of enterprises and small to medium businesses (SMBs) today. Mining and using relevant data is critical to developing better products, providing better customer experience and, most of all, creating and maintaining a competitive advantage.

An IDC study titled “Digitization of the World” predicted that the sum of all data created and stored would increase from 33 ZB in 2018 to a staggering 175 ZB in 2025. This creation and storage happens in three areas:

  • Enterprise data centers and all varieties of public, private and hybrid clouds

  • Servers and appliances located outside of data centers, including remote servers and cell towers

  • Endpoints such as laptops, smartphones, wearables and IoT devices

Source: Seagate Data Age 2025

Companies manage large amounts of data generated by consumer-facing applications and processes. This requires them to constantly upgrade and vary their technologies while lowering or maintaining the total cost of ownership (TCO). It helps maintain competitive advantages, reach new markets, improve customer service times and build better products.

Putting Data in the Cloud

When organizations use the cloud for data storage, networking and computing needs, they place applications in certain cloud depending on workload requirements. This makes storing and backing up data in the cloud more of a business process than a tech operation. In multicloud and hybrid environments, choosing cloud vendors and managed service providers (MSPs) also becomes a critical part of the process.

Cloud storage enables more storage and more accessible data in the following ways:

  • Allows organizations to locate data in specific regions for lower latency as well as adhering to compliance regulations

  • Lowers costs with monthly or pay-per-usage payment methods

  • Eliminates silos caused by traditional NAS appliances, which are complex to set up and operate

  • Does away with the need for dedicated file storage systems by virtualizing local storage devices into a unified pool that can be managed from a simple interface

  • A single namespace represents the file cluster, regardless of its size and the number of user sessions

  • Access is optimized automatically across workloads with intelligent sorting of hot and cold data and variable length I/O

  • Built-in software analytics from the vendor ensures early detection of anomalies and risks, as well as better visibility on user permissions and the files they’re accessing

Source: Nutanix Files

The San Francisco Bay Area Rapid Transport transitioned all of their finance, HR and maintenance systems that used traditional storage systems and HP servers to a private cloud architecture.

“It provides us with the performance we need for our Oracle environment, simple management, and all of the visibility we need to effectively understand what is happening with our database operations,” said Ravi Misra, BART CIO, in a case study by Nutanix.

[Read related case study: BART Switches to Nutanix Enterprise Cloud]

Modernize Datacenters and IT Infrastructure

The other side of the cloud coin for enterprise companies is a modern, flexible IT infrastructure. Intelligent infrastructure systems can rid organizations of the strangling necessity of basing data storage and access decisions on technology resources and limitations rather than the unique needs of the workload in question. This allows the business to focus on providing value to customers while scaling storage and associated computing capabilities to use increasingly larger amounts of data.

Integrating and modernizing organization-wide IT infrastructure brings the following benefits to businesses:

  • Distributed storage ensures there are no single points of failure and negligible impact to real-time performance

  • Adds resiliency with automatic failover, data redundancy and error correction by overwriting of bad data with replicated copies

  • Software-defined compression ensures high efficiency of file shares, eliminating vendor lock-in for hardware and devices

  • Backups can be scaled better and faster, non-disruptively, with software-based solutions.

  • Caching, data tiering, and data locality ensure performance acceleration while recording and accessing data

  • Technologies such as global deduplication, inline compression and erasure coding enable storage capacity optimization of file and data clusters

  • Disaster recovery is guaranteed with synchronous, asynchronous and near-synchronous replication

  • Handle use cases for unstructured data by hosting virtual machines, files, block and object storage on the same platform, eliminating the need for standalone storage arrays while reducing overall costs

Source: Nutanix Acropolis

The Chesterfield Royal Hospital in the UK found themselves in a situation where their legacy infrastructure wasn't able to handle their data demands and found it difficult to manage locally and remotely hosted databases.

“We had reached a point where our ‘make do and mend’ approach was no longer a workable option,” said David Sawyer, IT Technical Delivery Lead.

“Not only were we running out of storage, but we also had to implement multiple backup solutions just to meet the existing demands, let alone cope with future workloads.”

A Nutanix data modernization deployment created immediate, high-performing storage headroom for future growth, along with data protection and failover resilience for critical applications, while simplifying file management.

Be Data-Ready for Digital Transformation

Businesses across sizes and markets are shifting from product-centric to customer-centric models because of the ubiquitous availability and usability of information, making consumers more demanding of services.

As organizations handle increasing volumes of data, it is important to rationalize data commitments and view it as a strategic resource instead of a functional byproduct. Mission-critical data has to be tended to and invested in to strengthen and scale the business. CIOs need to take the lead in assessing data readiness of their companies, making sure the necessary budgets are allocated and staff is empowered to gather, mine and use data for driving operational and strategic success.

Featured Image: Pexels

Dipti Parmar is a contributing writer. She has written for CIO.com, Entrepreneur, CMO.com and Inc. magazine. Follow her on Twitter @dipTparmar.

© 2020 Nutanix, Inc. All rights reserved.  For additional legal information, please go here.