What is data storage?

Every piece of information you gather in your business is potentially important. Whether it’s a database of customer accounts and their purchase histories, human resources records for every employee, financial information such as payroll or accounts receivable, or intellectual property such as specifications for your products, that information needs to be stored somewhere so you can access it as needed to operate your business.

Forms of data storage

File storage Also referred to as file-level or file-based storage, this type is presented as files that are stored in folders and subfolders in a directory. If you’ve used a PC and saved a document to the hard drive, for instance, you have experienced file storage. Files are accessible through a path that users must know. File storage is the most common type of storage on computer hard drives and network-attached storage (NAS) devices. Block storage Rather than storing data in a complete file, block storage breaks apart data and stores it in blocks that can hold 256 KB up to 4 MB. The blocks are then placed randomly on the storage device – which doesn’t hinder access speeds because each block is tagged with a unique identifier. When a user or application needs a file, the computer queries the block storage system, which rapidly identifies and collects all the needed blocks and reassembles them into a complete file. Block storage is ultra-efficient because it doesn’t use metadata, which takes up room on the storage device. Because it’s so fast, businesses tend to use block storage when they want to scale up quickly and when read/write performance is a critical priority. Object storage This type of data storage separates information into distinct objects with unique identifiers and metadata. It doesn’t form a hierarchy like file storage and is considered “flat.” Metadata is a critical component of object storage and each object can have quite a bit of metadata, such as information about the creator of the data and keywords, as well as policies for security, privacy, and access. Object storage works well with APIs so it’s simple to use with existing software and systems. It also scales very well. It could be spread across hundreds or thousands of devices and locations and still be super speedy because all the data is stored in one namespace. It’s the storage type of choice for public cloud providers, such as AWS, and organizations that deal with a lot of unstructured data such as video files, emails, IoT sensor data, social media content, and more.

Types of data storage solutions

Direct-attached storage (DAS)Network-based storage: NAS and SAN

The importance of data storage

Long-lasting preservation of data – Digital data storage makes it easy to pool large volumes of information for long periods of time. Easier accessibility – Everyone can get the information they need, right from their desktop computers instead of having to physically go to a room full of file cabinets.More efficient data recovery – Stored data is easily backed up by making copies, so recovery is faster and simpler if a file gets lost or corrupted. Smaller physical footprint and easier scalability – There’s no need for physical file cabinets, which take up lots of room over time, and adding more digital capacity is simple.Potentially increased security – For very sensitive data, there are many more ways to secure and protect that information digitally using today’s advanced security tools and features.Easier collaboration across teams – Centrally stored data is accessible to every authorized user and can be accessed and shared among teams as they work together. More efficient document management – It’s easier to categorize and organize data digitally, and it can be done from a computer desktop or other connected device. Enhanced productivity and workflows – Saving data digitally takes less time than printing out physical pages and creating files, which must be stored in file cabinets.

How does data storage work?

When you store data on a computer, the digital information is saved to a device, where it remains until deleted. Storage differs from computer memory in a major way: While you can access information quickly from your computer’s RAM, that data is only available in RAM until your computer is turned off. Data storage, however, retains data whether the computer is powered on or off. That’s how it can store data long-term.Saving data on a computer storage device lets you access that information when you need it. For example, the accounting department can retrieve stored data about employee work hours and use it to carry out bimonthly payroll tasks. A sales rep can pull up information about a customer before a sales call to see recent orders and go over the last visit to remind them of the issues they discussed.

Types of data storage devices

Cloud storageHybrid cloud storageSSD and flash storageBackup storage

What is the best for storing data?

DAS can be a good choice for:Small businesses, branch offices, and home offices that want a cost-efficient solution and don’t need networkingHosting just one application and its dataSecuring and isolating data for compliance purposesSAN can be a good choice for: Large data centers or business-critical applications that need centralized storage resourcesLarge numbers of employees and an IT staff with skills to manage and maintain the solutionApplications that require very high performance, such as transactional databases Hosting large relational databases that need high performanceLarge enterprises that need a high degree of storage scalabilityA fault-tolerant environment for critical applicationsNAS can be a good choice for:Reliable, cost-effective storage for many users that doesn’t compromise performanceLarge volumes of unstructured data that need to be shared among usersApplications that aren’t performance-sensitiveArchiving or backing up dataServers hosting virtual desktop infrastructure (VDI) and virtual machines (VMs)Cloud storage can be a good choice for: Archiving or backing up data for disaster recoveryData lakes and data analytics platformsApplication development and testing Storing workloads during a data migrationStorage for cloud-native applications and their dataStorage for industry and regulatory complianceIoT and machine learning data

What is the safest form of data storage?

DAS can be considered secure in the sense that a hacker cannot infiltrate a network and carry out an attack. Many removable storage devices such as SSDs, USBs, and so on are considered “air gapped” if they aren’t on a networked system. However, if the device is damaged or lost, the data could be irretrievable. NAS is a fairly common target for hackers. The storage system is only as secure as your network and router, so it’s important for NAS users to take proven security measures such as regular updates, two-factor authentication, single sign-on processes, encryption, and so on. SAN is inherently more secure than NAS because its data protection algorithms stay consistent and because if your LAN is attacked, having your data stored on a separate network keeps it protected. Cloud storage is typically considered the most secure type of data storage because major cloud service providers, such as AWS, Google, and Microsoft have put billions of dollars into making it secure. It’s also secure because providers make multiple copies of your data to create a high degree of redundancy – so if a cloud server goes down, you can still access your data because it’s stored in various servers and locations.

Blog

Highlighting Key New Capabilities for major AHV update in AOS 6.7

By Bob Ball

September 7, 2023 | min

The Nutanix AOS™ 6.7 release includes a new major version of the Nutanix AHV® hypervisor (20230302.207), which has many exciting core improvements. Before we dig into some of the changes in the underlying AHV version, I wanted to highlight a key, highly requested AHV feature added in AHV 20230302.207 and AOS 6.7 - On Demand Cross Cluster Live Migration!

Migrating VMs Across Clusters

Many Nutanix customers will already be aware that the ability to live migrate VMs to another cluster - and indeed to a cluster in another Prism Central™ instance - already existed as part of the planned failover feature in the Nutanix Disaster Recovery™ solution. This feature has enabled a large number of VM movements between clusters since AOS 5.19, and we have always been keen on streamlining the user experience to make this key functionality more accessible, with as few as two steps to migrate a set of VMs.

With AOS 6.7, we’ve provided a new option right from within a VM’s Action drop down - “Migrate Across Clusters” which can be triggered for just one VM or a batch of VMs that need moving to another cluster.

This option allows you to move a VM within the local Availability Zone or to another Availability Zone that has been linked with the local Prism Central instance. As expected, you can then choose any of the clusters in the destination Availability Zone to be the destination of the virtual machine. Depending on the VM configuration, the migration to the destination cluster could be completed in as few as three steps; selecting the destination and then running a few checks that confirm there is enough space on the destination before starting the migration itself.

Under the hood, the cross-cluster migration functionality is backed by the heavily proven syncrep technology also used by Nutanix Disaster Recovery, with the migration occurring in a three primary phases:

Migration Checks

Ensures there are sufficient resources (e.g. CPU, memory, storage, GPU) on the destination cluster and warns if any anomalies are detected

Synchronize Storage

Ensures all VM disk writes are synchronously mirrored to the destination cluster while copying the VM storage to the destination cluster in the background.

VM Migration

Migrates the remaining VM state (e.g. Memory, CPU, and GPU) to the destination cluster, in the same way as ADS (Acropolis Dynamic Scheduler) does for all in-cluster migrations.

We really hope you like the simplification of how you can now access and experience simpler cross-cluster live migration. As these steps are built on technologies like syncrep and AHV’s live migration flows, they also benefit from improvements we are making in those areas without any further integration needed.

Faster AHV upgrades with High Speed Networks

During an AHV upgrade, the system needs to Live Migrate virtual machines from the upgrading node to another node so the VM does not experience any interruptions during the node upgrade process. The final phase of each of these live migrations will need to move the VM state between these nodes, with the bulk of this state being the VM memory, which needs to be transferred over the network while the VM is still running. Unlike the storage migration described above, where every disk write is mirrored in real time to the destination cluster, a different approach is needed to migrate VM memory because memory writes are typically expected to be several orders of magnitude faster than typical latencies observed over a network. Each AHV live migration therefore uses an iterative process to migrate the VM memory, with each iteration copying the difference in memory that had been changed by the running VM since the previous one. This means that the rate at which a virtual machine is modifying memory is a key factor in how many times we need to copy the difference in memory and therefore how long the upgrade may take.

The second key factor is how fast we can transfer memory across the network to the new node, and that’s where the high speed networks come in. If the available link between the two nodes is 20GB/s or more then the migrations may be able to benefit from using a larger number of parallel connections between the source node and destination node and we can transfer memory faster in each iteration, reducing the impact of the rate at which a virtual machine is modifying memory.

The end result of this improvement is that, during an upgrade (or any other operation which is placing a node into maintenance mode), the length of time each VM migration takes may be substantially reduced, providing much faster upgrades.

Major update to internal components

AHV is based on proven virtualization components such as the open source KVM hypervisor, with substantial optimizations and extensions to produce an enterprise-grade hypervisor. This major release of AHV includes an update to many of these internal components, bringing the latest performance improvements, bug fixes and opportunities for feature development to the platform.

With this major update to internal components, we were keen to include a new upgrade approach which is much more resilient to potential changes or customizations which may occur on the node. Such changes aren’t designed to be preserved across an upgrade, but have been historically problematic during the upgrade process. We’ve therefore introduced a new method to upgrade from AHV ‘8’ releases (AHV versions 20220304.x) to AHV ‘9’ (AHV versions 20230302.x): re-image based upgrades.

As the name implies, this re-image based upgrade approach performs a full re-imaging of the hypervisor during the upgrade process. In order to preserve various configurations of the system across an upgrade, however, we do make a copy of key configuration settings, such as the hostname of the node, and re-apply them after the upgrade.

Migrate VMs

Moves all VMs off the node to other nodes in the cluster so the node can be upgraded without affecting user VMs

Backup Configuration

Preserve hostname, network setup, CVM setup and other specific node configuration details

Reimage Hypervisor

Reboot the node into a reimaging environment, and perform the hypervisor reimage process

Restore Configuration

Re-apply the backed-up configuration and then reboot the node into the newly imaged AHV version

It is important to ensure that all modifications to AHV nodes are made using the supported methods to ensure they are preserved over upgrades, as both re-image based upgrades and in-place upgrades will restore certain node configurations to the values configured by the supported methods.

Next generation APIs for AHV Virtual Machine Management

AOS 6.7 also brings an update to the Early Access (EA) of the v4 next generation of APIs. Following the initial EA release of the APIs in AOS 6.6 we made some minor changes, and the latest API documentation can be found on developers.nutanix.com. The v4 APIs mark a substantial improvement over previous API versions, with more and more products enabled and an update from the intent-based v3 APIs to REST APIs in v4. We are aware that movements from the v3 APIs to v4 will require changes to existing systems, so we’re not deprecating our v3 APIs yet. We’d also love any and all feedback you have on the new APIs to keep moving forward and continue enabling more use cases.

That’s just the top four things to talk about in this exciting new AHV release, but also included are thousands of changes, bug fixes and performance improvements and other features. Please check the release notes for more details of what else is in store when you upgrade.

© 2023 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo and all Nutanix product, feature and service names mentioned herein are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. Other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s). This post may contain links to external websites that are not part of Nutanix.com. Nutanix does not control these sites and disclaims all responsibility for the content or accuracy of any external site. Our decision to link to an external site should not be considered an endorsement of any content on such a site. Certain information contained in this post may relate to or be based on studies, publications, surveys and other data obtained from third-party sources and our own internal estimates and research. While we believe these third-party studies, publications, surveys and other data are reliable as of the date of this post, they have not independently verified, and we make no representation as to the adequacy, fairness, accuracy, or completeness of any information obtained from third-party sources.

This post may contain express and implied forward-looking statements, which are not historical facts and are instead based on our current expectations, estimates and beliefs. The accuracy of such statements involves risks and uncertainties and depends upon future events, including those that may be beyond our control, and actual results may differ materially and adversely from those anticipated or implied by such statements. Any forward-looking statements included herein speak only as of the date hereof and, except as required by law, we assume no obligation to update or otherwise revise any of such forward-looking statements to reflect subsequent events or circumstances.