How Data Lifecycle Management is Different in a Hybrid Cloud World

Data has a lifecycle that keeps getting tougher to manage, especially in increasingly complex hybrid cloud environments.   

Almost 60 billion terabytes of data will be generated by 2020 alone, IDC predicts, and those numbers will only grow with the rise of data-driven IoT sensors, artificial intelligence and high-bandwidth 5G mobile networks. Data lifecycle management brings order to all this, using technology to rationalize the creation, storage, analysis, backup and deletion of data.

It's always been a handful to manage the data lifecycle, but the challenges have exploded with the proliferation of APIs, IoT systems, virtual machines, microservice and containers architectures, along with everything else IT teams use to give their employers a digital edge.

And then there’s cloud computing. Public cloud platforms can transform storage, archiving and disaster recovery. Private clouds can harden security and ensure compliance. It’s not an either/or question for most organizations these days — it’s both. That’s driving the move to hybrid clouds.

The question for IT teams and their employers is how to manage the data lifecycle effectively in the new hybrid cloud world. It requires a clear strategy and the right technologies to address problems that arise from siloed IT systems, lack of control or visibility across those systems, and specialized teams and vendors. Technologies such as Nutanix Mine, which comes integrated with Nutanix hyperconverged infrastructure (HCI) software, consolidate and automate backup operations to help manage the hyper growth and sprawl of data.

The Weight of Data

It’s reasonably quick and easy to move applications and virtual machines. Data, however, does not have that kind of flexibility.

“Data is sacrosanct — it has gravity,” said Priyadarshi Prasad, general manager of Nutanix Objects and Mine, software for storing unstructured data, protecting data and managing data lifecycles.

He explained that data must be stored properly, accessed easily and protected perpetually to ensure reliability and accuracy. It doesn’t have the mobility of apps or VMs. 

“It takes time to move data from one place to another, so maintaining a continuum of data lifecycle management in this hybrid cloud world is really a big challenge,” Prasad said.

Troubles often arise when hybrid-cloud technologies are incompatible with conventional private cloud tools. “It’s like fitting a square peg in a round hole in some sense,” Prasad said.

Modern data lifecycle management tools can square the circle with APIs that standardize data mobility, simplifying the process of moving data within hybrid cloud environments, Prasad said. 

“Using an API, you can start an on-premises or private cloud and move data to the public cloud if you like,” he added. Alternately, users can start cloud-native and utilize APIs to move data back to the private data center. He said the API-driven data movement occurs transparent to applications accessing the data, ensuring data accessibility while realizing data mobility.

Dealing with Silos

Mobility is central to data lifecycle management in hybrid-cloud environments because data must have the flexibility to move seamlessly between private and public clouds. Silos tend to impede data mobility.

Silos have multiple dimensions: tools, people and processes. IT teams often use separate technologies to manage storage, applications, virtualizations and networking servers. 

“They’re all silos with different management interfaces, troubleshooting techniques and so on,” said Mark Nijmeijer, a director of product management at Nutanix. 

Moreover, primary and secondary storage often create silos if backup and disaster recovery have separate architectures — requiring even more tools.

IT teams also create silos through their expertise and work processes, Nijmeijer added. 

“Backup is a completely different scale than managing virtualization, so backup teams are very specialized,” he said. 

Silos also happen during duplication and backup. Multiple copies of data in the primary architecture may become invisible when the data goes to backup. 

“You get this data sprawl,” Nijmeijer said. “And you might store a lot more data than you actually need.”

Achieving Scale

Public cloud platforms enable IT teams to scale rapidly in lockstep with new demand or seasonal business. In hybrid environments, primary and secondary architectures inevitably expand (or contract) on a scale that affects data lifecycle management.

“The moment you give me that infrastructure, I'm going to start generating data — new PowerPoints as an IT worker, new code, whatever it is,” Prasad said. That requires more attention to cybersecurity and data protection.

As the world adapted to the COVID-19 pandemic, many IT teams scaled up overnight with virtual desktop infrastructure (VDI) and desktop-as-a-service (DaaS) to serve their remote workforces. These technologies use file shares that require specific data-lifecycle approaches, according to Nijmeijer.

“You have to make sure those file shares are protected and can be easily restored and can scale out as well,” Nijmeijer said. “If you’re growing from 100 users to 1,000 users, then your file services need to scale out very, very efficiently as well.”  

Managing Backup and Disaster Recovery

Before the advent of low-cost cloud computing, tape drives were the go-to source of data backup.

“In the old world, tape was an order of magnitude cheaper than primary storage,” Nijmeijer said. “But the moment you go to tape, you lose both data and visibility of the data.” 

Recovering data from tape can take hours, days or weeks. Nijmeijer said hybrid cloud backup and disaster-recovery can make backup data available much faster at dramatically lower costs.

But there may be trade-offs in hybrid clouds. 

“Now you need a cloud admin, somebody with an AWS (Amazon Web Services) account, et cetera, et cetera,” Nijmeijer said. “You also lose data visibility. What’s out there? How do I index it? How do I know how many copies of my data are there?”

Another piece is the orchestration of data recovery. Orchestration pulls everything back together when recovering systems from backups. Software and automation are increasingly crucial to data lifecycle management, because IT systems and architectures are becoming so complex.  

Tools for Simplifying Data Lifecycle Management

Data lifecycle management is a straightforward concept. Applications, sensors and computing devices give life to data. At some point, data gets copied, analyzed and stored on a hard disk or memory chip. When it’s deleted, new data takes its place.

But things get tricky in an ever-expanding universe of data-driven workloads in hybrid cloud environments. At enterprise scale, it’s all but hopeless without automation.

That’s the challenge Prasad, Nijmeijer and their colleagues are taking on at Nutanix, one of the many tech companies developing sophisticated tools for data lifecycle management in hybrid multi-cloud IT systems. Nutanix pioneered the concept of hyperconverged infrastructure (HCI), which uses software to virtualize compute, storage and networking in a single management plane, streamlining processes and reducing total cost of ownership. The move to software-defined infrastructure is shining new light on how to best manage and move data.

A Nutanix service called Mine, for instance, leverages automation to unify backup and recovery in an easy-to-use interface that emulates the HCI model. That makes life much easier for IT pros to manage the data lifecycle in hybrid clouds. 

“We bring value by simplifying complex data life-cycle management operations,” Nijmeijer said. “We make it easy to manage data end-to-end, from creation to long-term archival, all under one platform. This saves IT people a lot of time. Time that they typically spend keeping the lights on in the data center.” 

Applying new approaches and automation to data lifecycles is increasingly critical as companies move to hybrid multi-cloud IT systems.

Tom Mangan is a contributing writer. He is a veteran B2B technology writer and editor, specializing in cloud computing and digital transformation. Contact him on his website or LinkedIn.

© 2020 Nutanix, Inc. All rights reserved.  For additional legal information, please go here.