The Secret to Google’s Datacenter and Why it Matters to Every Enterprise

In October 2012, Google finally revealed photos and details about their previously secret data centers. For years, people had speculated on their location and the trade secret that powered tools such as web searches and Gmail. Since Google released pictures of their data centers people are fascinated by the beautiful and slick architectures of web-scale companies that have redefined how the data center should be built and managed.

Pictures From Wired Magazine – Oct 2012

This very idea that everyone can have a Google-like data center is a core principle that drove the formation of Nutanix in 2009. To day this date, we continue to allow any enterprise to have a such a data center in minutes.

In this post, I want to focus on one of the most important features of this new type of data center – the storage controller. Google’s data center appliances, similar to Nutanix appliances, have what we will call a software-defined, scale-out storage controller versus a traditional disk-based storage controller that is embedded with most SAN & NAS storage units.

A software-defined, scale- out storage controller is a storage controller which enables a cluster to be aware of other nodes in a cluster with the benefits of scalability, flexibility, and failure reliability. Web-scale companies such as Google, Amazon, and Facebook, build out their own data centers by purchasing enterprise grade hardware and developing custom software to define their specialized versions of this modern breed of storage controller. Traditional SANs add scale-up capacity without adding controllers, while the Google and Nutanix approach adds capacity in addition to controllers.

Traditionally, the disk-based storage controller is configured as single or dual controllers per storage unit. Traditional controllers may support similar functionalities (compression, encryption, storage provisioning etc,) as scale-out controllers, however these functions do not have performance that linearly scales. These functionalities are perfect candidates for scaling and can scale with SW-defined scale-out storage controllers. In the traditional approach, capacity is added without adding controllers and compute intensive tasks can lead to significant bottlenecks.

Digging deeper, we set out to answer an interesting question at Nutanix: When will the number of software defined scale-out storage controllers in the data center catch up with traditional disk-based storage controllers in the data center (globally speaking)?

To run this analysis, we spoke to industry experts to approximate what portion of server and storage configurations in the world leverage a SW-defined storage controller. We compared this to the number of traditional storage controllers on storage units by key players such as EMC.

The answer surprised us. Take a guess! Our back of the envelope math estimates that this cross-over will occur within the next 12-18 month at approximately 3 million storage controllers – a time frame no one would have forecasted a few years ago.

Research Sources: SEC, Datacenter Knowledge, Gigaom, Web Research

Some interesting facts from our research:

  • The growth in software-defined scale-out storage controllers is attributed to web scale architectures. An article by Wired magazine highlighted this shift: In 2008, three server companies bought most Intel server chips. By 2012 – eight server companies made up 75% of Intel server chip sales with Google being “something like number 5 on that list.”
  • 2014 is projected as the year that enterprises rapidly begin embracing web scale contributing to the cross-over. In an ESG study on Private Cloud Infrastructure, 44% of small to large enterprises showed great interest or planned to even begin using integrated computing platforms such as Nutanix in the next 12 months.
  • This trend is further validated by the growth rate of server manufacturers such as Supermicro and Quanta which are categorized as “other” server vendors in analyst reports by Gartner and IDC. The “other” server category grew a startling 14% in late 2013 while regular server giants such as HP & IBM saw small increases and even losses.

Why does this matter to your enterprise? This trend represents a fundamental shift in the industry. If you are not thinking about scale-out storage controllers in the next year, you may be taking the riskier approach to your data center – an archaic, non-agile approach.

The bottom line: If you don’t expect to have a Google-like data center in the next year, you should think again. Times have a changed. A few years ago, most enterprises – small, medium, and large could not have thought about dedicating massive resources to build scale-out data centers. The Nutanix appliance running the Nutanix Distributed File System allows you to have a Google-like data center in under forty-eight hours and without thousands of PhDs. Are you ready to make a shift? The time has come to once again make IT a competitive advantage.

Special thanks to Bryan Byrne for his contributions.