Nutanix Hybrid Cloud Architecture Enables the Nation’s 7th Largest Public Utility to Increase System Resiliency, Support Rapid Expansion, Accelerate Application Performance, Simplify OT Management, and Reduce TCO
Public Utility
Austin Energy is the United States' seventh largest public utility. With a service area that exceeds 437 square miles, the utility provides electrical power to over one million customers in the City of Austin and surrounding areas. Austin Energy’s total generating capacity is more than 3,000 megawatts, provided by a mixture of wind power, solar power, biomass, natural gas, nuclear power, and coal. As a transmission provider, Austin Energy is subject to NERC regulatory compliance. Violations can result in fines up to as much as $1M depending on the severity of the incident.
Andy Ibarra is the manager of advanced technology and control engineering at Austin Energy. He and his team are responsible for maintaining the utility’s electrical grid transmission and distribution control systems, which includes the implementation of new technologies, ensuring electrical system security, maintaining the utility’s substations, and conducting all patching and compliance projects.
Time for an Upgrade
Austin Energy’s legacy electricity transmission control system performed adequately over the last 16 years, but the infrastructure was reaching end of life and due for an upgrade. The utility also had an advanced distribution management solution (ADMS) system that was implemented in 2012 on an SAN that needed more capacity. “We tried to expand our ADMS environment in 2014 to accommodate growth, but we were told it would cost over $1M to buy all of the extra storage systems,” noted Ibarra. “Since we didn’t want to invest more capital into that outdated platform, we decided to look for newer technology options.”
Austin Energy’s control engineering team also wanted to upgrade the utility’s PI Enterprise Historian system. “PI Enterprise records everything that happens on our control systems in real-time, just like the ‘black box’ on an airplane,” Ibarra explained. “That system is essential to our operations, since it provides all of the data our team needs to troubleshoot current issues, and to improve the performance and reliability of our electrical grid going forward.”
Finding Nutanix
Ibarra and his team brought in an enterprise solution provider, Freeit Data Solutions, and investigated a variety of infrastructure options for the upgrade. “We originally planned to purchase a large number of storage arrays to expand system capacity, but Freeit asked if we had looked at some of the newer hyperconverged offerings,” noted Ibarra. “They gave us a demo of Nutanix and showed us how easy it was to create VMs on a tablet in just a few minutes. We were very impressed with the ease of use and all of the analytics that come with the Nutanix platform.”
Running the POC
Austin Energy ran a POC of the PI Enterprise application on Nutanix in 2016. “PI Enterprise worked very well on Nutanix,” Ibarra reported. “It outperformed anything we had ever seen in the past. The easy scalability of the Nutanix platform was also very attractive. As we grow, we can easily keep adding nodes to the hyperconverged Nutanix solution.”
A Fast and Easy Deployment
After completing the POC on Nutanix, Austin Energy made the decision to deploy Nutanix throughout the infrastructure. Ibarra and his team deployed the Nutanix NX nodes at Austin Energy’s two geographically separated datacenters. All administrative systems ─ including Active Directory, SQL, and MySQL databases ─ are now replicated from the Nutanix systems at one site to the other, and the utility’s essential control systems are quad-redundant with dual Nutanix systems at each data center site. “It used to take two weeks to deploy all of our 3-tier systems,” reported Ibarra. “In contrast, it only took three days to get everything up and running on Nutanix – two days to do the installations and another day to configure the software. The ease and quickness of the Nutanix set up was quite impressive.”
Mission-Critical SCADA Systems
After deploying Nutanix for the PI Enterprise workload, Austin Energy turned their attention to the mission-critical OSI Monarch Control System and Schneider Electric ADMS. Freeit Data Solutions, Nutanix, and Austin Energy engineers worked with OSI and Schneider Electric developers to properly size the HCI environment, execute a proof-of-concept, and certify these two critical applications for use on Nutanix HCI and the AHV hypervisor. Previously, these applications were only certified for deployment on bare-metal systems. Austin Energy was successfully able to cut over to the new HCI environment in mid-2020.
Moving from VMware to AHV
Austin Energy also migrated from VMware’s vSphere hypervisor to Nutanix AHV. “We didn’t want to spend any more money on VMware licenses, so we switched to the integrated AHV hypervisor which offers all required capabilities," Ibarra said. “I was tired of having to ask for more licenses each time we grew our environment. As a public municipality, scaling the environment is difficult and time-consuming because we have to get approval from the City Council for any new infrastructure purchases. Not having to order and pay for the extra VMware licenses was a huge bonus for us.”
Ease of Management
Austin Energy’s OT team uses the Prism UI to manage the Nutanix environment. “Prism is a lot easier to navigate than the VMware interface,” said Gilbert Nguyen, senior analyst and infrastructure manager at Austin Energy. “It takes a lot of time to learn how to use the vSphere control panel. That’s not the case with Nutanix. By moving to Nutanix, our team was able to go from managing just two large control systems, to five large systems with the same number of OT admins.”
Simple, One-Click Upgrades
“Nutanix Life Cycle Manager (LCM) provides the ability to do one-click upgrades of the clusters,” said Nguyen. “I just upload the file, download the patch file, load it to the cluster, click ‘upgrade’, and then move on to other projects. It used to take several days to upgrade all of our individual servers, and I’d have to sit there and monitor the systems to make sure nothing went wrong. Nutanix is so much easier to manage ─ it’s a ‘set it and forget it’ environment.”
Leveraging the Nutanix Cloning Feature
“I recently had to build 50 Windows VMs for a new system I was setting up,” explained Nguyen. “I created a chart that showed how much CPU and storage each VM needed and determined there were just three sets of common specs. So, I created one VM of each of the three configurations and used the Nutanix cloning feature to make the rest of the VMs. As a result, I was able to create all 50 VMs on a Saturday afternoon. Installing 50 Windows boxes one at a time on our previous infrastructure would have taken me several days.”
Relying on Nutanix Pulse
Austin Energy’s OT team uses Nutanix Pulse to keep an eye on the health of their systems. “With all of the proactive alerting, self-awareness, and self-healing capabilities of the Nutanix systems, we are able to discover any potential problems before there’s an issue,” said Ibarra. “A lot of our apps and infrastructure also have the ability to phone home, but most of them rely on direct Internet access. Nutanix is one of the few companies that enables their phone home feature to be used from email. I really like that paradigm since I don’t want to ‘poke holes’ in our firewall with Internet connectivity to a third-party organization. That’s a huge benefit to us from a security perspective.”
Increased Reliability and Resiliency
Austin Energy was also able to improve system uptime by moving to Nutanix. “If our ADMS system ever goes down, we can’t control the power grid,” explained Ibarra. “We have to be able to recover these systems quickly to meet our goal of 99.8% uptime. With the legacy SANs and physical servers, we’d have to rebuild everything from scratch. It used to take at least a full day to get everything running again, but it only takes just 10-20 minutes to restore everything with Nutanix.”
“We had an incident a few months ago where the entire datacenter lost power,” said Nguyen. “After power was restored, the Nutanix nodes automatically came up without any problems. One initially came back up with an error message, but once the rest of the data center had calmed down, the error resolved itself on its own. There’s no way a SAN would do that.”
Eliminating the Need to Worry
“Our systems must run 24x7,” said Nguyen. “I used to walk through the data center several times a day, just to look for red lights. Since moving to Nutanix, I don’t have to worry about our systems all day and all night anymore. The Nutanix platform is rock solid, and I know it will email me if something requires my attention.”
Receiving Great Support from Nutanix and Freeit
Ibarra also had great things to say about the quality of Nutanix and Freeit’s support. “When we call Nutanix with a question or issue, we immediately get connected to someone that knows the answer,” he shared. “That just isn’t the case with our other vendors. In addition to working with the Nutanix team, we also rely on Freeit Data Solutions for installation services and support. Freeit’s team did a great job with the implementation and are always available when we need help. It’s great to have two great partners who work as part of our extended team.” In addition to implementation and support, the Freeit team has conducted numerous training sessions with the Austin Energy team to ensure they remain up-to-date with the latest recommendations, best-practices, and available features.
A Lower TCO
By moving to Nutanix, Austin Energy was able to shrink its datacenter footprint from five rows of cabinets to just two, resulting in significant savings in datacenter power and space. They also conducted a TCO study after implementing Nutanix which revealed the following savings:
“With the extra time our OT team has on its hands after moving to Nutanix, we are now able to take on more responsibilities,” said Ibarra. “Previously, when management asked us to add more applications or services, we either had to say no or explain why it would take an excessive amount of time and budget. With Nutanix, we can say yes to all of the new projects that come our way without any hesitation.”