Introduction
Nutanix Cloud Infrastructure (NCI) 7.6 with Nutanix AHV hypervisor 11.2 combine to make near bare-metal AI performance in VMs on specific systems a reality through a series of new functionality, optimisations and deployment simplifications. This allows management of the workloads needing the highest performance from within the Prism Central management console.
When considering AI workloads, the most important resource in the system - the GPU - needs to consistently be kept active. Individual GPUs can cost tens of thousands of dollars, and when putting 8 of them into a single HGX system, you really want to minimize bottlenecks that would prevent the GPU from being under utilized.
There are many different bottlenecks which can prevent the GPU from getting data fast enough to keep it busy, across all components of the system, including latencies when accessing local memory, between GPUs on the same host or even with GPUs on other hosts. Each of these latencies can be optimized in different ways, but add complexities to the configuration of the VM, container and workload. This blog post digs into some of the key improvements that allow AHV 11.2 and NCI 7.6 to give near bare-metal performance for NVIDIA AI workloads when running on certain HGX systems, but also how Nutanix brings VM management simplicity to these highly complex and technical needs.
VM and Device Topology: Why it matters
When software applications can understand how a physical or virtual machine is set up and configured, they can make more appropriate decisions, for example using memory in a way that understands the NUMA(Non-Uniform Memory Access) latencies and using devices in a way that understands the physical connectivity between devices.
NUMA-aware CPU, Memory and devices
Most modern systems have non-uniform memory. In the simplest example, memory is attached to one of the two CPUs in a dual socket system. When a CPU core accesses a particular memory location, that memory could be local (i.e., attached to the same socket) or remote (i.e., attached to the other socket) - accessing local memory or devices is faster than accessing remote memory or devices. It’s therefore important for the system to know where the CPU core is with respect to the memory and devices it needs to access so optimal scheduling decisions can be made - such as running a particular process on a CPU local to the memory that the process needs to access.
In the first diagram, the topology of the CPUs, memory and physical devices have not been taken into account. When the GPU needs to access system memory, or the CPU needs to access the NIC, not only does the data need to traverse the PCIe connections, but also the connection between the two sockets. These additional latencies add up to real impacts to AI workloads. The second diagram shows what the topology should look like for optimally selected CPU, GPU, NIC and memory with shorter paths between all of the components.
Figure 1. Poor NUMA awareness, with CPU threads and GPU on socket 1, and memory and NIC on socket 2
Figure 2. NUMA aware placement and device selection, with CPU threads, memory, GPU and NIC all on socket 1
GPU to GPU connectivity
When the needs of an AI workload exceed the capability of a single GPU device, the obvious answer is to use more than one GPU. Some hosts can support 8 GPU devices, enabling multiple devices to be used simultaneously for one workload. But when two GPU devices on the same host need to talk to each other, the latencies of transferring data over PCIe connections or even worse cross-NUMA boundaries can be very large. In the case of NVIDIA GPUs, this additional latency can be mitigated using NVSwitch, a separate fabric which directly connects GPU devices together. In HGX B200 systems, the local PCIe connection between GPUs could achieve 128 GB/s, however connectivity requirements between GPUs often exceeds that and the direct GPU to GPU connections provided by NVSwitch can run up to 1.8 TB/s. The use of NVSwitch does add complexity, however, as the connections between GPUs are dynamic and need to be configured based on the size of workload that’s needed.
Figure 3. Comparison of a slower GPU to GPU connection not utilizing NVSwitch and a faster GPU to GPU connection utilizing NVSwitch
GPU to Network device connectivity
Not only is the location of the device to memory and CPU important (e.g. NUMA awareness of devices), but the physical connectivity between devices is also critical to identify the most efficient way the devices can be used. GPUDirect RDMA is an example of optimisations that devices can use when the physical connections are known. GPUDirect RDMA can substantially reduce the latencies needed for GPU devices to send or receive data over the network by bypassing the CPU and system memory when the GPU and network devices share a PCIe switch, enabling more direct access between the GPU’s memory and the network device. This capability is particularly key when processing needs can’t be contained within a single physical host and need to be spread across multiple hosts.
Figure 4. Comparison of a sub-optimal selection of GPU and NIC which don’t share a PCIe switch, and an optimal selection where the PCIe switch is directly shared by the GPU and NIC
Implications for Virtual Machines
The previous sections show that having workloads being aware of the physical topology is crucial for optimal performance, but virtualization is designed to provide an emulated set of hardware to the virtual machine to simplify the management, so what has NCI 7.6 and AHV 11.2 added to satisfy these conflicting needs?
- vNUMA - Defining how memory is expected to be structured between CPU cores within the VM. If a VM is configured with 2 vNUMA nodes, then the memory and CPU is split equally between the two nodes. AHV is designed to map each vNUMA node seen by a VM onto a single physical NUMA node to improve in-guest scheduling decisions.
- Device vNUMA awareness - Defining the relationship between devices and vNUMA nodes, e.g. with 1 vNUMA node, all of the PCI devices must be physically on that vNUMA for optimal performance. Similarly, if 2 vNUMA nodes are needed then an accurate relationship between the PCI devices and vNUMA nodes needs to be presented to the VM.
- Dynamic NVSwitch Management - Right-sizing the connectivity links between GPUs on the same host to maximise performance for GPUs in each VM while the multi-tenancy boundary between VMs on the same host is respected and GPUs associated with different VMs on the same host cannot interact with each other.
- Network/GPU topological awareness - Selecting GPU and network devices that are closely connected based on the PCIe switch topology, ensuring the VM can use GPU Direct RDMA to bypass the system memory.
When scaled up to a VM running with 2 vNUMA domains, 4 GPUs connected in a mesh using NVSwitch, each with a matching network device the virtual machine will see topology that mimics the host’s real physical topology, allowing the VM to make use of all of these latency minimizing techniques
Figure 5. View of virtual hardware matches physical connectivity
The logic to place an AI workload needs many steps including the following: Select a host to run a workload on, understand the host’s physical layout, select GPUs, initialise GPUs, partition GPU connections, optimize memory placement, optimize vCPU scheduling, select network devices, enable ATS, configure ACS, create the VM layout, and then finally start the VM.
AHV has added all of these capabilities for HGX systems to NCI including functionality so that ADS is capable of placing these VMs efficiently, but defining the requirements for a VM can still be complex - not all VMs with multiple GPUs need NVSwitch management or NUMA awareness, so NCI 7.6 is also introducing a capability to make deploying AI-optimized VMs easy for all users.
AI-optimized VM profiles
When building an AI virtual machine, the above shows some of the things that need to be considered, but it’s worthwhile noting that the majority of AI virtual machines have all of the same needs. If the VM is going to use a set of GPUs and network devices, then vNUMA awareness for CPU, memory, and devices, NVSwitch definition and GPUDirect RDMA are all important. Therefore one challenge might be making sure that all of the correct settings are used and nothing is forgotten.
AI-optimized VM profiles solves this problem by providing easy-to-use profiles for each of the possible AI-optimized setups (8 GPUs & NICs down to 1 GPU & NIC in a VM). Deployment is as simple as selecting the profile and giving the VM a name. Profiles like the “ai.large.b200.8gpu.cx7_ib” describe exactly what you’re getting - eight B200s, with eight ConnectX-7 NICs running in Infiniband mode. The intelligence to make sure the right setup of NUMA nodes, device selection and device configuration to support the high performance needs for AI is all built-in.
Figure 6
Performance results
This blog includes details of the technical changes that Nutanix have implemented, but can these changes really make the claim of having near bare-metal performance for AI workloads in AHV virtual machines? Results from tests using NVIDIA Collective Communications Library (NCCL) show that yes, NCI 7.6 and AHV 11.2 really do provide near bare-metal performance in virtual machines for AI workloads.
To validate this performance, we set up internal test infrastructure using Dell PowerEdge R760 HCI nodes and NVIDIA DGX B200 GPU nodes, with the AI workloads running on the GPU nodes. NKP was used to manage the k8s worker nodes, created using the ai.large.b200.8gpu.cx7_ib AI-optimized VM profile.
With this test infrastructure the performance for all validated message sizes exceeded 99% of the bare metal results, demonstrating that the required technologies have been exposed to virtual machines in the most optimal fashion:
- GPUDirect RDMA - Sustained bandwidth over 99% of bare metal capacity
- NVSwitch - NCCL all_reduce_perf on a single VM with 8 GPUs produced algbw and busbw results across all message sizes exceeding 99% of the bare metal bandwidth
- Combined test - NCCL all_reduce_perf on multiple VMs with 8 GPUs in each VM showed algbw and busbw results across all message sizes exceeding 99% bare metal bandwidth
Summary
Our internal testing of NCI 7.6 shows that the near bare-metal performance needed for AI workloads can be delivered by VMs running on AHV, combining the simplicity and flexibility of VM management within Prism Central with the demanding needs for the most complex workloads.
AI VM profiles make the deployment of these complex workloads trivial to manage and integrate with other NCI functionality.
We’re really excited about what NCI 7.6 and AHV 11.2 have accomplished, and can’t wait to see how our customers are going to use NCI and NKP more with their AI workloads.
©2026 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo and all Nutanix product and service names mentioned herein are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).This content reflects an experiment in a test environment. Results, benefits, savings, or other outcomes described depend on a variety of factors including use case, individual requirements, and operating environments, and this publication should not be construed as a promise or obligation to deliver specific outcomes.
