One Subnet, One Path: How Nutanix Flow Keeps VM Migration Traffic From Splitting
By:
Amin Aflatoonian - Senior Product Manager
Anuj Gupta - Member of Technical Staff
Stephen Martin - Global Practice Expert, Network & Security
Migrating VMs from VMware by Broadcom to the Nutanix AHV hypervisor usually means keeping the same subnet alive in both places at once. That's the point of a subnet extension: the VMs can retain their IP addresses, helping reduce application reconfiguration and allowing the migration to happen gradually instead of all at once.
But it can create a problem that's easy to miss until you're staring at a routing table that doesn't make sense: for the length of the migration, that one subnet now has two possible ways out to the rest of the network. And a subnet with two active gateways isn't a safer subnet, it's a race condition.
This post walks through how Nutanix Flow is designed to help avoid that, using a real VMware-to-AHV migration as the example, with the actual routes learned by the physical network before, during, and after the cutover.
The Problem: Subnet Extension Gives You Two Doors, Not One
When a subnet is extended between a VMware/NSX-T environment and a Nutanix AHV cluster, the same Layer 2 network exists in both places simultaneously. That's necessary, it's what lets a VM move without changing its IP, but it also means two gateways are now capable of routing traffic for that subnet: the NSX-T T0/T1 gateway on the VMware side, and the VPC gateway on the AHV side.
If both gateways are active and advertising routes for that subnet at the same time, the physical network may not have a reliable way to know which path is correct. Depending on routing priorities and timing, traffic can end up split across both paths, or worse, sent toward a gateway that silently drops it. That's a routing blackhole, and it's the kind of intermittent, hard-to-reproduce problem that's miserable to troubleshoot in the middle of a live migration.
What We Actually Need During a Migration
The requirement is simple to state and, until now, harder to manage: VMs need to stay reachable throughout the migration, but only one gateway should ever be the active path for that subnet at a given moment. Not “mostly one path.” The workflow should be designed to support a single active path, minimizing windows where both are live.
That's what disconnected subnets, GA in Prism Central version 7.6, are built for.
Two Controls, Two Different Jobs
This workflow uses two related but independent controls:
1. Subnet connection state (per subnet):
Controls whether the subnet is attached to the VPC.
- 'Connected': local route exists and subnet is active in VPC routing.
- 'Disconnected': no local VPC route for that subnet. Its VMs lose VPC east-west reachability and VPC north-south egress.
2. Advertise Connected Subnets (per VPC):
Controls what is advertised externally.
- disabled (by default): ERP-based advertisement behavior
- enabled: advertise only connected ERP subnets (i.e., VPC subnet prefixes that are connected and within ERP scope)
In short: Subnet connection state controls subnet reachability inside the VPC.
Advertise Connected Subnets setting controls subnet visibility outside the VPC.
In this blog, we will walk through the process of migrating two subnets (Prod-100 - 172.16.100.0/24 and Prod-200 - 172.16.200.0/24) from a VMware NSX environment to a Nutanix Flow environment.
Phase 1: Before the Move
Figure 1: A simplified network diagram with an NSX environment and a Flow Virtual Networking environment, with the Flow Virtual Networking environment subnets in a disconnected state. Each environment has two networks (Prod-100 and Prod-200) VXLAN tunnels connect the two networks. BGP state is displayed, indicating that the NSX environment is currently advertising the two networks.
At this point, every VM on the subnet is still running on VMware, and all traffic, north-south and east-west, egresses through the NSX-T T0/T1 gateway. That part is unchanged from any pre-migration environment.
On the Nutanix side, the migration domain is prepared with a one-time VPC setup:
The Nutanix VPC has been configured with both NAT and NoNAT External Networks. The NoNAT External Network is the default route for the VPC. The “Advertise Connected Subnets” setting is enabled on the migration VPC, and the ERPs have been defined as a supernet (for example, '/16') that covers all '/24' subnets planned for migration.
Figure 2: A screenshot of the VPC creation workflow from Prism Central. Prod-VPC is being created, configured with both NAT and NoNAT external networks. The NoNAT network is the default route. The Externally Routable Prefix field is set to 172.16.0.0/16.
A BGP Gateway has been deployed to act as a route server on behalf of the VPC and is peered with the network infrastructure. If preferred, multiple BGP Gateways can be deployed for high availability.
Overlay Subnets Prod-100 and Prod-200 have been created, matching their NSX equivalents' addressing. Both subnets have been marked as disconnected. As a result, the BGP Gateway does not advertise those prefixes.
Figure 3: The Prism Central Subnet Creation workflow, showing the creation of Prod-100. The Connection State is set to Disconnected. The IP Assignment Service is Nutanix IPAM, with a Network IP Address /Prefix of 172.16.100.0/24 and a Gateway IP Address of 172.16.100.1.
Figure 4: The Prism Central Subnet List interface, showing Prod-100 and Prod-200. Both subnets are marked as Disconnected.
A VTEP Gateway has been deployed in the Nutanix VPC on the automatically created Nutanix-internal-vpn overlay subnet. A Floating IP of 10.10.10.50 has been assigned out of the NAT External Network’s IP pool to the VTEP Gateway's address on this subnet. A Network Policy is created to policy route the VTEP Gateway’s traffic through the NAT External Network.
On the NSX side, a network segment called NSX-VTEP has been created to act as a service network for VXLAN traffic.
An NSX MAC Discovery Profile is created allowing MAC Changes, MAC Learning, and Unknown Unicast Flooding. This MAC Discovery Profile is assigned to the Prod-100 and Prod-200 NSX Segments.
A VyOS Virtual Router is deployed to act as a VTEP in the VMware environment with its primary vNIC attached to the NSX-VTEP network, and additional vNICs connected to Prod-100 and Prod-200. A sample configuration for the virtual router is shown below. Connectivity on UDP-4789 is allowed between the VyOS Virtual Router's address on NSX-VTEP and the VTEP Gateway's Floating IP. However, the network infrastructure between the VMware and Nutanix environments also utilizes VXLAN, an alternate port should be selected.
Figure 5: An example VyOS configuration for the example environment, with notes explaining what the various lines are for.
The VyOS Virtual Router is added as a Remote Gateway in the Prism Central management console. Subnet Extensions are configured for both Prod-100 and Prod-200.
Test VMs are provisioned on the Nutanix Prod-100 and Prod-200 Overlay Subnets, allowing for the Subnet Extension to be tested and validated prior to migrating VMs.
Phase 2: VM Migrations Begin
With the subnet extension in place, VMs begin moving from VMware to AHV. The subnet extension keeps Layer 2 connectivity intact across both environments, so a VM that's moved to AHV can still be reached, and can still reach the rest of the network, exactly as it could before.
The AHV-side subnet is still disconnected at this point, even though it may now be hosting live VMs. Traffic from those VMs still egresses through the extension and out via the NSX-T T0/T1 gateway, because the subnet is still disconnected, the Nutanix-side VPC path for that subnet is not active, and the Nutanix BGP Gateway is not advertising that subnet prefix. There's no ambiguity for the physical network to resolve, because there's still only one gateway in the conversation.
This is the phase where the design pays off: you can move VMs incrementally, at whatever pace makes sense, without the routing picture changing underneath you.
Phase 3: Network Migrations Begin
Figure 6: A simplified network diagram with an NSX environment and a Flow Virtual Networking environment, showing the next stage of the migration process. The Prod-100 network has been disconnected from the NSX T0/T1 and connected to the Flow Virtual Networking VPC Logical Router. BGP state indicates that the Prod-100 subnet is now being advertised by Flow Virtual Networking.
Once a given network has been sufficiently migrated, generally once the majority of traffic is now originating from the Nutanix side, the network itself can be migrated. In our example, Prod-100 workloads are now sufficiently migrated, and so we can migrate egress of the network from NSX to Nutanix.
From NSX Manager, Prod-100's connection to the T1 Gateway is removed. This causes NSX to stop advertising 172.16.100.0/24.
On Nutanix, Prod-100 is updated to a connected state (to the VPC). This causes the BGP Gateway to no longer withhold the route, and advertisement for 172.16.100.0/24 starts automatically. Traffic for the Prod-100 network is now directed to Nutanix, and VMs remaining on the VMware instance of Prod-100 will utilize the Nutanix VPC for network egress.
Once all Prod-100 VMs have been migrated, the subnet extension and NSX instance for that network can be decommissioned.
Eventually, once Prod-200 reaches the same state, these steps can be repeated.
Figure 7: A simplified network diagram with an NSX environment and a Flow Virtual Networking environment, showing the next stage of the migration process. The Prod-100 subnet has been deleted from the NSX environment. The Prod-200 network has been disconnected from the NSX T0/T1 and connected to the Flow Virtual Networking VPC Logical Router. BGP state indicates that the Prod-100 and Prod-200 subnets are now being advertised by Flow Virtual Networking.
Phase 4: After migration
Figure 8: A simplified network diagram with an NSX environment and a Flow Virtual Networking environment, showing the result of the migration process. The Prod-100 and Prod-200 subnets have been removed from the NSX environment and have been connected to the Flow Virtual Networking environment. The VXLAN tunnels, the VTEP Gateway, and the VyOS Virtual Router have been removed.
With all migrations complete, the Subnet Extensions, VTEP Gateway, and Remote Gateway can be decommissioned. The Nutanix-Internal-vpn network can also be deleted; it will be recreated if it is ever needed in the future. If the NAT External Network was only provisioned to allow for a Floating IP to be provisioned for the VTEP Gateway, it can also be decommissioned.
The VMware environment can be decommissioned at this time.
Beyond VMware to AHV
This same mechanism isn't specific to VMware migrations. Anywhere a subnet temporarily needs to exist in two routing domains at once, including a subnet extension between two Nutanix clusters, connected/disconnected state is what keeps those two domains from fighting over the same traffic. The VMware to AHV migration case is simply the clearest way to see it in action, because the “other side” is a gateway you can watch stop advertising in real time.
The Takeaway
A migration shouldn't force a choice between “move everything at once” and “risk a routing blackhole.” Disconnected subnets let you extend a network, migrate at your own pace, and cut over with a single action that keeps exactly one path active at every point along the way, designed to help avoid blackholes, minimize manual route cleanup, and reduce guessing which gateway is authoritative today.
©2026 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo and all Nutanix product and service names mentioned herein are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. Nutanix, Inc. is not affiliated with VMware by Broadcom or Broadcom. VMware and the VMware product names recited herein are registered or unregistered trademarks of Broadcom in the United States and/or other countries.All other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).This content reflects an experiment in a test environment. Results, benefits, savings, or other outcomes described depend on a variety of factors including use case, individual requirements, and operating environments, and this publication should not be construed as a promise or obligation to deliver specific outcomes.
