Reverse Tunnel Architecture
This is the final part of our deep dive into reverse tunnel architecture. In Part 2, we covered how downstream Envoy initiates reverse tunnels through custom listeners and IOHandles. In Part 3, we examined how the upstream Envoy accepts, validates, and manages these tunnels through the reverse tunnel filter and socket manager. Now, let's bring it all together by tracing how an actual data request flows through these established tunnels.
Part 4: Life of a Data Request
Understanding how reverse tunnels are established and managed provides the foundation for examining how an actual data request flows from a cloud service to an on prem service through these tunnels.
Configuring the Egress Path
On the upstream Envoy (in the cloud), we need to configure how data requests will be routed to downstream nodes. This involves setting up an egress listener and a special type of cluster called a reverse connection cluster.
The egress listener accepts requests from cloud services and routes them based on request headers. Here's an example egress configuration.
The key piece here is the Lua filter that examines request headers and computes a host_id that identifies which downstream node or cluster should handle the request. The filter checks headers in priority order:
- x-node-id: If present, routes to a specific downstream node
- x-cluster-id: If present, routes to any node in that cluster (load balanced)
This computed host_id is stored in the x-computed-host-id header, which the reverse connection cluster will use for routing decisions.
Life of a Data request through reverse tunnels
The Reverse Connection Cluster
The reverse connection cluster is fundamentally different from standard Envoy clusters. Instead of creating outbound connections, it reuses pre-established reverse tunnel connections that were initiated by downstream nodes. As described in the Envoy documentation, the cluster configuration includes a host_id_format field that extracts the target node or cluster identifier from each request—this becomes the key for finding the right tunnel.
Step 1: Computing the Host ID
A cloud service sends this request:
When this request arrives at the reverse connection cluster, it evaluates the host_id_format configuration (%REQ(x-computed-host-id)%) against the request context. The Lua filter has already preprocessed the request, examining the x-node-id header and setting x-computed-host-id: on-prem-node-42. The cluster extracts this value as the target identifier. If the request had specified x-cluster-id: on-prem-cluster-west instead, the filter would route to any node in that cluster.
Step 2: Creating the UpstreamReverseConnectionAddress
Once the cluster has extracted on-prem-node_42 as the host ID, it needs to create an Envoy Host object representing that downstream node. But there's no IP address for this node—it's behind a firewall. So instead of a normal network address like 192.168.1.100:8080, the cluster creates a logical UpstreamReverseConnectionAddress that encodes the string on-prem-node-42.
This special address type implements the same interface as regular addresses, but its socketInterface() method returns the custom UpstreamReverseSocketInterface. This substitution allows Envoy's connection pooling logic to proceed normally while replacing actual TCP connections with reverse tunnel connections. The host is stored in a map indexed by on-prem-node-42, enabling reuse across multiple requests—critical for connection pooling and HTTP/2 multiplexing.
Step 3: Connecting Through the UpstreamReverseSocketInterface
When Envoy's connection pool tries to establish a connection to the UpstreamReverseConnectionAddress for on-prem-node-42, the UpstreamReverseSocketInterface intercepts the call and queries the socket manager for an available reverse tunnel connection to that node ID.
The socket manager checks its pools of idle connections and finds on-prem-node-42 with available sockets. It returns one from the pool, and ownership transfers from the reverse tunnel implementation to Envoy's standard connection pool. The socket transitions from "idle" to "used" state. The connection pool wraps the socket with HTTP/2 codec and opens a new stream carrying the /downstream_service/api/status request over the existing tunnel.
Step 4: Connection Reuse and Multiplexing
This first request to on-prem-node-42 creates a Host object and retrieves an idle tunnel socket. But that Host persists in the cluster's host map. When a second request arrives for the same node—say, GET /downstream_service/api/health—it resolves to the same Host and shares its connection pool.
Because HTTP/2 multiplexing is used, that single tunnel connection carries both requests as separate streams. The second request doesn't retrieve a new socket—it opens a new HTTP/2 stream on the existing connection. Even with thousands of concurrent requests to on-prem-node-42, you only need one reverse connection, as permitted by the max streams per connection limit set in Envoy.
The request travels down the reverse tunnel connection (established from on premises to cloud, but now carrying data in the opposite direction). The on premises Envoy receives the HTTP/2 stream, processes it through its filter chain, and forwards it to the local downstream_service. The response travels back through the same tunnel, and the cloud service receives it—completely unaware it traversed a reverse tunnel through a firewall.
This is the power of the reverse tunnel design: it inverts network connectivity assumptions while preserving Envoy's standard request processing pipeline, making services behind NATs and firewalls as accessible as any other upstream service.
References
©2026 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo and all Nutanix product and service names mentioned are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names mentioned are for identification purposes only and may be the trademarks of their respective holder(s). Code samples and snippets that appear in this content are unofficial, are unsupported, and are provided AS IS Nutanix makes no representations or warranties of any kind, express or implied, as to the operation or content of the code samples, snippets and/or methods. Nutanix expressly disclaims all other guarantees, warranties, conditions and representations of any kind, either express or implied, and whether arising under any statute, law, commercial use or otherwise, including implied warranties of merchantability, fitness for a particular purpose, title and non-infringement therein.
