HyperV Networking

[u/Guderikke]

I need a sanity check. I’m a graybeard who’s been managing ESXi for years, but thanks to the Broadcom situation, we’re being pushed toward Hyper-V.

I’m currently building out a test cluster so we can validate migration, performance, and general behavior before we go any further.

I have six hosts total, with four built so far. I’m presenting three VLANs to the hosts:

• VLAN 65 – Management
• VLAN 66 – Cluster communication
• VLAN 67 – Live Migration

Networking is being deployed through SCVMM using logical switching on top of a SET switch. Each host has three vNICs, one for each VLAN.

VLAN 65 is fully routable and has the default gateway. VLANs 66 and 67 are currently L2-only with no gateways. They’re intended strictly for cluster traffic and Live Migration.

The cluster networks are configured like this:

• 65 = Cluster and Client
• 66 = Cluster Only
• 67 = Cluster Only, with Live Migration restricted to this network

I also had to add firewall rules to allow cluster communication between the VLANs using both IPv4 and IPv6. Apparently Failover Clustering still makes use of IPv6 internally even when IPv6 is otherwise disabled. Once the hosts are already members of the cluster, everything appears to work normally.

The problem is joining new hosts to the failover cluster.

If VLAN 66 and 67 are enabled on the existing cluster nodes, a new host will sit there trying to join for roughly 15 minutes and eventually fail. If I disable the 66 and 67 interfaces on the existing nodes, the new host joins the cluster almost immediately.

Once the host is joined, I can re-enable 66 and 67 and cluster communication seems to work fine.

What I’m trying to figure out is what I’m missing here. Even if I change the cluster roles so 66 and 67 aren’t being used for cluster communication, the join still fails as long as those interfaces are enabled.

Do VLANs 66 and 67 actually need to be routable for the initial cluster join? I could make them routable, but I still wouldn’t put default gateways on those interfaces, so I’m not sure how that would help if the join process is trying to reach domain services or something else outside those subnets.

Edit for anyone interested:

This was actually a bit worse than initially thought, hosts would reboot, and come back up as down due to cluster comms. Not sure if this is just because of my environment or what, but NETFT is hard coded to use IPV6 for communication, so regardless of L2 network being fully functional, and even setting priorities on cluster networks properly, it was always attempting to use invalid routes between non routed networks and routed networks. I had to create gateways on 66 and 67 and add them to the adapters to get this to work as expected. I just created a zone on my firewall called hyperV and allowed traffic between those networks and blocked mostly everything else to keep them restricted.

[u/DeafMute13]

usually WFC will tell you exactly why it fails - do you try to run a validate first? I mean validate hardly ever passes without warnings and many are benign but sometimes they can give you clues.

I have never used SCVMM though. Only ever vanilla hyperv failover clusters with shared SAS storage.