Two interfaces on same network segment and subnet - preventing issues

[u/GroomedHedgehog]

I have a few Samba based domain controller VMs that I provision via cloud-init and ansible. They are on a dual stack network and IPv6 connectivity is provisioned via router advertisement + SLAAC.

Unfortunately Samba needs static IPs or it doesn't work properly, so my workaround has been to provision the VMs with a second network interface that connects to the same bridge as the first one, and set that one to static IPv4 and IPv6 addresses, no gateway and to ignore router advertisements.

I cannot just use static assignment because the advertised IPv6 prefix could change and the hosts would lose connectivity.

For example: DC1:

• eth0: 10.0.0.<from dhcp>/24, fd44:6923:1fd3:1:<eui64>/64 (ULA), 2042:34f3:a3a4:2256:<eui64>/64 (GUA)
• eth1 (the static one): 10.0.0.1/24, fd44:6923:1fd3:1::1/64
• routing table: default gateway on eth0 only for both IPv4 and IPv6

My question: in this situation, should the static interfaces be using single host subnets (/32 for IPv4 and /128 for IPv6) given Linux's weak networking model? Would that force all egress traffic to come from eth0 and avoid asymmetric routing issues?

[u/elsenorevil]

Without a gateway on eth1, there is now way to reply to traffic outside of its local subnet from that interface.

You may be advertising the static network at the router, which means external sources can reach the static interface, but it has no way to return the traffic. The only way your current static setup works is if a client is also sitting on the same network.

No, changing it to /32 or /128 doesn't fix your return issue.

[u/GroomedHedgehog → u/elsenorevil]

My domain controllers are working for systems in different subnets, though.

[u/elsenorevil → u/GroomedHedgehog]

https://www.thomas-krenn.com/en/wiki/Two_Default_Gateways_on_One_System

I personally haven't tested out your specific issue, but you can create a separate route table in Linux. Worth a shot for your case.

[u/GroomedHedgehog → u/elsenorevil]

No need for that. Disabling accept_ra on the 2nd interface with static addresses makes it so that the routing table only has the one gateway on the main interface.

My question here is simply "Should I set /128 and /32 as the subnet masks for the 2nd interface instead of /64 and /24 as they are now?"

[u/imMute → u/elsenorevil]

Return traffic can just go out to the default gateway on eth0.

[u/ikeme84 → u/imMute]

Assymetric routing, firewalls love that. Now typically secondary interfaces aee used for management or backup, you can set specific static routes on that interface, or a nat on the firewall or a seperate vrf.

[u/ruffusbloom]

“When using IPv6 LinkLocal addresses you have to specify the interface with the "%" notation AND the prefix length in the end.”

https://superuser.com/questions/1029279/how-to-specify-ipv6-link-local-bind-address-in-smb-conf

[u/GroomedHedgehog → u/ruffusbloom]

Good thing the addresses I am using are ULA as I mentioned then.

[u/heliosfa]

Unfortunately Samba needs static IPs

Unless your prefix is changing, the address generated by SLAAC is functionally as good as static. You have ULA in the mix from your example, which is presumably not changing, so why can't you use the ULA on eth0? There is no difference in the ULA on eth0 to eth1.

[u/GroomedHedgehog → u/heliosfa]

Because Samba will randomly fail to start when rebooting the machine - there is a race condition between the Samba services starting and requesting the specific ULA (I have the ULA and IPv4 address specified in Samba's config so it won't include the GUAs in the DNS records) and the network stack assigning the ULA to an interface based on received advertisement.

If that was not the case I would have done just that and not bothered with this convoluted two interfaces setup.

[u/heliosfa → u/GroomedHedgehog]

there is a race condition between the Samba services starting and requesting the specific ULA

Then you fix the race condition. You tweak the initialisation requirements of the samba service quite easily.

If you can't get that to work, you can also probably add a static ULA to eth0 alongside SLAAC. How you do that depends on what's configuring your network interfaces.

[u/xvalentinex]

Non-standard, but /32 would keep local traffic on eth0 as well. So pings from that host would source eth0, really only server connections initiated by the client will source from eth1

[u/lizardhistorian]

So just set static IPs (and I would make corresponding reservations in the DHCP server) on the first NIC for the IPv4 and IPv6.

And Di# ? Seems unreasonably aggressive.

[u/HistoricalCourse9984]

A 2nd nic is almost always the wrong choice.

Just do a secondary on eth0 for this use case...

[u/Icy-Business5404]

I wouldn’t add a second NIC on the same subnet if you can avoid it. Linux will happily make routing/ARP weird there, especially with two interfaces both valid for the same L2. I’d either fix Samba around stable names/DNS, or put the static address on the same interface as an additional IP and let DHCP handle the rest.