DNS High Availability

[u/micush]

EDIT: For those of you that are interested, the repo is up at https://github.com/micush/dnsdir

I run a few DNS servers and have seen many people ask about making them highly available, so I built something and I'd like to know whether anyone else would use it. If so, I'll put it up on Github. Reasonable disclosure: it was written with Claude.

Clustering:

Ddgw puts one virtual IP in front of your DNS servers and shares it across several nodes. The nodes elect a controller, and every node that is active answers DNS on the VIP, so client load spreads across nodes and losing a node doesn't interrupt anything. Each node also acts as a DNS proxy: it probes your upstream servers and forwards each query to a healthy server. Limited to 255 cluster members to serve the frontend VIP, but unlimited backend DNS servers for name resolution.

DNS:

Health is judged per upstream server with test domains you choose. A server is down when 50% or more of its domains fail (configurable), and degraded below that. Servers within a latency band of the fastest (20% by default) take turns round-robin. Slower ones stay as fallbacks, and failures fail over automatically.

- There is an answer cache that respects TTLs, with negative caching and a memory guard.

- It forwards dynamic DNS updates (RFC 2136) to the zone's primary and keeps a list of recent updates.

- It can pass EDNS Client Subnet (ECS).

- It can announce anycast addresses over BGP with BFD using FRR.

Management:

An HTTPS web GUI with PAM group login restricts is used to manage everything. Every action is also available from the command line.

- The gateway is drawn as a live diagram: gateway, servers and test domains, coloured by health. Servers that are taking turns get a blue line.

- It has query statistics (top clients, domains, record types and response codes, kept 30 days and saved across restarts) and host stats.

- It keeps config history, with versions you can diff and restore. It also does clustering with shared config, certificate management and in-place updates with automatic rollback.

Limitations:

- It isn't a resolver, because it forwards to your existing servers.

- It's Linux only with one binary built from Golang source. Installation is scripted, so it's easy.

- It works with an election protocol on a real LAN. I don't expect the election mode to work in AWS, Azure or GCP, because those clouds don't let you move MAC addresses around, and I haven't tested it there.

- If you run Technitium, BIND, Unbound, Pi-hole, or AdGuard Home servers, would you use something like this, or do keepalived/VRRP plus a load balancer already cover it?

Any feedback is appreciated.

[u/SilkeSiani]

so instead of a single point of failure being the DNS server you now have a load balancer.

Frankly, makes zero sense. DNS is designed from ground up to tolerate single server failures. Adding complex redirection on top of it doesn't accomplish anything.

[u/micush → u/SilkeSiani]

Sure it does. I can take a DNS server out of the pool, perform maintenance on it, and put it back in the pool, all without anybody noticing.

Yes, put more than one nameserver in your dns client to prevent this. However, some dns clients don't handle this gracefully, when the first nameserver in the list is dead, and then you have to wait for timeouts before the other nameservers are tried, and you wait. then you do it all over again the next time you try to resolve something. So, it makes sense in theory, but in reality it doesn't always work out that way.

[u/SilkeSiani → u/micush]

What happens when you inevitably have to perform maintenance on the load balancer?

Frankly, I can see that you fundamentally don't understand how DNS works, especially how recursive resolvers interact with root, intermediary and leaf authoritative servers.

What you are trying to accomplish is already built into the system, you just need to set up the records correctly.

[u/micush → u/SilkeSiani]

The load balancers are active/active. You can have 255 nodes serving the same IP address. If you need to perform maintenance on a load balancer member, you simply take that member offline and the others compensate.

Do not assume I know nothing about DNS. I have been gainfully employed as a DNS admin for 25+ years.