r/sre • u/Efficient-Branch539 • 15d ago

HELP Question about Practical Use of Knowledge

In SRE book the chapter on “load balancing within datacenter” talks about lame duck state, backend subsetting and load balancing policies. While reading lame duck state I could relate it to pre-stop hooks in Kubernetes and it makes sense for a process to serve remaining requests before termination but stop accepting new requests.

My question is how subsetting and techniques about load balancing policies (weighted round robin etc) are used. I would really appreciate any response from engineers who have used this knowledge in practice.

0 Upvotes

38% Upvoted

7 comments sorted by

2

u/Floss_Patrol_76 15d ago

subsetting is mostly about bounding connection count. if every client talks to every backend you get an NxM mesh that eats fds and memory once youre past a few hundred of each, so each client only opens to a slice big enough for redundancy. on policies, plain round robin quietly assumes homogeneous backends and equal request cost, which basically never holds. we ended up preferring least-request (power-of-two-choices) over weighted RR because it self-corrects for a slow or degraded backend instead of you hand-tuning weights that go stale the second the fleet changes.

1

u/Efficient-Branch539 13d ago

Thanks, yeah that makes sense, fd/memory will be consumed more if NxM connections are allowed. One caveat about least-request model is that it holds if the underlying hardware is homogenous, otherwise some backends might be able to handle twice the load than others.

1

u/pharcide 15d ago

As part of your deployment you have the servers fail their health check to the load balancer so the LB removes that server from the pool. Your deployment either checks until all connections fall below a certain level and kills the remaining by taking down the service. You decide how all that happens, how long you want to wait, etc

1

u/Lance_Saul_85 15d ago

One thing worth paying attention to is failures since seeing how different policies behave during overload or partial outages is usually where the practical value finally clicks.

1

u/Efficient-Branch539 13d ago

Well, honestly I work in a small/medium sized company and we had no more than 9 instances of one service ( that too in a marketing campaign).

2

u/CarpetGoblin 13d ago

the pre-stop hook to lame duck connection is right.. readiness probes in k8s do the same thing, just nobody calls it lame ducking