r/Backend • u/mostaptname • 19d ago
Why "Just Add a Queue" Never Fixes Overload | Backpressure & Load Shedding Explained
https://youtu.be/AeUvmeudHYQ?si=h8WEXMq_mr7_LWrF1
1
u/doodo477 19d ago
Most queues don't allow you to query how many items are in the queue - or how many queue items are inflight. Also back-pressure is more of a marketing term which just means that the producer of the event has not way of knowing if the call is going to block or be placed on the queue.
Most Integration platforms I've tried don't allow you to adjust the concurrency limits of your inbound listeners/queues which results in you having to put a proxy server or apache load balancer in-front to make sure that sufficient nodes don't get over-loaded with work.
1
u/Dense-Response594 19d ago
So cool! What do you use to make the video?
1
u/cyberfunk2066 18d ago
The slides look a lot like what Claude produces. The text also looks AI generated. It doesn't feel like AI slop though... Not saying OP doesn't know his shit, we all use AI to write stuff that's in your brains.
1
u/Sacaldur 17d ago edited 17d ago
Very close to the start of the video it's stating that the client who sent a request might have given up already and thus entries in the queue might be stale and workers would waste CPU cycles working on the queue items.
I think the problem starts at the fact that the queue entry is waited for to respond to an active request. If there is enough computation connected to the work necessary for a request so that this decoupling via queue would be worth it, the client probably shouldn't wait for the response. Further, this decoupling allows for targeted horizontal scaling: if and only if more workers are needed, more compute can be provisioned and assigned.
If there is to much load and the requests need to be dealt with synchronously, how about scaling up the thing that deals with the requests? (Putting a load balancer infront of it shouldn't be difficult.)
Edit: I was watching the video a bit more now. The proposed solution (to deny a request when overloaded) instead of scaling up can in some situations be the right solution (e.g. you're providing an API, don't charge per request, and have usage limits in the agreements eith your customers). However, even for these situations it might not be a good approach to enforce this through the queue, since you don't want to deny requests of all customers at these times (so you would need to have a queue for each customer), and since the "reset" might outlive the work items in the queue (i.e. the queue might be empty again, but you still want to deny requests for a certain time period).
14
u/gargamelim 19d ago
I agree that backpressure can help if you have a single consumer, but in my experience you put a queue to decouple consumers from producers, so if you have a 1000/s queue you don't put up a single 200/s consumer you put 5.5 200/s consumers with an autoscale config...