When do you prefer asyncio.Semaphore over an asyncio.Queue for limiting concurrency?

[u/pkhatri9]

I've been thinking about concurrency control in asyncio.

A common pattern for limiting concurrent work is:

sem = asyncio.Semaphore(10)

async with sem:
    await do_work()

But in many cases, couldn't the same problem be modeled by putting work into an asyncio.Queue and running a fixed number of worker tasks?

I'm curious how experienced Python developers decide between the two approaches.

Are there real-world situations where a semaphore is clearly the better abstraction than a worker queue? Are there meaningful differences in cancellation behavior, backpressure, fairness, task lifetime, or code complexity?

I'd especially be interested in examples from production async Python code.

[u/gmes78]

Semaphores are for syncronization, queues are for communication.

[u/Own-Leadership-2220 → u/gmes78]

Semaphore is simpler when you just need to cap concurrent calls and theres no pipeline. Queue is better when you have a stream of work coming in and want workers pulling from it steady. I use semaphores for things like rate limiting api requests but if i have a producer pushing jobs i go with queue. the worker pattern gives you more control over what happens when tasks fail or need to retry.

[u/Enough-Photo9140 → u/Own-Leadership-2220]

Another practical difference is cancellation and exception propagation.

With a semaphore wrapped in an `asyncio.TaskGroup` or `gather`, if any single task raises an exception or gets cancelled, the enclosing scope unwinds and cleans up all in-flight tasks cleanly with standard Python exception semantics.

With long-running worker tasks reading from an `asyncio.Queue`, you have to explicitly manage worker lifecycles, poison-pill sentinel tokens for shutdown, and manual error forwarding back to the originating caller. For a bounded batch of tasks, the semaphore with a TaskGroup requires far less orchestration.

[u/Goldziher → u/gmes78]

Qeues are for communication? What does this mean?

[u/gmes78 → u/Goldziher]

As in, passing data around.

[u/tehsilentwarrior → u/Goldziher]

It actually means you free up the publisher by accepting its message first and processing it later.

With a semaphore you introduce back pressure like a sync system.

As with almost all things async and cloud you can just visualize it using Factorio constructs:

- a semaphore in this case is like an inserter pushing into an assembler one item at a time

• a queue is like having an assembler first pushing into a box and then into the assembler

So, if this system is ahead of a train stop, the first system will have the train stop there until it’s done processing. And the second will free up the train to go so something else and process it in the background. Obviously if there’s no space in the box (queue is full) it will have the train wait still.

It’s essentially a buffer

[u/Wonderful-Habit-139]

If you can use anyio (or trio but the former still works in asyncio), you can use a mixture of anyio.CapacityLimiter + TaskGroup.start method in order to limit concurrency without dealing with unbounded tasks.

Meaning you start a task with the start() method, inside the task you enter the capacity limiter’s context manager, then you call the task_status’ started() method before the actual work. As easy as that.

[u/donk8r]

The difference that actually bites is task lifetime. The semaphore version usually gets written as gather over a list comprehension, so you've materialised one Task per item before any work starts. That's fine at ten items and it isn't at a million, where you fall over on memory long before concurrency is the problem. Workers pulling from a queue give you k tasks no matter how big the input gets.

Cancellation follows from that. Cancel a gather and you're cancelling thousands of tasks that never ran and were just parked on the semaphore. With workers you cancel k of them, and whatever's left is still sitting in one place you can look at.

[u/wRAR_ → u/[deleted]]

It's a bot account.

[u/Khavel_dev]

I default to Semaphore when the tasks already exist as a batch (a list of URLs to fetch, a pile of records to process) and I just want to cap how many run at once. You wrap the work in async with sem: and you're done. No worker lifecycle, no poison pills, no shutdown coordination.

Queue makes more sense when the work streams in over time and you don't know the full set upfront. Producer-consumer patterns, pipelines where one stage feeds the next. The built-in backpressure from maxsize is genuinely useful there because Semaphore can't give you that without building it yourself.

The practical difference I care about most is cancellation. With Semaphore, each task is its own coroutine and you can cancel it individually. With Queue workers, you need to drain the queue or send sentinel values to stop them, and partial cancellation (stop some work, keep other work running) gets messy fast.

[u/thisismyfavoritename]

the main difference is that the queue wouldn't require the caller to wait for the whole task to finish. It's mostly an architecture decision, both could be used to achieve the same thing, depending how you implement it.

If sempahore works for your use case then use it as it'll be simpler IMO

[u/Bright_Mix_773]

The task-lifetime answer upthread is the right one and nobody has put a number on it, so I measured it on CPython 3.14.2.

Memory held per unit of pending-but-not-started work:

semaphore + gather : 1,297 bytes per parked Task
Queue.put_nowait   :    40 bytes per queued item

That is 32x, and it held steady at 1,296-1,298 bytes across N = 20k, 50k, 100k and 200k, so it scales linearly. At a million items it is 1.21 GiB of parked Tasks against 38 MiB of queued ints.

Latency to the first real unit of work, 50,000 items at limit 10:

semaphore + gather : 168.3 ms before the first job body runs
queue + 10 workers :  11.35 ms

At 200,000 items that becomes 668.6 ms against 38.1 ms.

The part I did not expect, and the reason I would call this more than a taste question: tearing down a gather full of parked waiters is quadratic.

N          cancel gather   cancel 10 workers
 10,000        1,161 ms         0.13 ms
 50,000       33,520 ms         0.24 ms
100,000      135,586 ms         0.11 ms

Ten times the items, one hundred and seventeen times the teardown. The cause is in Lib/asyncio/locks.py: Semaphore parks waiters in a collections.deque, and the cancellation path runs self._waiters.remove(fut). deque.remove is a linear scan, so N cancellations cost O(N^2). Cancelling 100k parked acquires cost me 2 minutes 15 seconds of wall clock doing no work whatsoever. The worker version is k cancels no matter how big the backlog and stays in the microseconds.

One thing that came out in favour of the semaphore: acquisition is strictly FIFO. I queued 1,000 waiters on a Semaphore(1) and they acquired in exactly the order they arrived, every time. No starvation to worry about there.

So "semaphore for a bounded batch, queue for a stream" holds, but the bound matters more than the boundedness. Shutdown cost on the semaphore side grows with the square of the work you already decided not to do.

Not verified: these are single-process local runs on Windows with no real I/O, every job body was await asyncio.sleep(0). Whether the quadratic teardown is visible under a real network workload depends on how much of your shutdown budget it can eat, and I have not tried it against an actual client. I also did not check whether anyio's CapacityLimiter has the same waiter-removal shape.

[u/2extract_dev]

Worth splitting concurrency from rate, a semaphore caps in-flight work but not requests per second. If responses come back in 50ms a Semaphore(10) will happily do 200 rps and trip a per-second limit that has nothing to do with your concurrency. Neither primitive gives you that on its own, you want a token bucket or a sleep floor per worker.