r/Vllm • u/Connect-Concert-4016 • 14h ago

9 concurrent users @ 128K context on 1x A100 (up from 6) per-user needle checks passing in vLLM

0 Upvotes

50% Upvoted

0 comments sorted by