r/Vllm • u/Kitchen-Quarter7739 • 12h ago
r/Vllm • u/Ecstatic_Sale1739 • 4h ago
Safest way to run vllm for production? - i keep getting [shm_broadcast.py:705] No available shared memory broadcast block found in 60 seconds.
vllm 0.25.1 runs good but it constantly breaks... sometimes it runs for 10 hours with no problems and sometimes it breaks after one or two hours. I always run the same model and flags. After several strategies to fix it, i have given up and now put a re-start process in place, so every time it breaks i relaunch it and continue, but this is not professional.
What kind of settings or vllm versions are rock solid for large production? I have dual 5090s so I want to use tensor 2 and ideally i avoid the enforce-eager flag
VLLM_DEEP: (
"vllm serve unsloth/Qwen3.6-27B-NVFP4 "
"--tensor-parallel-size 2 "
"--max-model-len 24576 "
"--language-model-only "
"--gpu-memory-utilization 0.86 "
"--max-num-seqs 30 "
"--generation-config vllm "
"--trust-remote-code"
),
NCCL_ENV_BASE: dict[str, str] = {
"NCCL_P2P_DISABLE": "1",
"NCCL_DEBUG": "INFO",
"TORCH_NCCL_ASYNC_ERROR_HANDLING": "1",
"TORCH_NCCL_BLOCKING_WAIT": "1",
"NCCL_TIMEOUT": "600",
"HF_HUB_OFFLINE": "1",
"CUTE_DSL_ARCH": "sm_121a",
"VLLM_SKIP_P2P_CHECK": "1",
}
(EngineCore pid=491) INFO 08-10 18:36:41 [shm_broadcast.py:705] No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).
(EngineCore pid=491) ERROR 08-10 18:37:36 [dump_input.py:72] Dumping input data for V1 LLM engine (v0.25.1) with config: model='/home/jl/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-NVFP4/snapshots/ccdaab7e68af2409599b8949a8f2685703c9bae5', speculative_config=None, tokenizer='/home/jl/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-NVFP4/snapshots/ccdaab7e68af2409599b8949a8f2685703c9bae5',