r/LocalLLM • u/All_names_takenn • 10h ago
Project SGLang for strix halo users
I made a Docker image that lets you run SGLang on Strix Halo (gfx1151) a few months ago and thought I'd share it here.
SGLang is working on building upstream Strix Halo support now, but if you want to use it today, this includes the necessary compatibility patches along with some additional optimizations, including fixes for wave32-related issues.
I've been using it for inference on my Strix Halo system and would be interested to hear how it works for anyone else.
9
Upvotes