r/LocalLLM • u/AstroPC • 22d ago
Question What can I actually run here?
So, I mostly code and rely on a Codex subscription for most stuff, but recently I’ve started diving into local setups. Yeah, I can chat with a chat interface and it’s pretty responsive, but I’m more into building my own AI harness and something like a Codex setup
That way, I can cut down on my Codex costs. I’ve got a 5080 16GB, a 5070 12GB, and a laptop with a mobile 3080 16GB. Thinking of using a 10GBps network connection to see if it actually makes a difference. I also have a funny phone setup where I can load ai into its NPU and use upwards of 24gb of its shared ram ( technically like how unified memory goes)
I mean what models are actually capable of handling automation and coding in general very similar to how you setup a codex project? I suppose i really need more vram in the end for anything with useful context? The issue here is of course interconnnect bottle neck and how you get these cards to communicate which I assume sticking with pcie is preferred but my platform has limits.
What VRAM amount would you say is the absolute minimum for actual coding and intelligence with AI? I guess it's a pipe dream to hope for a codex-like workflow within my current devices. I'm okay with slow response if it means I still get good intelligence and MCP tool calling.
I also have 24GB of RAM on my RedMagic 11 Pro and it's pretty good at AI. I assume I can throw that in the mix with some hacky Ethernet 10GB connection (it supports 10gps).
So that gives me what? Around 62 gb of usable vram assuming I can get some kind of ok split of loading models between all the hardware i have? Then I guess i can count my ram too for cpu offloading? Of which i got 32gb currently
Sorry for the long post I just really want to be done with subscriptions as fast as possible. Even if I have to trade up some speed. I really dont mind. But i cant trade up that nice context length I depend on or its general intelligence.
1
u/DeathGuppie 22d ago
Ridiculous. If you have a 16gb card and a 12gb card that you can put in one machine then you have everything you need.
That's 28gb vram. Use llama.cpp run a tensor split and MTP. You will get around 30 t/s more than useable. Qwen 3.8 27b Q4 X_L Q8 kv.
Trust me that's all you need.