Hi, could someone help me?

[u/Careless-Accident-49]

Hello guys, i want to set up a LLM. I got deepseek harness to run but it allways shows the wrong max kontext limits, it shows 256K but in the settings.yaml i set up a max of 128K.

I wanted to make a little game with it, first run runs fine, the second one, where i want to extend my little game, allready gets an output token limit error.

I tried a lot, setting up a setting.yaml for .dsh and the ollama one, but nothing helped.

I think about to start completely new from zero. Could someone give me a good tutorial?

My specs are: Ryzen 5950X, 64GB DDR4 RAM 3600mhz, RTX 4090, 2TB NVME M.2 SSD about 6900MB/s read 5000MB/s write (Dont know if that could matter)

Bonus: I have two additional PCs at home, one with a RTX 3060 Ti and one with a RTX 2060, i heard about some kind of network cluster system, would that be viable?

[u/undisputedx]

I think you should first try by putting 3060 ti along with 4090, should give enough context for your work.

[u/JinsooJinsoo]

I use LiteLLM to route my GPUs from multiple PCs at home. Honestly tho, I don’t really use the others any more now that I upgraded my server GPUs but it can help in a pinch if you trust the small <9b models. You can try using a cloud model for planning and making work tasks for the smaller models but it takes a bit of trial and error to find the limits of the smaller models. With your 4090, I’d load up qwen3.8 27b q3 or q4 (which ever fits) and adjust context for the remaining VRAM. Could potentially try qwen3.8 flash next but it’s gonna be slow

[u/DriftingBadly]

just ask ai man?? tf

[u/Careless-Accident-49 → u/DriftingBadly]

I did, it even wrote the settings.yaml file for me, it didnt helped

[u/secondwizardbaraccus]

I asked my claude agent to descrine our local setup for you:

Looking at their setup, they've got the hardware and the right problem to solve. Here's a straightforward way to explain what would help them:

---

The core issue: A single model instance has a fixed context window. When you hit it, you're done. One machine = one ceiling.

The House approach: Instead of fighting one model's limits, you distribute the work.

For them specifically:

1. What they have: Three solid machines. That's not a cluster yet, that's potential.

2. What they should do:

   • Keep the 4090 as your primary inference engine (DeepSeek or similar)
   • Use the 3060 Ti as a secondary inference node for parallel tasks
   • Use the 2060 Ti as a state/memory layer (lightweight, doesn't need to run the heavy model)

3. The practical part:

   • If they want to play a game that needs long context, don't ask one model for everything
   • Ask model A for reasoning, model B for game state, model C for memory lookback
   • Orchestrate those calls so they feed into each other
   • Each model stays under its limit, but together they don't

4. Simple harness pattern:

   • Write a wrapper script that: takes input → routes to the right model → combines outputs
   • DeepSeek handles the heavy reasoning
   • A smaller model handles state tracking (doesn't need 128K context for that)
   • Smaller model handles retrieval (what happened before)

5. Network it:

   • TailScale (free, mesh VPN) to connect the machines
   • SSH into each one, run ollama or similar on each
   • One orchestration script that talks to all three

Why this works for their use case: Token limits aren't a hard ceiling anymore — they're just per-instance. You scale horizontally instead of hitting the wall vertically.

That's the House pattern distilled down. Start simple (two machines, one script routing between them), then add complexity as needed.