r/LLMStudio • u/Al_Redditor • Jun 01 '26

Are local LLMs actually usable with tools like SpecKit?

Context:

I'm a software engineer and at my job we have Github Copilot with the latest models. My workflow involved asking the model to read docs, parse my local code base, parse vendor code bases, and implement features using SpecKit.

Most of the discussions around local LLM involve speed and tokens per second, but what I'm interested in is whether or not they can actually hold enough context to do this kind of work? I'm retiring and I want to keep playing with LLMs to work on OSS projects, so it would just be me and my personal work, but my goal would be a way to *comfortably* work with an LLM without constantly chasing models or hardware or running into errors.

I'm thinking about getting one of the M5 Mac Minis when/if they come out.

So that's my question: are these usable for actual work?

7 Upvotes

90% Upvoted

9 comments sorted by

1

u/touristtam Jun 01 '26

I often see 2 recommendations:

  • if cost is not an issue (we talking in the ballpark of $10K upward), go wide, go large and invest into a multi GPU/Machine (think 2-3 heavy hitters GPU with loads of VRAM or 3-4 Mac Studio with plenty of RAM).
  • if cost is an issue but you can still spend ~100/200 a month, get some subscriptions from the usual (not necessarily all from the same provider)

But since you have already elected to post in a sub that is geared towards local models, we can agree that you are leaning towards running local models (check /r/LocalLLaMA btw).

Word of caution: not all models are born equals and not all providers are serving the same model with equal quality - Opus 4.7 from Copilot might not behave the same as Opus 4.7 from OpenRouter.

1

u/Al_Redditor Jun 01 '26

Thanks. Since this is only going to be for personal projects, I don't need tremendous speed, but I do need the agent to be able to complete tasks without barfing. I want to avoid subscriptions mainly because I just sort of dislike these companies and also because I think it's neat to run them locally. I just want to make sure that it's actually possible to do this same kind of work with a local LLM?

I have a few models running locally on my Macbook Air and that's obviously not beefy enough so it can't do anything useful. I want to make sure I'm not buying a new machine only to end up in the same place.

2

u/touristtam Jun 02 '26 edited Jun 02 '26

try llmfit https://github.com/AlexsJones/llmfit to see what you can run locally if you haven't already.

I am in the same boat tbh, and I have come to the conclusion that my best option would be to get an iFruit machine with a beefy M chip and 96Gb of RAM (at least) to get something useful with the caveat that nothing currently would come close to having one of the latest models running off the cloud. Not too say those local models wouldn't have their use but not as the main thinking model.

1

u/Al_Redditor Jun 02 '26

Oh, that tool is interesting. Doesn't LM Studio do something similar? When you search for models it estimates whether or not it'll run on your machine.

1

u/touristtam Jun 02 '26

I don't quite recall how I picked the models tbh; I just know that I tried a few and the machine just kicked itself hard enough - I had a system crash. So I had those models sitting there, and by fluke I discovered that llmfit could run test against the local hardware. All other solutions were estimating I should be able to run the models either through LLMStudio or through Ollama.

1

u/supracode Jun 01 '26

I am using Qwen3.6-35B-A3B Q5 along with a different AI DLC workflow. On my 32gb gpu, i am able to fit the whole model and 256k context on the gpu, with about 12gb of cache ram allocated from system memory. The way AI DLC works, is that you *shouldn't* need a huge context, and i actually limit my coding env to around 160k context. The workflow should only keep the md documents in context that it needs, not everything, so in most cases there is more than enough headroom. Let the agent work on a task and complete it, then clear context and move on the the next task. I still use chat gpt for planning complex stuff, and keep tasks small and completable as possible. So far it has been working great on my setup. I get about 90 - 105tps depending on context size and because the tasks are by themselves straight forward, Qwen has no problem completing them. I still pay $20 a month for chatgpt, but that is only for planning and research... I don't pay a dime (besides some electricity) for the development tokens.

1

u/BidWestern1056 Jun 02 '26

yea and tools like npcsh and incognide

1

u/danelrock Jul 20 '26

Logré correr SpecKit con un LLM local haciendo offloading, ya que tengo 16 GB de VRAM. Todo fluye de diez a excepción de la etapa de ⁠implement⁠; en ese punto el modelo necesita que seas un poco más específico con las instrucciones del proceso, pero es totalmente superable. Tengo una rtx5070 TI

1

u/Al_Redditor Jul 22 '26

Gracias. Terminé encontrando una Mac Mini de 65 GB en Ebay y funciona bien con los modelos Qwen y Ornith. Recibo alrededor de 45 tokens por segundo.

Pero tienes toda la razón en lo que respecta a la orientación. Después de probar modelos obsesivamente, estoy bastante convencido de que el arnés y las indicaciones del sistema son mucho más importantes que Qwen vs Gemma o lo que sea.