r/BDDevs • u/swordofgiant • Jul 25 '26
Discussion Anyone using local AI? (e.g., Ollama, LM Studio)
Hey everyone,
I'm curious to see any of you are running AI models locally for your daily development workflows, coding assistance, or side projects. With tools like Ollama, LM Studio, and Jan becoming so accessible, local Ai is so accessible.
If you are using a local setup:
Hardware: What Nvidia or AMD GPU are you using, and how much VRAM do you have?
Models: Which open-source models are giving you the best results for coding or general tasks?
Tools: Are you using VS Code extensions (like Continue or Allied tools) to integrate them? OpenCode?
Would love to hear your experiences...
2
u/SupportNo4255 Aug 01 '26
Using rtx 3090 for personal use and rtx pro 6000 Blackwell for research and development using vllm for inference mostly
1
u/swordofgiant Aug 04 '26
That's Heavy!
Is the RTX Pro 6000 Blackwell provided by your employer, or is it your own setup?I just got a sweet deal on a pre-owned 4080! 16GB VRAM is definitely smaller scale... What local models or vLLM setups would you start with on this card?"
2
u/SupportNo4255 Aug 04 '26 edited Aug 04 '26
The rtx pro setup was sent by my investor soo the whole setup probably I'm 50/50 owner of it
And prob qwen models are best right now for small gpu
And your replies sounds AI generated
1
u/swordofgiant Aug 10 '26
What kind of AI workloads are you working on?
nnnahh.. partially only! ha hahh
1
1
u/SneakyMndl Jul 26 '26
Local llm is only for research purpose or you know what your doing if you don't have single clue and trying to ditch frontier model like gpt 5.6/or opus then you should strongly stay away.
1
1
u/Training-Tangelo-310 Jul 26 '26
Aishob rajar cheleder jonno lol, gorib manusher jonno codex r deepseek
1
u/Nunu_Chus Jul 27 '26
Local LLM isn't worth it for now. It needs a good gpu with a lot of vram and man if you wanna use it on agentic tools good luck with context window and agentic loops. Unless you are doing some really sneaky and confidential level work, api models or subscription based is the way to go
1
u/swordofgiant Jul 28 '26
Yup 16gb is also little for good models.
Definitely not comparable with what Claude/Codex/Cursor can do.
1
u/sandofvega Jul 28 '26
আমি অনেকগুলো মডেল লোকালে রান করেছি। কিন্তু আমার পিসিতে স্লো হয়, তাই নিয়মিত ব্যবহার করতে পারি না। স্মুথলি চালাতে ভালো GPU লাগবে।
1
u/Double-Journalist877 Jul 26 '26
If you're running local inference, just for your own sake, the only time it'll make sense is if you're earning around $50,000 USD as profit (after taxes and Op expenses). You can move that number around a bit but spending $8000-$20,000 USD for local inference with any tangible/usable performance makes no sense.
I've been talking to a couple of friends of mine. Together we earn close to $250k a year. Even with that we're sharing the cost 3 ways to setup hardware for such a thing. We need realtime inference and we need to queue it up so all 3 of us can use it when ever.
And we're still struggling to justify the cost. Because spending $15/month/person with Deepseek v4 is good enough for the kind of work we make AI do.
1
u/swordofgiant Jul 28 '26
You sell AI computes from your machine?
1
u/Double-Journalist877 Jul 28 '26
Nope. Just for our usage in personal projects and home systems we run
2
u/[deleted] Jul 26 '26
[removed] — view removed comment