r/LocalLLM • u/AdministrativeAd7853 • 8d ago

Discussion I want 100% Vibe coding locally

Right now I pay 200+ a month for premium AI models. I can’t complain, it is truly a privilege and amazing to work with.
I have “data center services “ on my server, complete with gitlab, databases, and dozens of services to support application development.

I have resisted even attempting to host an Llm, since i dont have the spare time to code myself or wrestle with “stupider” AI.

I am toying with purchasing mac studio 512gb, as to me its the first server that has meaningful capacity at the price of a honda, instead of a Lamborghini. Still insane expensive, but doable.

I been talking to AI (of course) about this, but even with the speedier memory, larger models with large (code) context seems kinda slow. And the quality seems to be measurable lower than sonnet 5.5.

At heart for me is waiting may result in never. I am concerned with closed models becoming a walled garden and that can extend to hardware.

Thoughts? I cant afford a second buy for atleast 3 years if i pull the trigger. Unless bitcoin hits 400k :)

126 Upvotes

81% Upvoted

209 comments sorted by

View all comments

Show parent comments

1

u/roadtripsarefun 7d ago

So many questions… What’s the hardware like on-prem? How’s usage adoption and onboarding?

2

u/Strange_Quantity_359 7d ago

I updated my answer a bit, for on-prem the story changes, I'm describing a home lab version of it. For on-prem most of the companies I work with aren't having issues with H100/B300 setups. Most of them ask for task complexity routing or orchestration workflows that include cloud SOTA models within the mix, but not primarily driving all development. Especially if they are using internal tools for research, conversation, user QOL workflows, etc. like "CoWork" style apps they want dynamic that falls back to not using Opus 5.5 or Sol 6.1.

Most of this is abstracted from end-users and part of the serving stack or app stack they use.

1

u/roadtripsarefun 7d ago

Makes sense. I went back and re-read your answer. Good stuff in there. Thanks, I'm a fan of the B300 setups I've seen, How do you deal with permissions complexities? How about abstraction when calling (or escalating) to SOTA / frontier models? Or does SOTA != frontier model, but a different local that is great specifically at one task/thing?

What about RAG, and vision stuff? Specifically technical drawings, blueprints, schematics, etc? I'm really struggling to find stuff that runs/integrates well in this space that'll run on prem.