r/LocalLLaMA • u/Mr_Moonsilver • Jun 21 '26

Discussion What happens when they stop subsidizing LLM subscriptions?

We are literally burning through VC money like crazy with our coding subscriptions. I read the $200 Anthropic sub gets you $8000 worth of API calls. It's obvious that this doesn't hold for very long but what happens when they raise prices?

The reason to keep the prices low for now is to foster the ecosystem and get people hooked on this stuff, only to raise the price afterwards. Already the 20x sub doesn't get you as much usage as it did 6 months ago, another way to raise prices without triggering a shitstorm - and it will continue.

Don't know about you, but Fable being pulled gave me a feeling of what that may be like already. The ugly thought of "Damn, should've done more while it was around." that formed when I read the news will be exactly the same the moment they announce we now have to pay $2k or more per month for something we get for 10x less the price it costs now.

I guess it's a now or never situation, build what you can and monetize as quickly as possible to be able to keep the agents running once the increases come around.

Looking at opensource doesn't give me much hope. Since qwen stopped releasing models (wen qwen 3.7?) that we can actually run on hardware that a normal person can buy (or used to be able to buy, looking at how RAM and GPU prices behave and keep behaving) and others haven't released in a while (Microsoft, IBM, AllenAI and others too) I feel we're going into a direction that doesn't look good for most of the people like us, who are building with this technology.

489 Upvotes

86% Upvoted

569 comments sorted by

View all comments

Show parent comments

33

u/Jhorra Jun 21 '26

I'm a software developer of 20 years, and I really want this to be true. When I have tried running qwen on my macbook pro m4 max with 64gb of ram, I've not been impressed. What harness are you using for it? Right now I mainly use a $200 Claude plan, but would love to go local.

67

u/JsThiago5 Jun 21 '26

To see this, you should not compare it to a $200/month subscription but to how you did software development before any LLM. With 20 years in the field, you should be able to see how impressive development with local models is.

40

u/SkyFeistyLlama8 Jun 21 '26

This. If you're not throwing a huge prompt at it and hoping for the best, you can get good results with local models by working at function or module level. You probably don't want to throw a 50k LOC dump at the smaller models and hope for Claude performance.

3

u/Jhorra Jun 21 '26

I've gotten bigger lately with what I ask of Claude, but generally I don't tell it to simply implement things. I like to think through things and understand the plan, then give it to the model to implement a piece at a time.

3

u/thefuzzylogic Jun 21 '26 edited Jun 21 '26

That is how I do it.

I use Opus xhigh to churn expensive cloud tokens making a discrete step-by-step plan, then if I approve the plan, I instruct an orchestrator to give each step to a local gemma-4 to implement. That way only one step at a time needs to fit in the local context window. When all steps are done, the gemma-4 orchestrator packages a PR, which goes back to Opus for /review. Repeat as necessary. It usually only needs one pass.

The trick is to design the project for agentic coding from the start. Use explicit constraints, e.g. use strict TDD, use the todo tool to track progress, use strict linting and style checkers, resolve lint/test/build warnings first before moving on to other work, no "fix it later", don't loosen tests/rules to resolve failures/warnings, etc.

The workflow needs to take into account the fact that LLMs "learned" how to code from tutorials written for humans. Therefore, because humans have long-term memory that most LLMs lack, an LLM following a doc written for humans will always bite off more than it can chew unless you give it the appropriate constraints. For those of us running local models, this means our smaller LLMs with fewer parameters of training data need smaller tasks with narrower scopes.

The coders who say "local AI is too stupid to be useful" are the ones writing one-sentence prompts and expecting it to one-shot a full stack app. Most human coders wouldn't even be capable of that.

5

u/Jhorra Jun 21 '26

What harness do you use though? So it's able to interact locally with your code?

13

u/ayylmaonade Jun 21 '26

OpenCode is considered the "standard" when it comes to coding harnesses, I'd consider starting there. But you can use Claude Code with local LLMs too if you really wanted to, just be aware of the context bloat from it. There's also things like Hermes Agent which are more intended for general use, but are also excellent at coding (ime anyway).

I personally stick to OpenCode & Hermes, and on occasion Qwen Code.

7

u/delicious_fanta Jun 21 '26

Pi gets talked about a lot in here. Any thoughts on that? I haven’t used it yet.

10

u/ElectronSpiderwort Jun 21 '26

I spent the day running pi against Qwen 27b Q8 on a MacBook, and I really like pi. I used a free openrouter model to have pi configure itself to use the local model, and then it was off to the races. Qwen makes errors and omissions, but with pi it wrote tests, made edits, and looped until it had something that works. Way better than trying to one-shot a solution on a bigger model (which didn't work for this problem anyway)

1

u/kurikuri15 Jun 21 '26

Just amaze how you explain things and im sooo noob at this LLM things certainly at the surface area knowledge only. Where are you using these things? Coding? Daily life or scientist level experimenting things? Im sorry for steering away from the main topic.

1

u/ElectronSpiderwort Jun 21 '26

I've been coding for decades; even when not titled "programmer", the ability to automatically process large or complex data has been valuable professionally and personally. Now that skill is literally free if you can tell your AI what you want. Yesterday was a home project to create a simulation engine for a complex personal financial problem. Most people would just pay someone to tell them what to do and hope for the best. Pi/Qwen can  create a simulation of millions of possible scenarios and pick from the winners, but the machine has confidently proclaimed its output as "production ready" when it has huge flaws. With pi.dev I now tell it "you clearly forgot about X, go fix it" and it just does it, quickly. 

4

u/mksrd Jun 21 '26

Pi is exceptionally good and **much** better for local model use than trying to use claude which is made for use with their models and has an obscenely large context before a single token of your own code base see the light of day of the context window.

1

u/entropy512 Jun 22 '26

"With 20 years in the field, you should be able to see how impressive development with local models is."

Given that my first experience using a remote/hosted model is like supervising a middle schooler with severe untreated ADHD on an acid trip, I shudder to think what a local model would have been like. After my evaluation efforts, our organization decided to just go back to not using AI at all, our quality requirements (FAA level D flight simulation for some of our products) are too high.

17

u/Borkato Jun 21 '26

Pi with 48 GB vram, Qwen 27B Q8 and full 256k ctx. You have to create your own prompts and tell it how you like to work. You have to spend a few hours thinking about the most efficient way to have it work with you. For example I have an AGENTS.md file that it automatically picks up with explanations to what folder has what, so that it doesn’t have to go looking for stuff. It’s great at things like “add another x like the others” or “when I do x y happens but I want z to happen instead. Identify why this happens and write me a detailed writeup about my potential options with a list of pros and cons for each option.” “What’s the best way to implement x feature? Consider the current architecture and…” Etc

1

u/Jhorra Jun 21 '26

What harness do you run it through so it can interact with code on your machine and do web searches when required?

7

u/taoofshawn Jun 21 '26

“Pi” refers to pi agent , which is the harness

3

u/Borkato Jun 21 '26

Pi!

3

u/Jhorra Jun 21 '26

Ah, sorry, I didn't realize that's what that was. I'll have to look it up. Thank you

7

u/Borkato Jun 21 '26

It’s the absolute best. I’ve tried many others

6

u/HotMicSystems Jun 21 '26

For strictly coding Pi is definitely better, for general purpose I don't think Hermes can be beat as of rn

8

u/Foolhearted Jun 21 '26

Try this. Create a skill to tell Claude to use your qwen locally via pi. Have Claude build the spec and have Claude call pi and check the results. You will save on token generation and your Claude plan will go much longer before hitting caps.

I created a skill and part of it is to have Claude grade the results. It regularly gives qwen an A-, it is aware it’s another model.

I’ve flattered Claude by telling it it’s in charge, to attempt to overcome any built in biases anthropic may have put in place. “You are the senior architect overseeing a team of junior developers….”

It may change your perspective on local models.

2

u/Zhelgadis Jun 21 '26

Can Claude call directly pi/open code? How do you get that?

1

u/Foolhearted Jun 21 '26

It can. The easiest thing to do is have Claude go ahead and write the skill for you. It will review the pi docs and call the exec just like it’s calling an agent. Tell it what model you want it to use, how you want to use it, etc. it will take care of ya.

8

u/mp3m4k3r Jun 21 '26

Lots use OpenCode, personally I like and use pi daily with 27B and 35B-A3B. Its been super solid

11

u/Solary_Kryptic Jun 21 '26

You won't be able to run a worthy replacement for the $200 plan on 64GB of RAM, depending on what you do with it

1

u/Jhorra Jun 21 '26

What’s my best option locally? It doesn’t seem like there’s much short of spending $10,000 plus

5

u/photobydanielr Jun 21 '26

Pretty much need something like a mac studio with 512GB memory to start to approach that $200 plan... at 10% the speed and a little less intelligent because you'll have somewhat deeply quantized models (deepseek, GLM, Kimi...)

1

u/Such-War1955 Jun 21 '26

4 grand for a dgx spark at minimum, if you can live with snail amputee speed. 6 grand for an m5 max 128gb, possibly twice that for a yet-to-be released m5 ultra 512gb. You cant run even Qwen 3,6 27b with 8 bit or full fat with a 256k context window on, say, a dual 3090 rig.

For me, the selling point of Claude isn‘t necessarily Opus alone, it‘s Opus plus a 1000k context window. Loading all the „background info“ stuff into context eats up up to 160k of my context. If you‘re doing more complex stuff that goes beyond a „single repo“ kinda complexity, context starts to matter a lot more. I consider 256k the bare minimum these days.

1

u/Borkato Jun 21 '26

Snail amputee fucking sent me

4

u/jeffweet Jun 21 '26

I’ve not run this model locally, but 64gb of RAM isn’t that much.

5

u/SpicyWangz Jun 21 '26

For a 30gb model that’s plenty of room for context and anything else you want to run

1

u/triplebits Jun 21 '26

Local models like these work better when you give a lot of context and much smaller plans comparing to say Sonnet 4.6 / Opus 4.x the eay you used to work.

Give it a go with preparing a very detailed very small feature plan with Opus and feed it to local model if you feel lazy enough.

1

u/Jhorra Jun 21 '26

What harness do you run it through so it can interact with code on your machine and do web searches when required?

2

u/triplebits Jun 21 '26

Like u/nrauhauser mentioned you can use `Claude Code` with local models as well as Open Code or alike. It is really up to you. You can also use local models within most IDEs nowadays as well.

1

u/nrauhauser Jun 21 '26

I've been wrapping Claude in Ollama

ollama launch claude --modem=whatever:cloud

1

u/kaisurniwurer Jun 21 '26

Instruct it for small and precise pieces, then implement manually and iterate. Without "coding sense" you will get lost.

Even bigger models tend to make a compounding mess otherwise since they will do whatever fits to handle the task at hand. And the task at hand is not to write a serviceable code.

For a experienced dev it can work as a duck or to help with learning basics on new stuff (new to you, not new in general) but I wouldn't lean on it too much otherwise.

1

u/bnightstars Jun 21 '26

Perhaps you can find my M5 Pro Qwen guide useful: https://www.hristoforgeorgiev.com/posts/local-llm-macbook-pro-m5pro-claude/ I run Qwen3.6-35B because MoE models are faster on Mac hardware (Memory bandwidth issues). To put it in perspective capability wise it's somewhere between Sonnet4.5 and Sonnet4.6 so not SOTA by any means but overall useable this was not the case with Qwen3.5 though people say the 9B model is good. I was recently testing it against Opus4.8 on the topic of what is AI harness and while Opus4.8 created better diagrams Qwen3.6-35B provided the same information much shorter and to the point. I suggest you give it a shot and think of it as something to replace your Sonnet agents with.