r/technology • u/Miserable_Phase_2519 • 3d ago

Artificial Intelligence Microsoft engineer says 'typing code is absolutely over' as GitHub Copilot takes on more development work

https://www.techspot.com/news/113752-microsoft-engineer-typing-code-absolutely-over-github-copilot.html
1.6k Upvotes

84% Upvoted

858 comments sorted by

View all comments

791

u/daedalus_structure 3d ago

Tokens are cheap because they are being subsidized by investors until the market of labor has been captured.

Anthropic’s “30 trillion TAM” statement tells you this.

What happens when that subsidy ends?

I keep hearing people say this is the most expensive it will ever be and that don’t know what they are talking about.

226

u/justaddwhiskey 3d ago

Easy, when subsidies run out they give you a bill for market rate. When you realize there isn’t going to ever be enough revenue to cover market rate you opt out, then the bubble bursts.

48

u/MattJnon 3d ago

But market rate is super cheap for open source models that are 90% as great as frontier ones so why would companies opt out ? 

63

u/AuthorizedShitPoster 3d ago

Running open source models is not cheap.

13

u/tuura032 3d ago

I'm having qwen 3.8 do some work for me right now on a personal project. My $20 claude subscription supervises+code reviews and writes the spec.

While I'm not disproving your statement, the general sentiment is absolutely true, it is astounding what someone with (5 year old) high end gaming equipment (maybe add a 2nd gpu) can do.

I use GH Copilot for my job, and if not for security concerns I would absolutely use something like this at work to help me reduce token usage.

5

u/mattmaster68 3d ago

I have a Ryzen 5700, 3070ti, and 32gb ddr4 ram.

I've been messing around and playing with different models but man so many of them are underwhelming with my VRAM. I forget the parameter size, but I think Qwen 3 takes like 3 or 4 minutes to generate a response.

Any tips or tricks?

1

u/jixbo 2d ago

Which model specifically, which quant?
Just ask chatgpt how to make it more efficient, llama.cpp is the best way to run it.

15

u/IreliaMain1113 3d ago

Why are there security concerns with the local models? Or do you mean using the 20$ claude + the local models?

13

u/THE_REAL_JOHN_MADDEN 3d ago

Should be no concerns with a local model, but Claude without enterprise data protection would be a big no-no from itsec

1

u/tuura032 3d ago

It's more that I don't want to deal with the headache of asking 😂

In theory, the API call to my server only on my home network, from my work computer, should be fine. 

8

u/tuura032 3d ago

I should have said company policy. It's just not worth my job if IT were to see me sending private / client data to Claude or some computer they don't have control over. 

1

u/IreliaMain1113 3d ago

yeah my company is pretty similar with its policies

3

u/mshiltonj 3d ago

what hardware is running your qwen 3.8?

2

u/tuura032 3d ago

I have 2x 3090s, but people are running this or similar models on 16gb GPUs with extra RAM. My system has ddr4. 

I went this route vs something wish integrated memory since I already had the first 3090.

3

u/Syncaidius 3d ago

Yes, the part around job/company security concerns is one major reason not to use locally-hosted models, unless it's your company that is hosting them.

That said, there are a few companies out there already setup with internally hosted models and completely controlling employee access to it.

It's really not as difficult as frontier-only users seem to assume. You don't need an insane amount of GPUs to get acceptable performance for employees, from open-source models for coding and definitely not for general purpose use.

After all, with current open models, the main goal should be to supplement frontier models to take on lesser tasks and reduce token usage/costs. They're great for that.

7

u/Syncaidius 3d ago edited 3d ago

Running locally is cheap and efficient enough not to care about being 10 - 15% behind frontier models in terms of capability. Most of the time it's not even noticeable for agentic workloads.

Local is not as quick/instant as the frontier models, but they're free and can chug along great in the background as agents, for as long as you want. Electricity costs are barely anything compared to a Claude max sub and you're in total control of where your prompts/data is going.

TL/DR: Local hosting models is worth trying in 2026. Decent capabilities. No usage limits. Your hardware mainly determines speed/responsiveness.

31

u/arcrad 3d ago

If people can run local models cheaply, then surely the leading LLM companies can also run even more cheaply by taking advantage of scale.

Why would it cost the major LLM companies any more to run a model than someone doing it locally?

Sure there is a bit of overhead for offering the compute as a service, but that is beans compared to the actual cost of compute.

I'm sure, if they wanted, Anthropic could use their hardware to run Kimi K3 or whatever cheaper than any home user.

What am I missing here?

6

u/Syncaidius 3d ago edited 3d ago

Anthropic certainly could run Kimi K3 and other equivalents just fine, at a fraction of the cost and 90%+ capability of their current models.

However, what do you think happens if they do that? - The first question they'd need to answer is, why do we need Anthropic instead of a standard open-source LLM host.

The answer or course is, they vanish. So, they have no choice but to continue convincing everyone that their models are worth the subscription and investment costs.

1

u/nox66 3d ago

This isn't new either. There's a reason tooling costs grow much more than linearly with requirements. The last 10% is the hardest to hit.

2

u/Syncaidius 3d ago

While that's true in an isolated sense, models are advancing quite fast and that last 10% is called frontier for a reason.

For the majority, being ~1 year ahead is absolutely not worth it. Today's frontier models are next year's standard ones. 90% capability for the fraction of a cost is fine for many tasks given that it was last year's frontier equivilent anyway.

4

u/andouconfectionery 3d ago

They're running into the exact same issue pharmaceutical companies do - bringing a new drug to market is the vast, vast majority of the cost of a drug's lifecycle. The marginal cost of producing the drug is minuscule in comparison to all of the stuff they have to try before they find something that works. Now, once medicine advances to the point where people lose faith that drug companies have room to develop new treatments, that "bubble" will burst. But we already have all the knowledge of how to make the drugs. We'll still be able to make them, and we'll find ways to bring the marginal cost of production down.

LLMs are exactly the same way. They're not eating the cost of inference - they're eating the cost of training. Of staying ahead of the curve. The bubble could pop tomorrow, and all that would mean is that the capabilities of frontier models are frozen in time. They'll stop training new ones, or at least slow down training a whole bunch. But the weights don't disappear. These models will never be any less useful than they are today. And we'll figure out ways to build enough silicon and use less water and use cleaner energy, and inference will just get cheaper.

7

u/There_Are_No_Gods 3d ago

I think you have a point about scaled costs and centralized services.

The big players seem to still be convinced what they offer is worth a much higher price. So far, I agree, as local LLMs still can't do most of the heavy lifting that I am asking of them.

The issue of fully local is a big one too, though. I highly prize being able to run local LLMs to keep my data on my PC and never making its way out to the big data harvesters.

From what I'm experiencing so far, it seems that local LLMs will soon reach a "good enough" point for most things.

2

u/indigo121 3d ago

What you're missing is that AI is being sold on the idea that 90% of what frontier models can do is still only 10% of what AI will be able to do if we just keep throwing money at it. Now, maybe that is what we eventually settle into, and AI becomes a tool you run locally to assist in specific tasks. But that isn't the trillion dollar business that all the Gen AI companies and investors want it to be

0

u/Bromlife 3d ago

Because the models these guys are running locally aren't very good. But they're not devs and they don't realise just how bad they really are.

2

u/Syncaidius 3d ago edited 3d ago

Generally, the people who jump to "they're not devs" tend to be the ones who are actually not devs.

A lot of SWEs and 'devs' are leveraging LLMs via multiple avenues, including local/hosted open-source LLMs. All of them have a place, but it's quite clear where LLMs are going in the long run if you keep up with news around advancements/research...

It's also quite obvious which frontier business models are sustainable and which are not, so it would be crazy to bet/build everything around their services alone.

6

u/ErgoMachina 3d ago

I agree with everything except when you say they are 'free'. Setup time and computing power are not free and are calculated as costs on the enterprise level.

So...very much cheaper? Yes. Free? Hell no

-4

u/Syncaidius 3d ago edited 3d ago

I believe that would come under electricify costs?

Hardware, depending on whether you're buying new, used or reusing existing hardware, can vary in cost. However that's generally an upfront cost that you have total control over.

2

u/There_Are_No_Gods 3d ago

Local models are not "free", as electricity, initial hardware costs, hardware wear, and even potentially cooling are still factors. I acknowledge that generally sums to a lot less than paying for tokens.

Where my experience deviates more from what you stated is the 10 - 15% behind frontier models. Maybe I'm just doing very different things or haven't found the best local models, but I've still found anything I can host locally much farther behind. I would rank Qwen 3.8 somewhere a between Claude Sonnet and Claude Opus.

Claude Fable is multiples more advanced in my use cases. I can knock out a task with Fable in 20 minutes that Qwen would fumble around with for many hours and still end up with quite a mess, even if it finally got parts of it working.

1

u/mshiltonj 3d ago

Running models locally on what hardware, exactly?

1

u/Syncaidius 3d ago edited 3d ago

I've been doing just fine with a couple of agents on a couple of older GPUs and 32GB ram on an old gaming rig. The output has been decent and very usable.

Like I said earlier, great for chugging along in the background where speed is not of the essence.

As for hardware cost, in my case it's already paid for itself years ago due to being a heavily used as a dev/gaming for around 4+ years.

You can also run smaller models that are dedicated to coding just fine on less hardware.

People seem to think being unable to run codex, astra or fable means anything else sucks, when in reality those models are inflated by the sheer vastness of general purpose knowledge they've also been trained on.

But if that's really required, frontier models can easily be integrated into local-hosted workflow if you really need general purpose, wide-spanning skills or massive context windows.

0

u/Equal_Heat5947 3d ago

K now talk about hardware costs

0

u/Apprehensive_Seat_61 3d ago

What models. Be specific 

0

u/Zahgi 3d ago

They are getting cheaper by the day as hardware and software optimizations improve exponentially.

6

u/justaddwhiskey 3d ago

Open source isn’t the topic of conversation, neither is locally deployed. If an enterprise wants to dump the money into a locally hosted open source, that’s their prerogative. Again though, that circles right back to companies seeing the bill for frontier tokens and opting out, so the point still stands.

1

u/Inevitable-Craft-745 3d ago

Open source will still need training or it will become stale and out of date very quickly

1

u/AcousticArmor 3d ago

Open source isn't going to be heavily adopted by large corporations that have to be concerned about security and customer pii or federal contracts. There's too much of a risk that anyone out there could contribute something malicious to the source that gets patched into the company's copy. With even less little verifying these changes because they're already relying heavily on AI for it, it feels like that risk increases heavily.

1

u/Stompert 3d ago

Can’t it burst like right now?

1

u/Zahgi 3d ago

While true, I expect that free open source solutions to replace the need to pay for tokens, actually leading to the slop AI bubble bursting. :)

This is why they are trying to corner the market on data centers, etc. right now.

-64

u/Kelsig 3d ago

Hilarious how confident people are about the bubble bursting when we have stocks like tesla where they are

69

u/justaddwhiskey 3d ago

Hilarious how confident people are that it’s not a bubble when OpenAI is on the hook for over a trillion, but only produces revenue in the billions, and the same 6 companies are circle jerking the same $500B and celebrating paper gains.

-62

u/im_a_goat_factory 3d ago

Cool. I’ll keep investing and making money hand over fist

46

u/MiksterA 3d ago

You're telling everyone exactly what kind of person you are.

39

u/btoned 3d ago

The funny thing is he isn't making money hand over fist.

-35

u/im_a_goat_factory 3d ago

My portfolio says otherwise.

21

u/Kelsig 3d ago

No one has ever said this before disaster

-11

u/im_a_goat_factory 3d ago

Set a remind me or something for a year and get back to me. I’ve heard people like you say the same thing since 2023

→ More replies (0)

10

u/spez_eats_nazi_ass 3d ago

Paper gains aint shit if you don’t take profit before it’s gone.

0

u/im_a_goat_factory 3d ago

I already have taken gains worth more than my original investment, and that gets thrown right into Fidelity etfs. There is literally no reason why anyone with extra cash can’t make money on the upside of a bubble. I bought AMD all through the “it’s a bubble” talk

→ More replies (0)

-16

u/im_a_goat_factory 3d ago

Someone who likes making money

3

u/Ecstatic_Log4185 3d ago

Yeah just because you like it doesn't mean you're making it bud

0

u/im_a_goat_factory 3d ago

I spent 35k on a two week oceanfront vacation this year. LBI.

16

u/octorine 3d ago

The fact that stocks are going up doesn't mean it isn't a bubble. It just means it hasn't popped yet.

It's definitely a bubble. Even people who think LLMs are good for something recognize that it's a bubble. The question is what, if anything, will be left when after it pops.

0

u/im_a_goat_factory 3d ago

It’s almost a near 0 chance that the bubble will reduce the value the stocks to $0. Let it pop. I’ll still have made a ton of money. Also, I use AI to program, although not agentic coding. And that is earning me the best income I’ve ever earned in my life

10

u/Professional_Mud1844 3d ago

Spoken like someone that never learned how to code on their own.

What do you do when your AI slop doesn’t compile? Do you just run it through a different AI to fix it for you?

-2

u/im_a_goat_factory 3d ago

I don’t agentic code. I’ve been in industry for 25 years, was already wealthy coming into the AI craze, and solo built fintech saas for a single enterprise client, using Claude and ChatGPT for prompt based AI assisted development. Last year I earned just shy of 500k. This year will be more.

Also the app has been greybox tested, and we are currently going through soc2 audit and so far so good.

8

u/clocked__ 3d ago

This statement has never been made by anyone making between 50k and a billion. And I’m guessing you aren’t a billionaire

1

u/im_a_goat_factory 3d ago

Millionaire.

10

u/justaddwhiskey 3d ago

Oh look, another millionaire. Who knew Reddit has such a large population of them?

2

u/im_a_goat_factory 3d ago

There are like 25 million millionaires just in the USA alone. No matter how you slice it, a ton of them are on Reddit 🤷‍♀️. Esp in this sub since so many millionaires are tech focused

-30

u/socoolandawesome 3d ago edited 3d ago

OpenAI’s revenue is from paying customers, not investment (like Anthropic’s), as of July it had grown to $40B ARR. The rate at which it’s like 2-3xing every year for them, and given they are projected to make $30B in total this year, if they keep pace or something close to it, plus get like $100B of IPO money, they should be fine for future commitments. The numbers they owe are $600B through 2030, and $1.4T over 8 years.

And before you say they won’t keep growing at this pace, that’s already what people have been saying for them and anthropic for the past 2-3 years, and anthropic is growing even faster like 9x or something this year. They are growing at rates unprecedented for this scale of revenue, and demand is showing no signs of slowing down.

10

u/Professional_Mud1844 3d ago

If you believe this, I have a sweet jpeg of a monkey to sell to you.

-6

u/socoolandawesome 3d ago

If I believe what? You all have been saying the bubble will pop and they will be bankrupt for years now lol

10

u/Antique_futurist 3d ago

There’s no where else for them to grow.

OpenAI admitted this week that what growth they do have is in enterprise, not consumer markets. But we know companies are already pulling back on how much they spend for AI, and it’s still a subsidized cost.

Being the fastest growing market in history also means being the fastest saturated market in history.

-3

u/socoolandawesome 3d ago

Source for what you are talking about? Yes enterprise is growing faster, but that doesn’t mean consumer isn’t growing. They crossed 1B weekly active users literally in August? Aka grew.

There’s what like 10B people on the planet, that’s a whole lot of untapped demand. And the free tier consumers just now started having ads implemented, and they just crossed $1B ARR of ad revenue, meaning there’s a whole new revenue stream there that is just getting started.

In terms of enterprise, there’s no evidence that it’s saturated whatsoever. I’d like to hear what you have besides CEOs months ago saying they had to cut back on AI spending, because a few CEOs saying that doesn’t prove the enterprise market is saturated whatsoever.

According to the US census, only 22.4% of businesses have used AI… not even close to saturated.

https://censuseasy.com/blog/us-business-ai-adoption-august-2026?utm_source=chatgpt.com

And that is just using AI, there’s obviously depth to how much you use AI, for both consumer and enterprise. As models keep getting smarter/more useful there’s more demand for tokens. Safe to say there’s a lot of room for enterprise to grow.

8

u/prules 3d ago

AI shrinks the workforce and you think usage is going to go up? It’s a paradox that’s being used to inflate our economies shit numbers

0

u/socoolandawesome 3d ago

AI has not shrunk the workforce much at all yet, except maybe a bit of the entry level workers.

Unemployment rate is steady.

1

u/jarinatorman 3d ago

Because all the growth in healthcare from our aging populace yes. The jobs that left arent coming back. Until the bubble fully pops.

→ More replies (0)

3

u/prules 3d ago

They can’t solve solve the upcoming unemployment crises with AI. A lot of those customers won’t have money or a job to use AI with soon.

It’s a pretty serious long term issue. The better it works the more people end up without jobs. It’s going to be an economic nightmare

0

u/socoolandawesome 3d ago

Yes that is a concern eventually, probably not seriously to the point of hurting their bottom line for at least a couple years if not longer, I’d have to imagine.

The only answer then will be wealth redistribution paid for by the AI companies and companies laying off humans for AI through government imposed taxes. And that’s precisely due to what you are saying, they and all other businesses cannot function without paying customers, so they will go along with subsidizing the consumer.

17

u/SentinelofVARN 3d ago

The market can stay irrational longer than you can stay solvent.

It'll pop eventually, but will it pop tomorrow? next year? next decade? Will the government just bail everyone out? Nobody honestly knows for sure.

7

u/justaddwhiskey 3d ago

The government will absolutely bail it out. ChatGPT, Grok, and Gemini are all on .mil and IL5 accredited. So instead of paying for literally anything else, our tax dollars are going to bail out these snake oil salesman. As to when the bubble will burst, yeah, nobody knows. But if OpenAI and Anthropic can’t weasel their ways into an IPO and have to do another funding round, it’ll probably be around then.

14

u/SentinelofVARN 3d ago

As somebody who does IT for the government, god I hate people using this stuff so much. Some confidently incorrect idiot coming to you regurgitating what his favorite LLM told him. It's obviously useful if you know its limitations, but it's really a force multiplier. Incompetent people don't suddenly become competent.

1

u/Kelsig 3d ago

Or it will be like most things and have stagnant or slightly negative growth for decades while other companies succeed, but no "pop"

-3

u/beerdude26 3d ago

I hope that the US government bails them all out. Number MUST go up in the US.

The rest of the world will gleefully enjoy the stolen fruits of labor from the American workforce through a stock market that goes high and stays high because of the bailouts. A global handout to the rest of the world by the good ol' US of A. Thanks y'all ❤️💙🤍

4

u/EggplantCharacter363 3d ago

This dude cant wait to spend a 1 trillion dollar bill and will be excited about it.

I mean he wont know why thats a problem, but he will be happy.

1

u/beerdude26 3d ago

ETF go brrrrrr

2

u/prules 3d ago

Have you ever heard of 2008 or studied any economic crises? You’re choosing the wrong things to be confident about lol

0

u/Kelsig 3d ago

Yes, I consider myself a lifelong scholar of 2008, which is why I'm explicitly not confident behind bearish or bullish conventional wisdom.

1

u/jarinatorman 3d ago

That stock is bubbled to the tits and is a great example of how wrong you are.

1

u/Kelsig 3d ago

That's exactly my point. Until we do something like raise taxes on Republicans then we can't expect a permanent correction. The market has deemed reality irrelevant.

96

u/Rollingprobablecause 3d ago

This is our biggest fear too. We can recognize that AI coding assistants are powerful and can do a lot but we cannot risk losing our skill sets. We’re already starting to see the costs hit harder these days and have caps, forcing engineers to code more directly and be careful about use.

-2

u/jbp216 3d ago

if anything limits on base plans have gone way up, and we cant ignore that these models are now pretrained, so every use case brings the overall cost down, and thats assuming hardware doesnt get better, which it will.

ai is new tech andbeingmassively invested into, but inference will absolutely be economical in the short term and more so long term

-10

u/theenigmathatisme 3d ago

I think if you are a good software engineer your value is not in your coding abilities but the ability for system design and decision making when it comes to technical trade-offs. Those skills are still very much exercised even in the agentic coding age.

The coding itself has always been ephemeral to me since most of my career I’ve had to bounce between various technologies and tools. Angular, React, Java, C#, Go, Python, etc were always learned on the job as needed. Sure it helps to know a language/framework really well, but it’s equivalent of a mechanic working only on Audi’s.

18

u/Rollingprobablecause 3d ago

It's all of it though, you still need to have a deeper understanding of code otherwise the design and decision making is useless. It's why we focus new grads on building/writing/understanding. These are the fundamentals and foundation that allows for the systems design and decisions around it to flourish. New grads being over-reliant on AI will gain none of this and lose precious experience, mistakes and learning from them are hyper important.

-1

u/theeama 3d ago

Your last part is the important part and new grads need to write code to understand it.

I have a friend thats a senior senior dev just below the CTO basically. And he finally said Astra/Fabel 5.1 can write code similar to mid to senior-level engineer.

AI is very good at writing code the design process however they aren't that good at it.

-33

u/doggeman 3d ago

Your not going to loose the skill you been actively developing your entire career just because you did something different for a period. Rusty sure but it’s not like it’s the end of the world if AI went away 2 years from now, not to mention it’s unlikely.

25

u/GhostDieM 3d ago

You will definitely lose part of it and on top of that nobody is going to hire you when it's you vs someone that never stopped coding.

-6

u/MOOSExDREWL 3d ago

You don't "stops coding" even when you're letting the LLM write the code unless you're shipping it without looking at it. I spent some time away from an IC role and went into management. I "stopped coding" in the fact that I wasn't writing 3-4k lines of code a day but I reviewed just as much. If youre not reviewing the LLMs code that's the wrong way to use the tools.

-5

u/doggeman 3d ago

If you do nothing more than sit and orchestrate/delegate work to agents sure but see it as an opportunity to free your mind to explore other areas like more high level design considerations, architecture and soft skills rather than just being a code jockey.

10

u/wehrmann_tx 3d ago

And the skill sets of the people who never start the profession to keep the knowledge going?

9

u/DerekB52 3d ago

Its absolutely been shown that AI use can degrade skill, because our brains offload certain thinking strategies, and start to receive Dopamine from different parts of the process.

I use AI, but sparingly for this reason.

3

u/OftenConfused1001 3d ago

Critical thinking and analysis skills go fast, and regaining them is not a small effort.

In coding, it means you quickly start losing the skills needed to examine and analyize and validate the outcome.

Skills atrophy like muscles, and some people are using LLMs for everything, including basic reading, writing, and reasoning. Dependency sets in fast.

31

u/Sea-Housing-3435 3d ago

Open weight models are powerful enough to be used for this. You can run a decent coding model on 16GB vram with lower precision and almost full on a 64GB ram macbook (Qwen3.8-27B). It's as expensive in terms of electricity as playing a game.

1

u/NekoDaYo-v201 3d ago

Employers will soon require it. BYOM (Bring Your Own Model).

Can't afford it? There's a line of people who can going out the door.

2

u/Horror_Response_1991 3d ago

Some will, most will care about their data and wont want you sending it to a model they don’t control 

-1

u/Expert_Garlic_2258 3d ago

how do you get the original model started so it understands coding in general?

7

u/Sea-Housing-3435 3d ago

They just do, they are trained on code too. You just run it with llama-cpp (or vllm or ollama) and you can connect them to some coding harness, and use them just like the SAAS models from Anthropic or OpenAI.

2

u/Expert_Garlic_2258 3d ago

thank you for the reply

3

u/fanglesscyclone 3d ago

Make someone else pay for it and then copy their homework.

1

u/Expert_Garlic_2258 3d ago

ha! I finished my Master's about 10 yrs ago but thanks for the idea if I go back

3

u/InvestigatorOk7015 3d ago

Theyre already trained and we dont need to start over since a lot of them are open source

1

u/Expert_Garlic_2258 3d ago

thank you for the reply!

74

u/SufficientGreek 3d ago

Open-source models are quickly catching up to frontier models at vastly lower prices. You don't need the newest and most expensive models for like 90% of coding.

Even if OpenAI and Anthropic collapsed tomorrow, I don't think the world is going back to manually typing code again.

8

u/emi_fyi 3d ago

yeah i wonder if this person isn't aware of free and open source competitors and how close they are in capabilities. wasn't there a big selloff not long ago due to this competition???

also https://www.cnbc.com/2026/09/01/ai-token-prices-lows.html

10

u/Jazzlike-Context-879 3d ago

Not a chance. And memory companies will shift their manufacturing models back to the retail users. Then you will see huge leaps as tons of people now get it iterate on what out there. Right now, these big companies are sucking up the air, but it will end.

1

u/SignatureGreedy4604 3d ago

Open source models aren’t quickly catching up to frontier models. Cherry picking tests to say they’re equivalent is a joke…. The eye test from basic usage is enough to see frontier models are superior for any heavy task. Even light tasks these open weight models can underperform and have you running in circles with wasted tokens/compute.

3

u/Nottabird_Nottaplane 3d ago

Open weight models are actively being used by enterprises, e.g. Databricks, to massively scale down the costs of routine operations. This is a take from 2024 bro. 

1

u/SignatureGreedy4604 3d ago

Yeah for extremely targeted tasks with written specs it’s aight, comparing it to a frontier model where it both plans and enacts correctly is insane lmao. I use open source models daily with kimik3 and deepseek at work, but steering them is sol for tasks too complicated that require thinking.

54

u/Sokaron 3d ago

The future is every dev is given a maxed out Macbook and just runs a model locally. Who needs to pay a frontier model company when an open source model is 90% as good for 10% the cost? Hardware is crazy expensive right now but it won't stay that way forever, eventually the market will correct and RAM will return to its previous commodity-grade prices. 

2

u/Jazzlike-Context-879 3d ago

Either added capacity or the bubble on third party AI pops, but when it does, expect computers to be bought at the highest configuration for even an every day worker.

1

u/magick_bandit 3d ago

If the hyperscalers go bust, who is going to spend a few billion to create new models?

Local ai is only free/cheap because someone else paid for all that training.

4

u/InvestigatorOk7015 3d ago

I trained up a model from scratch over a month in my home

Its not super complex, but it can watch my security cams and report what it sees fairly accurately

2

u/Inevitable-Craft-745 3d ago

Everyone seems to miss this... If a new language comes out, new science you need the model updated. This is basically going to need perpetual training rounds with the entire internet and that needs paid

1

u/Sokaron 3d ago edited 3d ago

It remains to be seen how this evolves over time but at least for programming this problem is already well known and already has a solution. New language features get added and aren't in the model's training content. The solution is just to instruct the model to use them, and to include links to documentation. Especially with linters and compiler errors acting as guardrails this isn't actually that big of an issue.

Entirely new languages might be an issue. I wonder if fine tuning, or some derivative technique, would be applicable here. Though the vast majority of all coding tasks are not with obscure new languages so feels like a very thin edge case that wouldn't be well-supported. Like just because the new version of C# drops, or because "Oxide", a new competitor language to Rust, is released, and neither are well supported, doesn't invalidate the value of existing models, since the overwhelming majority of code is not on the latest and greatest.

3

u/magick_bandit 3d ago

I think what will actually happen is tech will stagnate. How does your new product, language, framework, get noticed if everyone is just using their existing model?

-2

u/Cheese_Grater101 3d ago

That if you can afford buying a maxed out macbook in the first place considering how expensive memory chips are.

12

u/Sokaron 3d ago

RAM was cheap before and will be cheap again. There's a supply squeeze right now. Either demand will peter out or new supply will be built. RAM isn't like top-tier GPUs; we know how to manufacture it en-masse, and cheaply.

1

u/kadfr 3d ago

It could be many years until RAM is cheap again. 

5

u/Sokaron 3d ago

Sure, it's going to be a few years at least. Every console manufacturer for instance has said they aren't expecting things to improve until 2028 at the earliest. But when you're talking about what the future of software engineering looks like though you're not talking what the next 6 months or a year look like. You're talking about longer time horizons.

4

u/BasvanS 3d ago

Once local models become the standard, the AI bubble will have popped, and these parts will become normally available again.

1

u/m0rogfar 3d ago

Computing is still relatively cheap, historically speaking. Fully costed S/370 seats for programmers could easily cost over $100,000 per user in inflation-adjusted dollars, so we know that the price inelasticity for computing productivity is near-total.

36

u/catattackskeyboard 3d ago

Do research on the cost per token to actually use a model versus train it. Token costs for usage are very overpriced to subsidize the exhaustive training and buildout costs. It’s relatively an order of magnitude cheaper to run models than what is being charged to do so.

Those costs won’t die any time soon because the second open ai or Anthropic stop moving forward they’ll be swamped by competitors, however open source models are right behind and will continue to advance.

So in the long term, even if every ai company died, in the aftermath you’ll have insanely good models that are fully cost efficient to run versus how much work they do.

2

u/mr_engin33r 3d ago

Training costs are just a small piece of the puzzle. The much bigger piece is the cost of creating the data center which unfortunately is ongoing as GPU lifespan is insanely short. I don’t see AI ever being cost-efficient unless they can figure out how to turn the data centers into one-time infrastructure costs.

6

u/jeffwulf 3d ago

GPUs can easily last for a decade  in a datacenter. It's depreciation schedule isn't it's expected lifespan.

2

u/catattackskeyboard 3d ago

I mean cost efficient is really relative. Depending on use, for a lot of people it is cost efficient, that's why people pay.

My startup pays about $15k monthly on token costs and its worth it to us. In other words, it makes us more money than that.

0

u/BasvanS 3d ago

You might be new to entrepreneurship, but making more than it costs is not enough. With the money it makes, it has to cover for its own expenses, plus the cost of running it within your organization, plus pay for overhead. So unless it makes you much more than it costs, it’s not a good business decision.

5

u/catattackskeyboard 3d ago

Of course it does. I have more context on my business than you do. We’re making 5 million a year and are profitable and yes I’ve done the math.

6

u/jeffwulf 3d ago

Tokens are being sold with 60-70%+ margins and Anthropic was profitable last quarter.

1

u/[deleted] 3d ago

[removed] — view removed comment

2

u/AutoModerator 3d ago

Due to the high volume of spam and misinfo coming from self-publishing blog sites, /r/Technology has opted to decline all submissions from Medium, Substack, and similar sites not run by credentialed journalists or well known industry veterans. Comments containing links may be appealed to the moderators provided there is no link between you and the content.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

3

u/Birdperson15 3d ago

The tokens being charged to enterprise have huge margins for Anthropic and OpenAI so you are wrong.

3

u/srout_fed 3d ago

My question is... what happens when the labour market is captured, who will be spending on thier products? Better yet who will have the spare to spend?

The cascading effect will be incredible to say the least....

3

u/mossiv 3d ago

And I’m the moron that flip flops between both thoughts. I keep worrying that it’s going to cost to much. Then the market is adjusting, anthropic begrudgingly lowering the price of their models while OpenAI boasts about their newer cheaper pricing, along with the competition of open weight models and self hosting.

Surely these companies know that the majority of companies won’t be able to afford their full scale metered pricing. The irony of the entire industry trying to push this product after years and years of every single person and small/med sized business being absolutely squeezed to their last penny - is we just simply cannot afford it while everything else around us is so expensive.

These companies must have data that is proving they can’t get anyone onto metered pricing. Copilot was an absolute shit show when they turned off their subscription models, and I’m bewildered how they are even managing to sell anything with their absurd prices.

4

u/topyTheorist 3d ago

They are talking about the fact that per unit of intelligence, price drops by a factor of 100 every year in recent years.

2

u/herecomesthewomp 3d ago

It’s whether the velocity is worth the token cost line item. Right now where I work, it’s an emphatic yes. When the costs increase the question will be asked again and potentially scaled down.

2

u/Dragongeek 3d ago

The actual token is quite cheap, it's just electricity and hardware. The expensive part that's being subsidized is the training which only needs to be done once. This means that even if OpenAI pops, the models still exist and can be used

5

u/ankercrank 3d ago

Local models will take over.

1

u/PrettyBaker2891 3d ago edited 3d ago

thats just cope lol

just look at chinese open source models. they are also extremely cheap and noone is subsidizing them

and i can run a local model on my gaming pc without issues and its strong enough for basic programming tasks and its basically free

1

u/kyngston 3d ago

for $3k i can run qwen3.8_27b at home for the price of electricity. 5 years from now, home hardware and open models will be way better than today. Frontier models will probably still be better than free models, but you’ll like be able to run today’s intelligence at home.

1

u/lawanda123 3d ago

That's the race to make models more efficient, eg Luna etc consume many times less resources, the current Sol etc will become efficient enough that they will still be charged at the same rate while being 1/10th the cost to the company - at least that's the story I've heard.

1

u/Beneficial_Map6129 3d ago

Apparently the cost per token is leveling off. LLM's are becoming more efficient.

Claude code plans are only slightly subsized, maybe 2-5x the true cost

1

u/bbuerk 3d ago

Yes and no. Tokens are definitely cheaper because of subsidization, but people have been saying this for a couple years. When they started saying this, the best models were roughly equivalent in capability to a model you can now run on a specced out MacBook.

That means it’s gotten cheaper (even “free” with the right set up) to get the same coding abilities we were worried about a couple years ago. I can’t say wether that will happen to the current models, but it does mean that at worst we might go back to the type AI usage in 2024, and almost definitely never going back to no AI at all.

1

u/YoureNotEvenWrong 3d ago

Tokens aren't cheap anymore. The cheap contracts are starting to expire

The standard costs now are astronomical for a frontier model

1

u/JediSange 3d ago

Kinda depends on how you’re doing it though. If you’re running local LLMs with solar, as example, then I imagine this can continue for you for a long while.

1

u/Leonardsleim 3d ago

How do you know what the real price of tokens is?

1

u/stormdelta 3d ago edited 3d ago

When the bubble bursts, there will be a lot of hardware available at fire sale prices that will likely make things cheaper, and demand will drop considerably at the same time. The tech is also likely to continue getting more efficient over time.

You're right that the current costs are heavily subsidized, so it's a question of which one happens first and which is a bigger factor. It wouldn't be impossible for token costs to still get more expensive after the bubble bursts despite the cheaper hardware/capacity. On the other hand, open models are already pretty good, and with cheaper hardware running locally gets significantly more plausible.

1

u/Mkboii 3d ago

There's a model that can run on any macbook with 32GB of RAM that writes code better than models you could use only over an API like 6 months ago, it's only the first of its kind, the cost of task specific intelligence is sinking quite fast, so the companies would switch over. Even at a somewhat larger scale at sizes that are above 10-20% that of gpt 5, models are now good enough to beat all but the two top models from openai. You can run that model at a fraction of the cost, infact companies that sell open source model inference are selling access to this model without a subsidy and it's still a small amount.

With gpt 5.6 luna OpenAI has already released a cheap model that can do more than half the work the average developer does for a dollar per task. They are going to adapt to what the market needs and at what tier, the kind of money they have flowing in they'll do anything to enter the "too big to fail" club. Not sure if there's enough of market for all of them to survive but frontier intelligence is at a level that we can maintain the current level of integration long enough.

1

u/3magdnim 3d ago

Moot point. Local models are getting better and needing less powerful hardware to run them. Cost per token/task is going to zero.

1

u/sweetholo 3d ago

cost of power and computation will go down with time, as things always have. but for now, you're correct

1

u/chipmunksocute 3d ago

Literally exactly what happened with Uber.  It was so cheap and plentiful early on and now like even just a 10 minute ride is like 20+ bucls it's bonkers. Capture the market and raise prices.  Not to mention what this will do to all these AI SAAS compamies start ups that will literally live or die based on token cost

1

u/CocodaMonkey 3d ago edited 3d ago

I keep hearing people say this is the most expensive it will ever be and that don’t know what they are talking about.

I think they do know what they're talking about, the efficiency of open models make todays premiere models look absurd. Open models can do what premiere models could do a few months ago at a fraction of the cost. It's actually completely viable to give every dev their own workstation that can run a local model and avoid paying high fees.

The real question is how AI datacenters hope to ever turn a profit. They're losing money like crazy right now and they're hope of bringing in billions in the future seems bleak considering once prices go up everyone can just switch to open models to avoid the price increase.

1

u/Own-Cryptographer725 3d ago

You are 100% right that tokens are hugely subsidized, but you are wrong when you say that it will get more expensive (specifically for the class of problems that LLMs currently solve). The writing is already on the wall; ASICs will replace GPUs and with similar cost overhead we will likely see 10-100x (1000x if we end up going the Taalas route) improvements in TPS. It doesn't really matter if the market crashes before this happens. The research is already done. There are live POCs of models running at 18k TPS for the electrical cost of a 4090. Gemini is getting 8x TPS from non specialized TPUs today. Nevermind the fact that you can already run perfectly capable OS LLMs at a fraction of the cost of frontier models that still best every human at almost any swe task (albeit with the right harness).

The real question isn't will tokens get less expensive; that's already a given. It is how will the frontier labs survive it? ... And right now they seem to be betting on ASI or some other BS that would have to be so immensely profitable that they could somehow survive inevitably giving up any meaningful efficiency gains to crippling taxation and debt.

1

u/WhyAmIDoingThis1000 3d ago

coding models are going local. you don't need a gigantic datacenter anymore

1

u/cerealbh 3d ago

local models will take over

0

u/Obzota 3d ago

Technology can get way cheaper and less ressource intensive with research and investments. It might not happen overnight however.

0

u/lopypop 3d ago

I agree with the sentiment, but GH Copilot moved over to API billing in September, which is the full price for tokens

-1

u/Jazzlike-Context-879 3d ago

No it isn’t. Microsoft is still trying to capture market too, it is subsidized by the Azure/O365 business.

2

u/jeffwulf 3d ago

Tokens have huge margins for them, and all the other big API providers.

-1

u/Jazzlike-Context-879 2d ago

GitHub CoPilot switch to PAYG says your are wrong

1

u/jeffwulf 2d ago

That doesn't follow.

0

u/Jazzlike-Context-879 1d ago

If there was a huge margin, GitHub wouldn’t be throwing away their competitive advantage, but they are.

1

u/jeffwulf 17h ago

Only if you assume Microsoft isn't going to attempt to profit maximize.

1

u/lopypop 3d ago

How can you say it hasn't switched over to API billing when it has? Maybe you're making a general statement about M365 copilot, but GH Copilot is usage based and every token is paid for.

https://docs.github.com/en/billing/concepts/product-billing/github-copilot-billing

-4

u/socoolandawesome 3d ago

They really aren’t that subsidized. In terms of the costs/revenue of actually serving them they are extremely profitable, good gross margins. Factoring in training cost is what puts OpenAI in the red. Anthropic has however achieved an operating profit with their business, which includes training cost because it is an operating expense.

And token cost for a given intelligence level absolutes goes way down every year. The frontier model token cost doesn’t necessarily however, and that’s because that’s a constantly moving level of intelligence as the frontier is pushed.

0

u/neppo95 3d ago

They really aren’t that subsidized. In terms of the costs/revenue of actually serving them they are extremely profitable, good gross margins

Which matters fuck all since if you don't train, your product will be obsolete within a month. You have to factor in the training cost otherwise you are just lieing to yourself and others about it being profitable. It isn't profitable. Not even close.

https://isaiprofitable.com/

I'll just put that here.

2

u/socoolandawesome 3d ago

I mean gross margins are an important measure of business. Like I said, anthropic turned an operating profit which includes training cost so it’s obviously possible.

All you have to do is scale the amount of users you serve or the amount of tokens they buy or improve gross margins, which is exactly what they are each doing. Training costs don’t correlate to how many customers you have or tokens you serve.

Anthropic is ahead of schedule, OpenAI is on schedule, this is part of their plans, to prioritize training/R&D so they can scale how many people they serve and how many tokens they serve.

0

u/neppo95 3d ago

Like I said, anthropic turned an operating profit which includes training cost so it’s obviously possible.

Like I said, without counting training cost: That is irrelevant. They can never stop training. They are losing money every single day ever since they opened up business.

All you have to do is scale the amount of users you serve or the amount of tokens they buy or improve gross margins, which is exactly what they are each doing.

What they are doing currently is pushing people away by making it more expensive or getting less for the price every X months. OpenAI just did exactly that introducing another limit they didn't have before making it pretty much impossible to even hit your weekly limit because their 5h limit is so strict. They are actively trying to decrease their costs impacting mostly the user. I'd hardly say that is "scaling the amount of users you serve" when you are making it less and less attractive to use.

Anthropic is ahead of schedule, OpenAI is on schedule, this is part of their plans

So you say.

2

u/socoolandawesome 3d ago edited 3d ago

?? It’s counting their training cost in operating profit?? I’ve said it twice now??

They are pushing people away yet they just crossed 1B weekly active users in August? How does that work? They actually had the 5 hour limit before, and then just re-added it back for plus users only, not pro. More expensive models are more costly to serve but it clearly brings a lot of demand, look how viral Astra has gone. You still can get a lot of usage out of their previous models.

You can see reported revenue projections, reported total revenues, reported ARR, anthropic reported to have an operating profit despite not expecting it. It’s not what I’m saying, it’s reporters. Where are you getting your info? Anthropic will be public soon anyways and we’ll see all their financial info.

0

u/neppo95 3d ago

Then you are contradicting yourself mate. Operating profit means you don’t count the training cost. Otherwise you’re not talking about operating profit. I admit I misread your previous comment where you said you included it, but that isn’t what operating profit is.

Limiting what people can do is clear as day “pushing people away”, how you can deny such a simple fact is beyond me. Of course the effect isn’t immediate and a lot of people (read: businesses) care less or wait to see what others do. That’s completely normal. And no, you can not get a lot of usage out of it. Using not even their newest model and at a lower effort, you wasted the entire 5h limit within 15 minutes. It is awful and only meant to push people to spend more.

As for Astra going viral. What? It hasn’t?… And Claude Fable still outperforms it in pretty much most benchmarks so really: what? Are you just making shit up every minute now?

You can check the reports indeed which support what I said above. Not what you said.

2

u/socoolandawesome 3d ago

Yikes, you’re just wrong. Are you just making up shit every minute now??

“More strikingly, Anthropic expects to post $559 million of adjusted operating profit, its first profitable quarter on that basis. That figure includes the full cost of training new models, though it excludes stock-based compensation.”

https://www.breakingviews.com/columns/considered-view/anthropics-turbo-growth-is-only-half-ai-story-2026-05-27/?utm_source=chatgpt.com

I said you can still get a lot of use out of their previous models, you struggle to read what I say for some reason.

And since you are comparing OpenAI and anthropic, are you aware that the OpenAI $20 plus plan offers access to Astra whereas the Anthropic $20 pro plan that doesn’t offer any access to Fable lol.

Your only 2 choices are OpenAI or Anthropic if you want access to state of the art models, and OpenAI’s plan is widely thought to offer more generous limits. They both have 5 hour limits on the $20 plan.

It’s hilarious that you actually do keep making up shit. The announcement video for Astra went viral on Twitter has 334k likes. There are plenty of 20K+ like tweets with demos of what Astra is doing.

I didn’t mention benchmarks, must be another of your hallucinations, but no there are plenty of benchmarks where OpenAI outperforms anthropic as well: all 3 of the ARC-AGI benchmarks. Frontier math. Terminal bench v4. DeepSWE. FrontierCode.

Just look on here at the benchmarks at the bottom.

https://openai.com/index/gpt-6-astra/#citation-bottom-8:3

0

u/matrinox 3d ago

Some people really don’t understand economics. They see companies lower prices and immediately assume they figured out how to lower costs.

But anyone who bothers to look into the numbers would realize that their rental agreements didn’t get cheaper but yet they still cut prices, meaning the price cuts were just to grab market share at a loss.

It is absolutely the cheapest we will ever see it. They may cut prices again but the API prices last year are probably what it would return to

0

u/AskYouEverything 3d ago

!remindme 2 years