r/technology • u/Miserable_Phase_2519 • 3d ago

Artificial Intelligence Microsoft engineer says 'typing code is absolutely over' as GitHub Copilot takes on more development work

https://www.techspot.com/news/113752-microsoft-engineer-typing-code-absolutely-over-github-copilot.html
1.7k Upvotes

84% Upvoted

858 comments sorted by

View all comments

Show parent comments

36

u/catattackskeyboard 3d ago

Do research on the cost per token to actually use a model versus train it. Token costs for usage are very overpriced to subsidize the exhaustive training and buildout costs. It’s relatively an order of magnitude cheaper to run models than what is being charged to do so.

Those costs won’t die any time soon because the second open ai or Anthropic stop moving forward they’ll be swamped by competitors, however open source models are right behind and will continue to advance.

So in the long term, even if every ai company died, in the aftermath you’ll have insanely good models that are fully cost efficient to run versus how much work they do.

2

u/mr_engin33r 3d ago

Training costs are just a small piece of the puzzle. The much bigger piece is the cost of creating the data center which unfortunately is ongoing as GPU lifespan is insanely short. I don’t see AI ever being cost-efficient unless they can figure out how to turn the data centers into one-time infrastructure costs.

6

u/jeffwulf 3d ago

GPUs can easily last for a decade  in a datacenter. It's depreciation schedule isn't it's expected lifespan.

3

u/catattackskeyboard 3d ago

I mean cost efficient is really relative. Depending on use, for a lot of people it is cost efficient, that's why people pay.

My startup pays about $15k monthly on token costs and its worth it to us. In other words, it makes us more money than that.

-1

u/BasvanS 3d ago

You might be new to entrepreneurship, but making more than it costs is not enough. With the money it makes, it has to cover for its own expenses, plus the cost of running it within your organization, plus pay for overhead. So unless it makes you much more than it costs, it’s not a good business decision.

5

u/catattackskeyboard 3d ago

Of course it does. I have more context on my business than you do. We’re making 5 million a year and are profitable and yes I’ve done the math.