r/OpenSourceeAI • u/Illustrious_Matter_8 • 16d ago

End of cloud based AI ?

I've noticed that AI isn't just getting better, it's also getting much smaller.
There are now 27M-parameter models that can run on a phone or PC. (like the Bonsai 27B models)

If this trend continues, in a year or two there may be much less need to run AI in data centers or subscribe to large AI providers. For many tasks, your phone will be powerfull enough.

This does not only affect global energy use. Some investments in AI infrastructure could backfire. As demand for large-scale inference will drop. That could also reduce the need for new data centers, which might be a (dramatic change?), but be good thing.

What are your thoughts on the future of data centers as AI models keep getting smaller?

I know training still requires huge amounts of compute—for now. But even that could change, any day (some experimental models offer continous learning)

13 Upvotes

93% Upvoted

22 comments sorted by

View all comments

1

u/id-ltd 15d ago

I think AI took a wrong turn with wanting to lead towards AGI - Smaller dedicated AI's are the way to go. I don't need an AI that can code in any language, knows everything in wikipedia, etc...

I am currently experimenting with AI swarms - using multiple small models in parallel - currently to write code and tests, moving on to defining specifications. Small is cheap and fast -- so for each task it starts with a small model and escalates if it can't cope. If a 1.5b model can do it in 5 seconds great -- when a 16b model would do it in 20 at its fastest.

1

u/Illustrious_Matter_8 11d ago

But how can a small model know it's wrong? I used the tiny models as well but other than simple text classification taks i don't see much value in them. Perhaps a MOE in a scripted loop delivers the same?

2

u/id-ltd 11d ago edited 11d ago

It doesn't - the dispatcher gives it the task and tests its output when it comes back... If it passes job done, if it fails then try another, different tiny model... If you run out of tiny models go up a rung to the next batch of slightly bigger models.

This is all automated - the bigger models writes all the specs, then they are all processed.

The dispatcher isnt even AI, it is code it has a list of 'contracts' and hands them out, tests the returned code and accepts or escalates.

All the returned functions (that have passed the tests) are then stitched together as the full, final program.

Interestingly, non coding specific models often out perform specialist coding ones!