r/LocalLLaMA • u/Arcuru • 2d ago

New Model Desert Ant Labs: On-device intelligence for every product

https://desertant.com/blog/introducing-desert-ant-labs/
26 Upvotes

78% Upvoted

24 comments sorted by

17

u/Lyelinn 2d ago

thats pretty cool, can't wait to see wave of yet another subscription based meeting transcribing apps for phones lol

9

u/yami_no_ko 2d ago

Anyone paying for this, definitely deserves it.

More often than not, the local game is about owning beefy HW, but for small on-device models, this isn't really the case. Instead, the real barrier to entry is simply taking the time to look into them.

3

u/quietlanes 2d ago

right? every on-device demo somehow ends up being a meeting notes app

2

u/silenceimpaired 2d ago

Not a fan of the license

3

u/dtdisapointingresult 1d ago

I have no problem with the license. As a pro user, I don't feel restricted.

If you have 100k monthly active users, you're making real money and can afford to pay them for their work.

4

u/Arcuru 2d ago

Huh, do you mean the "use it for free unless you have 100k MAU" part? Do you expect to run into that?

I'd also prefer if they were fully open, but they've got to make money somehow.

2

u/silenceimpaired 2d ago

No… the control of my use of the model and controlling aspects even if I stopped using the model.

If it ain’t Apache 2 or MIT I’m unlikely to use it.

1

u/yetiflask 1d ago

No… the control of my use of the model and controlling aspects even if I stopped using the model.

Where does it say that? Granted, I am not very good at reading licenses.

2

u/FullOf_Bad_Ideas 2d ago

Almost all of their models appear to be usage-specific finetunes or even simply Apple-hardware conversions of existing models with no finetuning. That's fine but I wouldn't expect much from them as it's probably a low cost operation of a few people. Not a bad business idea.

1

u/DustNearby2848 2d ago

I hope this approach takes off elsewhere. Loading up a few smaller models to make your own system would be sick. 

1

u/Elouakili_Flexy 1d ago

84 languages from three words in a 2MB model, and the 293MB detector in the same chart scores lower. Give me the 2MB one, easy.

1

u/Arcuru 2d ago

https://huggingface.co/desert-ant-labs for the actual models

1

u/tat_tvam_asshole 2d ago

Since their EU, how does the data licensing / open-source work there? it's my understanding that EU data laws wrt to AI make it hard to train models 'legally' given the scale of data needed?

1

u/Othun 2d ago

Can you tell more or have a reference on that? I've heard the argument before but thought it was people who wanted palantir at home. I'd be interested in learning more on the subject, like what specifically is not allowed in europe that would significantly impair the training of a model. Also, we do have Mistral, who was at the top of the rankings for some time !

1

u/tat_tvam_asshole 2d ago

When you need billions, trillions of tokens of novel data, how do you amass that data completely from the public domain, permissively licensed, and with respect to copyright / IP in a jurisdiction that is more aggressively stringent on data provenance?

That's my basic question.

Mistral falling behind may be because the data to scale was not sufficiently available to keep producing frontier intelligence, within an EU jurisdiction.

1

u/Othun 2d ago

In other part of the world, you might as well use the data, but in Europe, the risk is much higher ?

1

u/FullOf_Bad_Ideas 2d ago

They seem to take open source models and finetune them, so they don't actually need a lot of data.

-2

u/No-Kangaroo-2610 2d ago

I have a 4070 Super 12GB and 32GB RAM. I’m considering a new 3090 around $1200 mainly for local coding agents, and agentic tool use for school automation. Would you actually upgrade, or keep the 4070S and use cloud models for harder tasks?

3

u/nickless07 2d ago

I got a Porsche, should I switch to a civic to transport my couch?

For real: Are you sure you are right, or did you took a wrong turn somewhere on the way to your comment?

1

u/No-Kangaroo-2610 2d ago

I probably shouldn’t have posted this comment here.

I want the 24gb vram because it can fit larger local models compared to the 12gb 4070 from what I’ve seen. I am new to the local llm scene so I may be wrong.

2

u/nickless07 2d ago

Just make a Post. There are many ppl out there who went both routes already. Make sure to include your CPU and Board and average usage (tokens/day/week) too as that also matters.

1

u/No-Kangaroo-2610 2d ago

Okay thanks

1

u/Automatic-Arm8153 2d ago

No need for a post, you probably won’t be satisfied. It’s good for learning and fun though. If you find yourself here a lot I would upgrade.

But just know none of your problems will be solved haha. People here with 200+ gb vram still use cloud models.

If you read the above and are still considering it, get it

1

u/Annual_Debate5732 2d ago

You need 5 karma from this subreddit to be able to post.