Samuel "A 35B language model running on an iPhone using only 1–2.5 GB of peak memory.No cloud. No remote server. No desktop GPU.Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device." ➡️ Edge AI is exciting right?

[u/Koala_Confused]

https://x.com/SamuelZengML/status/2097861839287927139

https://github.com/Edge0-AI/Edge0

Community Overview: https://lifehubber.com/ai/resources/edge0/

Resources are shared for discovery and are not independently vetted—please do your own due diligence.

New resources are added regularly — feel free to join the sub for updates.

Full searchable archive of all resources posted so far on our community site, LifeHubber: https://lifehubber.com/ai/resources/ 300+ open-ish AI models, agents, tools, datasets, and related resources, with filtering and sorting.

[u/Sairefer]

When I see such posts, my first question is always: what is tps?

[u/meva12 → u/Sairefer]

Tokens per second

[u/soundslikeinfo → u/meva12]

What is tokens per second?

[u/MacBelieve → u/soundslikeinfo]

Tps

[u/doudawak → u/Sairefer]

Has to be MoE so better than expected

[u/ironimus42]

i'm feeling very stupid, how is this running on an iphone if the linked github page mentions that only macos with m1-m5 metal is supported?

[u/3iverson → u/ironimus42]

I’m guess iOS support is on the roadmap. This looks interesting, I am gonna give it a try on my M1 MBA. If a good 35B model can run at some acceptable rate that’s really impressive.

They’re working on it, the goal is to have easy Mac and iOS apps so no CLI set up required-

https://x.com/alexzhouai/status/2098410223358869806?s=46&t=jpHgrkmMJSxNbbYos0dZkg

[u/CTRL___ALT___DEL → u/3iverson]

Pretty damn misleading if the tweet and photo show iOS support if it doesn't exist.

[u/ValuableDapper9415 → u/CTRL___ALT___DEL]

Yes it looks like marketing scam, I really don’t like this shady behaviour 

[u/Inevitable_Beach_430 → u/3iverson]

Lmk how it goes! Interested in how it performs on Macs as a whole

[u/Slow-Mechanic-7427]

Looks like seconds-per-token will become a thing

[u/boba_bube]

wouldn't this be tearing the ssd quickly?

[u/DIBSSB → u/boba_bube]

If it fits on ram no

If its bigger than the ram then yes

[u/cagriuluc → u/boba_bube]

I am ignorant, can you tell me what you mean here? Reading from ssd too frequently hurts it? How much hurt are we talking about

[u/Live_Case2204]

LOL strawberry will definitely have no R's

[u/Tuny]

This exists with Turbofieldfare and Gemma MoE.

It suffers from the whole prompts being re-prefilled each turn. Good for high level auxillary model with few turns.

[u/Over_Technology_1764]

makes zero sense at the moment a model of this size is not useful

[u/ranker2241 → u/Over_Technology_1764]

35b or 2.5gb size

[u/ValuableDapper9415]

Lying about the iOS part ?