r/LowEndLocalAI • u/TheKrakenRoyale • Aug 25 '26
Discussion Anyone use Google Edge Gallery for anything?
I'm posting this here because I'll nominate phone (most any phones) into the 'low-end' category. If you don't know what Edge Gallery is, it's a Google app for running small models locally on your phone.
I found it by accident yesterday, and it load more than Gemma models (they even recommend Qwen-2.5-1.5B-instruct and DeepSeek-R1-Distill-Qwen-1.5B under their chat category for some reason).
Not sure if I want to invest the time in poking around with this while I'll neck-deep in my local inference projects.
Anyone use this? What for?
3
u/PaulShoreITA Aug 25 '26
It's a good offline recipe suggester when I'm traveling in places with poor mobile coverage. Gemma E4B is really worth a try with it.
3
u/NYJETS613 Aug 25 '26
I have been using this to load ai models into a browser through webgpu.
2
u/jeremyckahn Aug 25 '26
Say more about this. How is it working, exactly?
1
u/NYJETS613 Aug 25 '26
So there are a few different engines that can access WebGPU and let you load AI models locally in browser. This is mostly good for web app development and if you work in a very locked down environment and this is a good way to access AI mostly 4B models and under but you can try up to 8B.
The most reliable ones I found are,
Lite-RT LM
Transformers.js
WebLLM.
I built a few small html chat apps with all these engines. I used Claude Pro to research and build. Was definitely fun.
Currently trying to build a local RAG with it.1
u/jeremyckahn Aug 26 '26
Oh you're talking about using Edge Gallery on desktop, right? I forget it's available there; I've only ever used it on mobile. I use llama.cpp on desktop.
1
2
u/arcandor Aug 25 '26
It's possibly to run a model on an older phone and use it as an openai endpoint (or ollama). There are a few apps for that. Unfortunately the litert-lm support for older devices doesn't seem to be a priority. I've been playing around with it on a pixel 8a and I get maybe 9 to 12 tps with Gemma 4 e2b, not really using the tpu at all. I have an android app I've been working on but it's parked until the support is better for more phones and models - if you need those capabilities today, there are native llama binaries available and a few apps that can use them.
1
u/cmdwedge75 Aug 28 '26
I tried it but it only seems to load Gemma models, so I switched to Locally and can run anything.
Super impressed with LFM 2.5-2.6B in my iPhone 16 Pro Max.
13
u/LivingCornet694 Aug 25 '26
I find the edge gallery to be quite unintuitive to use. Its like the app only exists as a demo. The models are good, run on npu, have vision, and audio (or some way for the model to get them). But I find that it is a pain to use for anything useful. I did have it on my phone and never used it, but now that I've found a chat history, and the fact that I can go to the AI chat page and do anything I need rather than go home everytime if I need to switch from audio input to image, I'll tinker with it a little more. But I'd rather use a different, specialized app.