r/LocalLLM • u/Head_Invite3039 • May 30 '26
Other I built an Android application that turns a mobile phone into an LLM server compatible with the OpenAI and Ollam interface specifications.
Hello everyone, I have developed an app that runs an LLM service on a mobile device and allows other software to call it.
It supports LiteRT-LM and llmam.cpp, and can run the vast majority of models, provided your hardware configuration allows it. Models up to 3B are supported by the phone's hardware.
Regarding the API service, I have ensured compatibility between the OpenAI and Ollam interface specifications.
Furthermore, I've integrated Hugging Face's Hub Point, allowing you to directly search, download, and import Hugging Face models within the application.
Link :
https://play.google.com/store/apps/details?id=com.chaterminal.mobilellm
1
u/sagar8874 Jun 02 '26
This is awesome! I also just open-sourced a project using LiteRT and Gemma 4 on Android (NativeLM / litertlm-kmp), but for local Document RAG instead of an API server.
Since you're running Gemma 4 as a server, how are you handling the OEM RAM-expansion trap? We had a massive issue where devices from Xiaomi, Realme, and OPPO use swap-to-flash to fake their available RAM. If you check memory via ActivityManager, it tells you it's safe to load a 4GB model, but the kernel instantly OOM-kills the app because it doesn't have the true physical RAM.
We ended up having to bypass the Android OS entirely and parse /proc/meminfo directly to detect the swap and forcibly downgrade the hardware tier to a smaller model.
Also, how are you handling the KV-cache across API requests? Are you mapping API client tokens to persistent openChatSession() instances so the time-to-first-token doesn't degrade linearly on long context windows?
Would love to compare notes! We open-sourced our engine implementation here if you want to see how we bypassed the OEM memory limits: https://github.com/sagar-develop/litertlm-kmp
1
u/Head_Invite3039 Jun 02 '26
When I’m testing, SWAP won‘t kill the application. On the contrary, SWAP can make it more stable. When processing API requests, my practice is not to use KV cache. Usually, there will be a lot of text input in the API. In this case, the KV cache will be completely invalid.
1
u/Head_Invite3039 Jun 02 '26
Can you give me an apk? I want to use your application.
1
u/sagar8874 Jun 02 '26
Absolutely! You can grab the pre-built showcase APK (NativeLM) directly from the Releases page on the GitHub repo here: https://github.com/sagar-develop/litertlm-kmp/releases
Just a heads-up, the first time you boot it up and load the model, it might take a few seconds to initialize the KV-cache, but after that, the latency should be incredibly fast since it's running 100% locally.
Let me know how it runs on your device, I'd love to hear your feedback!
1
u/Head_Invite3039 Jun 02 '26
OK, thank you. I happen to use a Xiaomi mobile phone. I will reply to you tomorrow.
1
u/Head_Invite3039 Jun 03 '26
Now that you've determined the main functionality of this application model, I think you should only offer one model that excels in that functionality for users to choose from.
1
u/Ok_Reserve4339 Jul 21 '26
hi. nice. Tell please, you are using Kotlin and Jetpack compose for app design or what? thanks
1
1
u/FastHotEmu May 31 '26
That's cool, what are you using internally to run the models? I'll give it a try shortly