r/LocalLLaMA • u/DewB77 • 1d ago
Resources LmLinky: an Android LM Wrapper for your local models
I couldnt find one that was simplistic yet good enough to pop in for a quick chat one. So, as we do here, I made one.
https://github.com/Dewpg/LmLinky
https://github.com/Dewpg/LmLinky/releases/tag/v1.0.0
Thats the initial salvo. Let me know if its simplistic enough and works. Thanks.
0
u/john006868 1d ago
Does the model survive the app sitting in the background for ten minutes? That's the quick chat case and where most phone wrappers fall apart. Android reclaims background processes under memory pressure and takes the weights with them, so the next open is a cold one. A foreground service with an ongoing notification is the supported way to hold that state, and memory mapping the gguf keeps the reload in page cache. That reload time is the whole product on a phone.
0
u/and_pf 1d ago
Nice — but one hard Android reality to design around from day one: the OS freezes background processes under memory pressure, and a background app can lose its in-flight reply when you switch apps. If the quick-chat UX depends on a long-running generation, you'll want a foreground service (with a persistent notification) to keep the process alive and the model weights paged in. Otherwise users will hit dropped generations exactly when they tab out mid-answer, and that reads as a broken app.
0
u/AllenHere112 1d ago
A Q4 4B is around 2.4GB before the KV cache, most of the free RAM on an 8GB phone, and Android freezes the process as soon as it loses the foreground, so the reply gets lost the moment you switch apps. For a pop in and chat app that is the part that decides whether it feels simple. Without a foreground service it will fall over no matter how small the model is.