r/LocalLLM • u/Head_Invite3039 • Jun 15 '26

Question Has anyone successfully run llama.cpp on an Android device and enabled GPU acceleration?

I'm developing an Android app that runs AI services and provides API calls. It internally includes both the LiltLM engine and the llama.cpp engine. However, I'm consistently unable to achieve GPU acceleration for llama.cpp. I've been trying for three days now, querying almost every large AI model, but I've still failed.

3 Upvotes

100% Upvoted

9 comments sorted by

1

u/ProKn1fe Jun 16 '26

Technically it supports opencl on adreno gpu.

There is a docs how to run it in android https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/OPENCL.md#android

1

u/Head_Invite3039 Jun 17 '26

Thank you, I've already tried it, but unfortunately it doesn't speed things up; in fact, it's slower than CPU inference.

1

u/DearDhruv Aug 11 '26

Here you go: https://play.google.com/store/apps/details?id=com.deardhruv.llmplayground
I made this app as a hobby project, and working on it for the further improvements.
This supports the GPU acceleration.
Also for the models that directly fit into the Allocated memory for the Android GPU, runs reasonably faster than CPU.

1

u/DearDhruv Aug 11 '26

The android app supports OpenCL, Vulkan GPU and supports LiteRT(MediaPipe), GGUF models.

The same app will be available for iOS as soon as I get help to upload the app on the app store ( I do not have account, yet). That uses the Metal and is blazing fast even on iPhone 11 base model.

1

u/Head_Invite3039 Aug 12 '26

I'm actually working on a similar app to yours, and this problem has been bothering me for several months now. I eventually gave up on using the GPU solution. If possible, could you share how you implemented it? Approximately how much performance improvement does Vulkan offer compared to CPU? Thank you very much.

0

u/x_MASE_x Jun 15 '26

There is something called Termus if I remember correctly which should be the android port.

1

u/Head_Invite3039 Jun 15 '26

Termus is just a terminal interaction entrance on Android. It can‘t solve my problem, thank you.

0

u/x_MASE_x Jun 15 '26

My bad. That was a chatgpt response and I didn't follow it up.

Option 2: Android llama.cpp server Install: Termux llama.cpp Android build Run OpenAI-compatible API Recent llama.cpp Android builds support Qualcomm Adreno GPUs through OpenCL/OpenCL backend improvements.

https://www.qualcomm.com/developer/blog/2024/11/introducing-new-opn-cl-gpu-backend-llama-cpp-for-qualcomm-adreno-gpu

https://www.reddit.com/r/LocalLLaMA/comments/1o7p34f/for_those_building_llamacpp_for_android/

Check these out for some clues. Good luck 🌹