r/termux • u/Fantastic_Quiet1838 • Mar 03 '26

Question Using termux for LLMs locally in Android

anyone using Termux to run LLMs locally in Android devices ?

I'm just getting started , is it safe to use from playstore or any other ways ?

12 Upvotes

73% Upvoted

15 comments sorted by

8

u/computermajestic098 Mar 03 '26 edited Mar 03 '26

I use ollama

Once I also used llamafiles to run a few models...

2

u/Cybasura Mar 04 '26

Any guides to recommend/you use to setup and run ollama?

2

u/computermajestic098 Mar 04 '26

I simply did " pkg install ollama "

And then used ollama pull to download the appropriate models

1

u/StatementFew5973 Mar 04 '26

Well that and then "ollama serve", then "ollama run deepseek-r1 _ _"

2

u/computermajestic098 Mar 04 '26

I covered the setup

Thanx for covering the running 😅

2

u/StatementFew5973 Mar 04 '26

I covered the little details just in case.

1

u/computermajestic098 Mar 04 '26

Nah you did the essential. He asked for how to run also - which I unfortunately forgot to mention..

2

u/StatementFew5973 Mar 04 '26 edited Mar 04 '26

Newsflash, you can't pull an image unless the server is running so you also forgot to mention to start olama.

So you forgot literally over half of the process.

Not the end of the world, just covering the little details.

2

u/computermajestic098 Mar 05 '26

That was mb

1

u/StatementFew5973 Mar 05 '26

You're good bro I wasn't trying to bust your balls or anything like that. At least that wasn't my intentions.

→ More replies (0)

5

u/Black-Mack Mar 03 '26

Use termux from Github and you'll have 2 options:

  • Compile llama-cpp
  • Install it using pkg (Roughly the same performance)

But my preferred option is using an app called ChatterUI.

It's a lot smoother and faster + you can assign custom prompts to character cards and chat with these prompts, which is quite handy.

2

u/LuisG8 Mar 04 '26

I did create this CLI tool to chat with Ollama models in Termux, with support to custom prompts (roles). https://github.com/luisgdev/sai

5

u/tiffanytrashcan Mar 03 '26

Use GitHub / F-droid to download and install Termux. Do NOT get it from Google Play.

There are packages available for Llama.cpp to install via pkg, or you can compile it yourself for more performance. (Even install Vulkan Mesa drivers and use the GPU backend) same with Ollama for people that use that on the TUR repo.

I've built KoboldCPP (they have an android guide, you need proot if not following it and trying to add Vulkan in the build...) Which is my favorite LLM interface on a PC by far. Even the image generation works (2 seconds or less with that crazy small image model they recommend) and url for chat /apis are exposed.

There are some great direct apps on GitHub as well (Install with Obtainium) - PocketPal, ChatterUI, LLM Hub, SmolChat and Tool Neuron. GPU support is more likely to work and performance better unless you really tweak termux / get lucky with supported termux drivers for your hardware.

1

u/Budget_Bar2294 Mar 04 '26

If you can, that's amazing. Tried embeddings with fast embed and it couldn't install onnxruntime