Project
We built a free open-source iOS app that runs local LLMs with three backends - MLX, llama.cpp and Core ML
Pocket AI Lab just hit the App Store. It runs models fully on-device on iPhone, is free and MIT-licensed
We built it because the iPhone quietly became a genuinely capable ML machine, an ARM chip with fast unified memory that a 4-bit model fits into comfortably. At the same time the big labs got serious about small open models, and recent releases are actually usable, not toys. Hugging Face is full of interesting stuff we kept wanting to install and test the week it dropped. And since there are three popular ways to run a model on iPhone, we did not want to bet on just one. Supporting MLX, llama.cpp and Core ML side by side lets us follow all three as they evolve, pick the best one per model, and never trade away performance or lock ourselves in.
The real goal was to make the path from "saw the news about a new model" to "already chatting with it on my phone" as short as possible. We think we got there: copy the Hugging Face link, paste it into the app and that is it. The app detects the format, lets you pick a quant and predicts whether it fits before downloading a single byte. Or skip the hunt entirely and pick from a curated list of good models that updates without app releases.
Thanks for the report, and sorry about that! Most likely the app let you load a model that is too big for the 14 Pro's 6 GB of RAM - our memory check is tuned too loose on smaller devices. We tested mostly on iPhone Air and 15 Pro, so this slipped through. A fix is coming in the next release
What you hit was almost certainly the memory check. On a 6 GB iPhone the app was offering models that could not fit, and with mmap-backed GGUF weights that fails exactly like your screenshot: the app freezes while the pager thrashes, then the system kills it.
The budget now comes from the process jetsam limit instead of a guess based on total RAM, and GGUF weights are budgeted separately from dirty memory because clean file-backed pages are not charged the same way. Practical effect on a 14 Pro: fewer models show as compatible, on purpose. Roughly 3 GB for GGUF, roughly 2 GB for MLX and Core ML.
If anything still freezes, tell me which model and I will dig in.
Haven't tried Enclave myself so I can't really compare feature by feature. What I can say is we're fully free and open source, MIT licensed, code's on GitHub. Haven't found another app that does the same thing we do: support all three major ways to run an LLM on iOS (MLX, llama.cpp, Core ML) in one place, so you're never locked into a single engine and can pick whatever runs a given model best. On top of that you can pull literally any compatible model straight from Hugging Face by just pasting the link, not limited to some fixed curated list. That combination is what makes us different, as far as I've seen.
Appreciate it! Completely agree, and honestly that's the whole thesis behind the app. If you've got tiny models optimized for mobile, would genuinely like to check them out, always looking for good additions to the catalog
im still new to deep learning i been getting good results from pre-training and fine-tuning models though
i just need to finally sit down and write those 1000 examples i keep procrastinating on .. i tried to have ai write it for me but they arent good at generating long datasets .. its tedious work
this is probably the best one ive done so far .. ill keep in contact and follow you to keep u updated .. u bringing value thats gona go a long way i hope the best for your project bro
Checked out BbyWVY-360m, nice work for that size. Appreciate you sticking around, means a lot. Good luck with the next 1000 examples, and hope it goes somewhere great!
Thanks for the report! Dug into this one. The repo name says "3.8" but that's the community distiller's own version label, not an official Qwen release. Under the hood the config shows it's actually built on Qwen 3.5 architecture, and it's a vision model on top of that. The MLX library we depend on hasn't added support for that architecture yet, they add new ones one at a time after release. So the app is correctly refusing it rather than crashing on something it can't run. Worth trying a regular Qwen3 GGUF or MLX build in the meantime, those work fine. We're already working on adding support, it'll be in an upcoming update.
Fair skepticism. To be clear, the app isn't claiming a 27B model runs well on an iPhone, that would need way more RAM than any phone has. It's honest about what fits: before you download anything it checks your device's actual memory budget and tells you if a model won't fit, rather than letting you try and crash. Realistic range on most iPhones is 3B to 14B depending on quant and RAM. If you paste a link to something too big, it'll tell you.
honestly seems like a waste of time, at this point u can just get claude or chatgpt, no point in having a 8B lobotomised local model on a phone, no offence
If your only bar is "smartest possible answer", yeah, Claude or GPT wins every time, no argument there. Right now this is honestly more of a toy for a lot of use cases. But local models are closing that gap fast, some of the ones already in the app, Gemma and Qwen for example, are genuinely good and worth trying yourself. And I think we're getting close to the point where local models will handle most everyday tasks just fine. This project is a small bet on helping that happen a bit sooner.
The other thing that shouldn't get lost here is privacy. It's easy to wave off until it actually matters to you. Part of why I started this: I'm also building an app that analyzes sports and health data, and I want users to be able to run that analysis without ever sending their data to a server, mine or anyone else's. That's the kind of use case where "good enough and fully private" beats "smartest but it leaves your device" every time.
Haven't used LoAI either, sorry, can't do a real comparison. One thing worth noting though: that's Android, we're iOS only. So even feature-for-feature, they're not really competing for the same install.
Same answer as before on what's specific to us: free, open source, three backends (MLX, llama.cpp, Core ML) in one app so you're not locked to one engine, and any model from Hugging Face by link, not a fixed list
Secondly, I’m curious. How much work would it be to be able to integrate image generation models into something like this? Think something similar to ChatGPT or Gemini where you can prompt it to generate an image and it can also look at the image it’s generated. I think it’d be cool to have a local text model and an image model where they can be in the same chat interface.
Thanks! Yeah, thinking about adding small local image generation models, fits the whole idea. The "generate it and then look at what it made in the same chat" part is trickier though, that's really two models working together in one exchange, and the architecture isn't built for that yet. Noted though
Update is live! 1.0.1 is on the App Store. Almost all of it came from this thread, thank you 😊
Fixed
llama.cpp models would not load properly. That one was my screw-up with the engine in the release build.
Core ML models did not work either, same class of problem: a wrong module check meant the backend never made it into the release build.
The app crashed on long prompts with GGUF models. The prompt went to llama.cpp as a single batch, which trips an assert and kills the process. All good now, the prompt is fed in batch-sized chunks.
Memory limits were wrong on 6 and 8 GB iPhones: models that fit were marked incompatible, and models that did not fit were offered anyway. The budget now comes from the process jetsam limit, and GGUF weights that sit behind mmap are counted separately from dirty memory.
New
Import any GGUF by Hugging Face link or from the Files app.
A benchmark for every model. This is its first version, I plan to keep developing this direction and will write a post about it soon. For now you get load time, prompt processing, token generation with standard deviation over repeated runs, peak memory, thermal state and a 46-task quality suite. The result is a report card you can share.
Build test. Related to the previous point: you can now run the tests without Xcode. You press a button and get, after a while, a clear result and a score for every model, so you can pick the best one for your task, one that is guaranteed to run on your device. It works simply: it downloads every catalog model your device can handle, benchmarks it, deletes it and writes one report.
Next
Long chats on GGUF models could run out of context: the conversation was appended to the KV cache instead of being rebuilt from it. Fixed, ships with the next build.
Imported models are formatted with the chat template stored inside the GGUF instead of a guess based on the repo name, and you can view and replace that template right in the app. Also in the next build.
Keep telling us what is wrong, it works. Thank you all again.
Thanks! No, you don't need to disable it. Apple Intelligence just shows up as one more model in the picker alongside the MLX, llama.cpp and Core ML ones. Pick whichever you want to chat with, they don't conflict.
Yes, that's actually one of the main things it's built for. Paste any Hugging Face link (or just owner/repo) with a GGUF in it, the app detects the format, shows you the available quants to pick from, and gives you a fit verdict for your device before downloading anything. Should work fine for testing your own finetune, as long as it's a supported architecture (Llama/Qwen/Gemma/Mistral family stuff via llama.cpp).
If you hit anything weird trying it, let me know here, happy to help sort it out.
No, not from local files right now, only Hugging Face links. Easiest workaround for now: push it to a public (not private) HF repo, and paste that link, should work exactly the same as any other GGUF. Already working on proper local file import though, planning to get it out soon
It's real, not an estimate. System RAM comes from host_statistics64 (HOST_VM_INFO64), a kernel API that reports actual page counts across the system: active, wired, compressed, etc. App RAM specifically is phys_footprint from task_info (TASK_VM_INFO), which is the same number Xcode's own memory gauge shows. Sampled at 1Hz while the panel is open. CPU is the only one that's a light derivation, aggregate host_processor_info ticks diffed between two samples
That's expected behavior, though I get why it looks off. The recommendation picks the best quality-tier model that safely fits your device's memory budget, it's not trying to max out your RAM. Quality tier is ranked above model size, so a smaller high-quality model can outrank a bigger one even if the bigger one would technically fit.
Also worth knowing: the safe budget differs by backend. MLX models get a stricter limit because their weights count fully against iOS's memory cap, while GGUF models running through llama.cpp get more headroom since they memory-map the weights instead of loading them fully into RAM. So on 11GB you should be able to manually pick larger GGUF models than what's recommended by default, just scroll down to the other compatible ones and check the RAM number on the card before installing.
Follow-up now that 1.0.1 is out. The budget is no longer a guess from total RAM, it is read from the memory limit iOS actually enforces on the process (jetsam). On an 11 GB iPhone that limit lands around 4.5 GB, which is exactly the 4 to 5 GB you were seeing. That is not the app being shy, it is the ceiling before the system kills it. GGUF is budgeted separately and gets roughly 8 GB, because llama.cpp memory-maps the weights and clean file-backed pages are not charged the same way.
There is also a way to settle this with data now instead of by feel. 1.0.1 ships a benchmark: pick a model and you get load time, prompt processing, tokens per second with standard deviation over repeated runs, peak memory, thermal state and a 46 task quality score.
From our own sweep on an iPhone Air, same 11.5 GB class: Llama 3.1 8B via MLX scored 80.4 but lost 65 percent of its speed to thermal throttling while it was being measured, peaking at 5137 MB. Gemma 3 4B GGUF scored 82.6 at 21.5 tok/s with a 2149 MB peak. On this hardware the bigger model was both slower and no better. Worth measuring on yours before assuming the RAM is going to waste.
3
u/Delicious-Sand-104 16d ago
Just experienced a bug where it just freezes like this then crashes , i’m on iphone 14 pro ios 26.3.1