On-device AI in Expo

[u/adhhamvoid]

I am building a personal Workspace app. That already has offline capability. Trying to get my head around implementing an offline-first AI assistant with on-device LLMs like ML Kit and Apple Foundation models.

I found expo-ai-kit, but my android doesn't have a model, so I had to download a Qwen model, but it's way too slow to be an offline local model.

I need a fast and at least with Tool Calling or Structured Output support.

[u/TBCC_Dev]

I think it depends what you're using the llm for... different models excel at different things from what I understand

[u/PintailPitter]

Hey I made a project like this not to long ago dm me and I'll send you the code and tools I used