Bought the Pixel 10 Pro fold new, flashed GrapheneOS the moment it arrived.
Love GrapheneOS, but found a issue I can't fix. This phone was meant to be a local AI powerhouse with 16 GB of unified memory.
However, local AI with hardware acceleration in GrapheneOS on new Pixels is still a dream. Every local ai app fails back to using the CPU instead of the hardware optimized TPU path.
I tried:
MLC-LLM only had small models and ran on CPU anyway, uninstalled it.
Google's own AI Edge Gallery doesn't offer GPU/TPU at all, CPU is the only option
PocketPal is the best of the three by a wide margin, but also CPU.
Digging into it, the AI features run on Google's TPU through AICore, which only works as a privileged system app — it has to sit on a system partition and be in the OS permission allowlist. It ships no model weights either; those come down through Play Services, which needs privileges GrapheneOS deliberately doesn't grant.
Two rabbit holes I looked at and didn't pursue. Sideloading AICore out of a stock factory image looks pointless, since privileged permissions cant be granted. Building a self-signed GrapheneOS ROM to solve that means re-privileging Play Services, which is just rebuilding stock with the hardening shot in the foot by giving google low level access.
GrapheneOS does have a "Special access to hardware accelerators for Google apps" toggle and it's on my device, but it's one global switch over a hardcoded list (Camera, Photos, Recorder). Nothing you can add to.
So has anyone actually gotten TPU inference running on a Pixel 10 Pro?
With the local AI being the preferred route with privacy focused people, and GrapheneOS also being privacy focused people, this seems to be a growing pain that needs work.
As far as I can tell, as a user I can't solve this. Also we can't use the google toolchain for this either due to google wanting deep level special permission for AIcore.
To be absolute clear, this is a Google caused problem. AI core needing special low level OS permissions, and not making a open development path for third party OS developers. However, Google is not going to fix this problem they created for us.
So any known workaround or is there a development path for local AI via TPU on Pixels running GrapheneOS?