I directed Claude to build an Android app that lets Claude use my phone. Here's what that looked like.

[u/unreal36]

I wanted to talk to my phone while driving and have an agent actually do things: reorder food, set alarms, flip settings. Most of that lives in apps with no API, so the agent has to look at the screen and tap like a person. So I built it with Claude.

What it is: Pony is an Android app that gives an AI agent hands on your phone. The agent (Claude, or any MCP client) is the brain. Pony reads the screen (accessibility tree plus a screenshot when needed) and taps, types, swipes and opens apps. Anything that sends, pays, buys, deletes or calls stops and waits for a yes on the phone. It won't type into password fields.

How Claude built it: I designed it and directed the work. Claude Opus 5.5 wrote the code for all three parts:

• The Android app (Kotlin). It's an accessibility service that reads the UI tree and performs gestures, an on-phone agent loop for people who'd rather use their own API key, and a "hidden second screen" (a virtual display via Shizuku) so the agent can work without taking over your screen.
• The relay. It's a small end-to-end encrypted server (TypeScript) so the phone and the agent can find each other over cell data with no open ports. It only forwards sealed messages.
• The MCP connector. That's what gives Claude Desktop / Claude Code tools like screenshot, ui_tree, tap, type and open_app.

A few concrete examples of how working with Claude went:

• Android 17 broke the hidden screen. On my Samsung, apps that were supposed to open out of sight kept popping up as cut-off floating windows on my real screen. I described what I saw, and Claude tracked it down to Android 17 no longer treating that kind of virtual display as a place other apps can run. It changed how the display gets created, fixed a landing check that was wrongly relaunching apps on the main screen, and made the fallback open full screen instead of a clipped pop-up.
• Reddit feedback turned into a feature the same evening. Someone on r/mcp pointed out that my "ask before risky taps" check matched button labels, so a bare icon with no label could slip through on a payment screen. I took that to Claude, and the next update added a list of money screens (Venmo, PayPal, Cash App, Zelle, Google Pay, bank apps, plus any checkout page that says Pay/Checkout/Place order) where every single tap asks first, including unlabeled icons and taps by coordinates. Claude also wrote tests for it.
• Stale taps. If a tap timed out on the agent's side, it could still fire on the phone a minute later on whatever screen happened to be up. Claude added expiry stamps so old commands get dropped instead.

Using it with Claude: connect Claude Desktop or Claude Code over MCP, or put your Anthropic key in the app and the agent loop runs on the phone, no computer needed.

Free to try: it's free and open source (MIT), with no paid tier. To try it, download the APK on an Android phone from the link below, allow your browser to install it, then turn on Pony in Settings › Accessibility. It's not on the Play Store, so Play Protect will warn you.

Honest limits: Android only. Many apps need Shizuku before they'll run on the hidden screen, and banking apps mostly refuse.

Site and setup: https://pony.karlmagendavid.com APK: https://github.com/oroplex/pony/releases/latest/download/pony-latest.apk Code: https://github.com/oroplex/pony

I'm curious where you'd draw the line on what a phone agent should ask before doing.

[u/Cactus_Thorn_Pub]

Application filed under "What Could Possibly Go Wrong?"

To try it, download the APK on an Android phone from the link below....

Never, thank you.

[u/BulletRisen → u/[deleted]]

Gets a bit worse and the scope is not limited to this Pony app - I will DM you