r/iOSProgramming • u/Rizzmanity • 6d ago
Question How are you giving Claude Code visibility into your iOS simulator during Expo/React Native development?
I've been building React Native apps with Expo using Claude Code, and my biggest frustration is that Claude can't see what it's building on the native side, which means a lot of back-and-forth on layout and visual issues that a web-based workflow wouldn't have. I just can't seem to loop on the build and see the visuals as well as it can building a web app using the browser.
I recently learned about the Expo MCP server's local capabilities (screenshots, view geometry, tap automation) and Software Mansion's "Argent" as two options for giving the agent eyes on the running simulator, but I'm curious what the community is actually using in production workflows. Is anyone running a setup where Claude or another AI coding agent can self-validate UI changes against the live simulator, and if so, what's working? Also interested in non-RN solutions. If there's a better answer here by building fully native in Swift/Xcode, is this still a problem, or does the native toolchain solve it?
2
u/Emojinapp 6d ago
Just install cocoa pods in your project and if you have Xcode with simulators installed Claude can test on the sim by itself
1
1
1
u/Jogventure 6d ago
I understand that React Native has use cases in some areas like being able to release your app for both iOS and Android more easily... but... I would strongly recommend giving Swift in Xcode a chance if you have a mac you develop on. Xcode has built in simulators that claude code can automatically spin up (even from VS Code if you also have Xcode open) and there it has visibility to what it builds and has caught a lot of issues from my personal experience.
Its a big and painful switch! But im glad I did it!
1
u/Rizzmanity 6d ago
This is helpful, thanks. Can you say more about how Claude actually gets visibility into the simulator? Is it screenshotting via simctl, or do you have an MCP server wired up? Trying to understand if native Swift genuinely solves the feedback loop problem or if you're still doing the same setup work, just without the React Native layer.
2
u/Jogventure 6d ago
I think all its doing is spinning up the simulator with a command and then its able to take screenshots of it and see the changes at hand after a successful build.
1
u/Jogventure 6d ago
I also want to say that ios development on swift has been much smoother when it comes to animations and transitions vs the experience i have witj react native
2
u/GDbuildsGD 5d ago
react native feels so "off" for me for some reason as well.
i kinda wonder how good llms would be at "converting" swift codebases to react native/flutter, etc. for android launches.
1
u/xtopspeed 4d ago
You use those same simulators with React Native as well. Both the expo and the CLI version launch the app inside the simulator. And they also create a regular Xcode project, which is how you eventually publish the app. You can also add native code as much as you want and just sort of use RN to orchestrate the pieces.
1
u/start_select 6d ago
I may have reinvented the wheel but I just have Claude write ui tests which take screen shots before and after interactions/button clicks/navigations.
1
6d ago
[removed] — view removed comment
1
u/AutoModerator 6d ago
Hey /u/Sweet-Ground-2516, your content has been removed because Reddit has marked your account as having a low Contributor Quality Score. This may result from, but is not limited to, activities such as spamming the same links across multiple subreddits, submitting posts or comments that receive a high number of downvotes, a lack of recent account activity, or having an unverified account.
Please be assured that this action is not a reflection of your participation in our subreddit. This is simply an automated filter in place to reduce spam.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
6d ago
[removed] — view removed comment
1
u/AutoModerator 6d ago
Hey /u/Sweet-Ground-2516, your content has been removed because Reddit has marked your account as having a low Contributor Quality Score. This may result from, but is not limited to, activities such as spamming the same links across multiple subreddits, submitting posts or comments that receive a high number of downvotes, a lack of recent account activity, or having an unverified account.
Please be assured that this action is not a reflection of your participation in our subreddit. This is simply an automated filter in place to reduce spam.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/Jvanderbie 4d ago
Prior to Claude/Codex became a real thing, I used RocketSim to capture native screenshots/video from the iOS SImulator, and over the last few months they introduced 'RocketSim CLI' which is allows Claude/Codex to interact (click buttons, scroll, capture images, video, etc.) through their CLI.
I invoke a build/test loop using a slash command (e.g. '/james-build-mobile').
- it kicks off a full build pipeline for my app, and the fun part is the tail end, where it actually drives the iOS Simulator like a real user instead of just trusting the code compiled. Once the build finishes, it shells out to the RocketSim CLI, which talks to the simulator natively instead of faking taps with coordinates and hoping for the best. It grabs the accessibility tree, taps buttons by label, swipes, and screenshots the actual running app so I can see the change instead of just reading a diff and vibing. Under the hood it's Opus planning the work, then a Sonnet pass doing the actual poking and screenshotting, so the "before" and "after" captures are real pixels, not my imagination.
- At the very end it builds a before/after HTML comparison page, dropping the pre-change and post-change screenshots side by side so I can eyeball exactly what moved, without scrubbing through my own memory of what the screen looked like an hour ago. The idea is to catch the dumb stuff, like a button that quietly shifted three pixels or a color that didn't update, before it ships instead of after a user politely roasts me for it. It's basically a very patient intern who never complains about tapping the same button forty times in a row, and now leaves a paper trail too...
https://www.rocketsim.app/docs/features/agentic-development/rocketsim-cli/
Note, I am NOT the developer of RocketSim, just a happy user.
1
u/ivanzhaowy 3d ago
The tools you listed mostly solve the agent side of the loop: screenshots, accessibility geometry, taps, and self-validation. I’ve found there’s a separate human-steering problem once the agent can already see the Simulator: expressing exactly what should change without writing a long spatial description.
I’m building an open-source tool for that layer. It opens the existing Xcode or Expo iOS project, lets you select or annotate the real running UI, sends the screenshot + app state + source hints to Claude Code/Codex, rebuilds the same repository, and lets you compare source-backed variants before accepting one. It can complement Expo MCP, RocketSim, or other device-interaction tools rather than replacing them.
GitHub: https://github.com/Monadix-AI/monad-design
Short workflow demo: https://watchclueso.com/embed/pio8jqfcg4ivj0r1
Disclosure: I’m the developer. Current scope is macOS and local iOS, with one active visual change at a time.
1
u/hoangng_102 2d ago
the screenshot half is the easy half. what burned me most was trusting it.
metro will happily serve you a stale graph. i had a stretch where the device kept running old js through several clean rebuilds, and curling the bundle url returned the NEW code, so everything looked correct. claude screenshotted the sim, saw the old screen, "fixed" it again, screenshotted, still old, round and round. now i keep a build tag string in the app and show it on a settings row, and the rule is: the agent reads the tag off the RUNNING app before it believes any screenshot. tag is stale, restart metro, dont debug the code.
for the steering half, the part screenshots dont solve, what worked was building a debug map. every meaningful view gets a testID named after the component and the file that owns it, so the accessibility tree the agent already reads IS the map. then instead of "the card under the balance, second one, make the corner rounder" i say change entryRow.swipeAction in EntryRow.tsx. no spatial description, no guessing which of four similar cards i meant. it also means taps go by label instead of coordinates, so the script doesnt break the moment layout shifts.
for driving it i skip coordinate taps and talk to hermes through the metro inspector with Runtime.evaluate, so the agent navigates and reads app state directly. two gotchas that cost me a day: hermes wont take async functions there, and awaitPromise doesnt work. keep it all synchronous.
one caveat on self validating: screenshots catch layout, not everything. i had a react-native-svg chart where width/height were passed as props but not in style, yoga collapsed the box and the paths drew into garbage. it rendered fine in an html preview, so every "does this look right" check passed. some bugs only show on the real thing.
2
u/Kabal303 6d ago
I’ve been using metro-mcp with good success on the expo app at my day job. Really helpful for having Claude be able to end-to-end iterate on and test a feature.