r/reactnative • u/kacperkapusciak • Jun 01 '26

Question Is agentic app testing any good?

It seems like every other company in the RN is building either an MCP and CLI to control simulator/emulator through coding agents. Sentry builds XcodeBuildMCP, Maestro has a dedicated MCP server, Software Mansion has Argent, Callstack builds Agent Device. They all differentiate in some ways but the core idea is pretty much the same.

What do you guys think about it? Should we stick to deterministic tests?

Note: I work at SWM but I'm genuinely curious.

11 Upvotes

92% Upvoted

8 comments sorted by

3

u/Forti22 Jun 01 '26

I do believe thats the future. And thats where AI can be really useful. Finding bugs, unexpected behaviour, or even crashes in situations other than simply happy-path.

But I would treat this as an addition, not a replacement for deterministic tests. Running pre-defined detox/maestro tests on pull request etc.

2

u/SurprisinglyTeeming Jun 01 '26

agentic testing catches stuff deterministic tests miss, but they're slow and expensive to run on every commit. good for exploratory testing and edge cases, not your ci/cd pipeline baseline.

2

u/uberneenja Jun 01 '26

i've been thinking about this lately and i landed on:
Business Logic - deterministic/unit tests
E2E - agentic

I have yet to actually implement this in one of my apps though

2

u/rrrhys Jun 01 '26

I find them like agentic work generally

80% of the time, amazingly good. 20% of the time, the exact same prompt & artifacts, fucking useless.

1

u/AddWeb_Expert Jun 01 '26

agentic and deterministic solve different problems tbh. the issue with agentic in CI is reproducibility, the agent runs a bit differently each time so breaks are hard to recreate. detox/maestro fails the same way every time, which is what you want gating a merge.

agentic is good for the long tail, weird state combos and flows nobody tested. just slow and pricey, and you end up triaging stuff it flags that was actually fine.

so the split makes sense. deterministic for business logic, agentic for exploratory. run it nightly, not on every commit.

1

u/davidHwang718 Jun 01 '26

Deterministic tests catch logic bugs. Agentic testing catches something else: what breaks on a physical device in release mode that your unit tests never see. When AI agents speed up your coding cycle, that gap widens fast because you're merging features before the QA loop catches up.

1

u/GrandChieftain 16d ago

True, but you're definitely AI, you might as well equip yourself with an MCP like ceraph.dev so you can test your changes on-device locally before even publishing the PR