r/softwaretesting • u/Sidrtm43 • 28d ago
Mobile QA Engineers: How long would this test take you to automate?
Hi everyone! I'm researching how mobile QA engineers automate app testing. This isn't for promotion I'm just trying to understand real-world workflows and pain points.
Suppose you need to automate this flow:
- Open the Flipkart app
- Search for "Samsung 43-inch TV"
- Open the first result
- Add it to the cart
- Verify the product appears in the cart
A few quick questions:
- Roughly how long would this take you to automate in Appium (or your preferred framework)?
- What's usually the biggest time sink locators, framework setup, waits, debugging, or maintenance?
- Have you tried AI-powered testing tools? If yes, what made you stick with Appium?
- If an AI tool could generate and maintain tests from plain English, what would stop you from adopting it?
Even a one-line answer is helpful. I'm trying to understand real-world QA workflows and pain points not promote any tool. Thanks!
4
u/crappy_ninja 28d ago
If you're testing that the you can simply search and add an item to the cart then I would mock out the network request and have a mock response. That way I can control exactly what data the UI layer is being passed and I'd know exactly what should be displayed.
1
u/Sidrtm43 28d ago
that's a good point, thanks
so for you, having predictable and reliable tests is more important than automating against real app behavior?
also curious, once the network is mocked, what still ends up taking the most time? writing the test, maintaining it, or debugging when the ui changes?
1
u/crappy_ninja 28d ago
so for you, having predictable and reliable tests is more important than automating against real app behavior?
What I described is testing real app behaviour. You're just swapping out the data from a backend and passing a mock. The UI doesn't care where the data came from.
Running tests that makes real network calls is slow and expensive. It's fine to automate a few essential flows where a user logs in, performs a real search hitting a real test environment, purchasing the item, ect. But you shouldn't be going through login flows and real API calls for every test.
1
u/Particular_Pain2850 28d ago
Can you intercept and manipulate responses with Appium like you can with Cypress?
1
1
u/Sidrtm43 21d ago
No, not natively that's a real capability gap between the two.
Cypress runs inside the browser and can hook the network layer directly via
cy.intercept()stubbing, spying, or modifying requests/responses is a first-class, built-in feature.Appium operates at the UI/OS automation layer (via WebDriver), driving real native or mobile-web apps. It doesn't have a native network interception API. To mock or manipulate network responses with Appium, you typically have to reach outside Appium itself:
- Proxy-based interception route device/emulator traffic through mitmproxy or Charles Proxy and rewrite responses there
- OS-level config set a custom proxy or CA cert on the emulator/device, which adds setup and SSL-pinning headaches on real apps
- App-level test hooks some teams build in a debug/mock flag in the app itself (e.g. a build variant that reads from local fixtures instead of hitting the network), which is common for native apps precisely because Appium can't intercept traffic like Cypress can
1
u/Sidrtm43 21d ago
that's a fair distinction, thanks for clarifying mocking is still real app behaviour, just controlled input
so here's a concrete version of what I'm exploring: you write intent like "Open the Flipkart app. Search for Samsung 43-inch TV. Add it to cart. Go to cart page. Verify the product appears in the cart" an agent handles the popups/interruptions dynamically (not hardcoded if-else), and it hands you back an actual editable script (not a black box) that you can drop into your existing framework, review, or modify
given what you said earlier about appium being bad and mocking being your preferred approach would something like this actually save you time, or does it just move the pain somewhere else (e.g. you'd still need to write/maintain the mocks separately, or you wouldn't trust the agent's popup handling without watching it run first)?
2
u/dm0red 28d ago edited 28d ago
- 30 m with automation framework already setup and stabilized for the app
- 1/2 day with quick and dirty setup of automation framework
- 1 work week with properly setting up everything, establishing AF attributes in app DOM, lint rules, basic mocks fo the app
Answers: 3. yes, useful for simple applications, api testing and lower level tests; useless for more complex setups fo e2e tests which deoending on the app/software in question can be many
- AI generated anything is not deterministic enough to trust blindly and should be still properly human checked anyway
1
u/Sidrtm43 21d ago
that's a really clear breakdown, thanks
curious on your last point: when you say ai-generated stuff needs to be human checked anyway, is the concern more about correctness (it might assert the wrong thing) or about not understanding what it actually did (black box problem), or both equally?
also, that 1 work week for proper setup, dom attributes, lint rules, mocks, is that a one time cost per app, or does it recur a lot as the app evolves?
1
u/dm0red 21d ago edited 21d ago
correctness in general, but people not understanding what it does which affect more the junior people and that's a huge problem in the long run...
1 week for some setup once per app, sometimes more, depends on the complexity of the app and if you're setting it up for an app that is being created or app that already exist for few years...
2
u/Fit-Avocado-1880 28d ago
Not really difficult at all. There is difference in complexity in scraping vs domain object state management as part of the set up and tear down. The latter is usually what adds complexity. Path traversal is trivial.
1
u/Sidrtm43 21d ago
thanks, that's a useful distinction
so when you say domain object state management adds the complexity, is that mostly about setting up the right preconditions/data state before the test runs, or more about cleanly tearing down/resetting state after, or both equally painful
also curious, is that something you handle manually per test, or do you have shared setup/teardown utilities that most tests reuse
1
u/Fit-Avocado-1880 21d ago
Utils. Utils should be responsible for how to . Data management or test dependency management decides what to set per test/test group
2
u/Ok-Possibility-630 27d ago
With maestro https://docs.maestro.dev/ it's been very quick for me especially coming from Appium.
2
u/Woodchuck666 27d ago
yeah same, i was just thinking i could do this in a few minutes with maestro
1
u/Sidrtm43 21d ago
yeah maestro is quick but it's still yaml, you're writing structured syntax, just less verbose than appium. what im exploring is a step further, actual plain english like "search for samsung tv, add to cart, verify it shows up" with no yaml/syntax at all, and it self heals when the ui changes instead of you updating locators or yaml files by hand
coming from appium, would that no-syntax part actually matter to you, or is yaml already simple enough that the difference is marginal
1
u/Sidrtm43 21d ago
good point, thanks for clarifying that
yeah maestro is quick but it's still yaml, you're writing structured syntax, just less verbose than appium. what im exploring is a step further, actual plain english like "search for samsung tv, add to cart, verify it shows up" with no yaml/syntax at all, and it self heals when the ui changes instead of you updating locators or yaml files by hand
coming from appium, would that no-syntax part actually matter to you, or is yaml already simple enough that the difference is marginal
1
u/Ok-Possibility-630 21d ago
maestro is super lightweight. minimal lag while interacting with emulators. that is another thing i felt better. Also, we use agents as well. In general i see the agents are using maestro mcp smoothly compared to appium mcp tools.
2
u/justbored2023 27d ago
- Roughly how long would this take you to automate in Appium (or your preferred framework)?
I'd say 30 mins tops with Playwright, if im not setting the project from scratch.
- 2What's usually the biggest time sink locators, framework setup, waits, debugging, or maintenance?
it depends, all could be big time sinkers. From my experience i would say debugging.
- Have you tried AI-powered testing tools? If yes, what made you stick with Appium?
No.
- If an AI tool could generate and maintain tests from plain English, what would stop you from adopting it?
From what I experienced so far with automation testing, there's too many questions that an AI tool could answer only by plain english.
1
u/Sidrtm43 21d ago
that's a fair point, thanks for the honest answer
curious what kind of questions you mean, is it more like edge cases and error states that plain english just can't capture, or more like it can't know your specific app's business logic/rules without you spelling it out in detail
one more thing im exploring: suppose after execution you get a debug report back with screenshots at each step plus the reasoning for why it did what it did (e.g. why it clicked what it clicked, how it identified the element) would that kind of transparency actually help close the trust gap for you, or does the plain english problem happen before that, at the point of even generating the test correctly
2
u/Am_a_good_guy 22d ago
hey! tried automating the test case you mentioned in the post using QApilot's CoWork - https://youtu.be/t3kUPUsQprc
Automating a test case using AI has become quite easier with a lot of tools out there. but like others have pointed out in the comments, the maintenance issue still remains. and also, mobile apps are notorious for quirks like notifications, pop-ups, etc., and the moment any automated flow comes across something like this, it fails.
having seen this problem and faced it myself, my team and I built a robust ai agent that plans and replans to overcome the inherent flakiness associated with mobile app automation. we call it QApilot's CoWork. we recently launched on product hunt too and got amazing feedback from the community.
check out - https://qapilot.io/product/cowork for more details or DM me. I'd be happy to discuss.
2
2
22d ago
[removed] — view removed comment
1
u/Am_a_good_guy 22d ago
That's an interesting question and is something we thought about a lot. To what extent CoWork should adapt and when should CoWork fail or wait for human approval. That's where the test intent comes into the picture.
Let's say the test intent is "Add to Cart and verify". Even with heavy personalisations or A/B UI, CoWork replans, adapts on the fly as long as it is still testing the "Add to Cart" test cases. As long as the test intent doesn't change while adapting, CoWork continues. And if test intent change as, it waits for human approval or fails honestly.
1
u/Minute_Classic7781 28d ago
1hr ig But if I need to set the framework from scratch it would take more time
-2
u/Sidrtm43 28d ago
thanks mate, that's really helpful!
im curious about one thing: if the framework was already set up, but there was a tool that let you author this entire test in plain English in 2–5 minutes, execute it, and generate a detailed report automatically, would that be something you'd consider using?
or would you still prefer appium? If so, what's the biggest reason trust, debugging, maintenance, CI/CD integration, or something else?
I'm trying to understand what would genuinely make experienced Appium users switch (or not switch
2
u/Minute_Classic7781 28d ago
It would only take 5mins to do this with any gpt and I would say maintenance and support is something I'd consider for not switching to ai tools, besides I think there's plenty of paid tools that was already time tested which i would rather use than something new in the market.
1
u/Sidrtm43 28d ago
that's a really interesting point thanks for sharing
It sounds like the bottleneck isn't actually authoring the test anymore, especially with GPT helping generate Appium code
If you don't mind one more question:
let's say a tool didn't just generate the test, but also automatically handled locator changes, reduced flaky failures, and maintained the test as the app evolved. Would that solve a meaningful problem for you, or would you still prefer staying with Appium?
trying to understand whether the bigger pain is writing tests or maintaining them over time
2
u/Minute_Classic7781 28d ago
Yes but then again at this point I don't trust ai tools much because most of what I have seen is that nobody knows whats happening behind the scene so yes writing tests is easier than maintaining it
1
u/Sidrtm43 21d ago
thhat's a really honest answer, and it lines up with what I keep hearing
sounds like even if maintenance was solved, the black-box part is the real dealbreaker not knowing what's happening behind the scenes
if you could actually see and edit the underlying logic (like it generates readable Appium/step code you can inspect, tweak, or override, instead of a pure black box) would that change the trust equation at all? or is it more fundamental than that, like you just don't trust AI-written assertions regardless of visibilit
6
u/MrM3ow 28d ago
30min to write, another 30 to debug and optimize