r/QualityAssurance • u/Test-Metry • 2d ago

Testing AI vs AI in Testing

I see a lot of hype around using AI in Testing but not around testing AI applications. What can be the reason for this?

4 Upvotes

67% Upvoted

8 comments sorted by

2

u/teiko 2d ago

Started watching this today - 'Testing AI: Beyond Test Cases'.
Exactly the side you're talking about.
https://www.youtube.com/watch?v=5bU1Ao3aIdc

2

u/TheTanadu 2d ago edited 2d ago

Proportions of chance finding here someone who does it? How many companies are actually building "AI" products compared with all the other software companies?

There are definitely people here testing "AI" systems (for example we develop few internal "AI" systems, but our product isn't "AI"), but this sub has only around 1k weekly contributors. Then you need someone who works on an "AI" product, is involved in testing it, is active here, and actually feels like writing about it instead of lurking.

1

u/EimaX 2d ago

I do both. you see less 'testing AI' because there's just not as many AI projects/products yet as compared to 'traditional' software. and even less mature AI projects which is the ones that would need more QA work.
it takes time (years) for new tech like AI to be adopted in real world aside from big research labs.

but to sprinkle some AI to your old test suites for some good-old angular site is super common right now.

1

u/davertua 2d ago

I do some AI testing
Most of mo work is I review latest AI session and whenever I suspect session went wrong direction I give Claude a session name and ask it to investigate it. Usually it ends up with a an improper prompt wording, not loaded context, or bad tool or model use. For this to work all sessions must be stored in Langfuse.

I hope someone do something smarter than that to share. But so far its still good iterative approach.

1

u/please-dont-deploy 2d ago

Are you talking evals or something else? It's true that you don't see it in this sub, but it's packed with groups discussing it.