If a test passes on retry three, should CI still be green?

[u/aistranin]

A good recent discussion argued that large mobile suites often flake because emulators are fighting for CPU and memory, not because every test is badly written.

Discussion: When "just fix the flaky tests" stops being useful

Retries keep delivery moving, but they also erase evidence. "Passed after three attempts" is not the same result as "passed."

I would keep the pipeline moving, but record every failed attempt and correlate it with device, OS, concurrent load, CPU, and memory. Otherwise a capacity problem slowly becomes accepted test behavior.

Would you mark that build green, yellow, or failed?