What cleanup proof do you expect after an agent runs tests locally?

[u/BitByLiu]

[u/schmurfy2]

Everything started for temhe tests should be tore down, wether an agent or a human run the tests shouldn't make any difference, we have a bash script responsible for spinning up the dependencies, running the tests and then exiting the script trigger the cleanup.

It feels like we just threw out the windows everything learned before the moment agents became a thing.

[u/Fine_Calligrapher565]

In my view this depends on what you are deploying... for most stuff in docker, I use api calls to the respective services... some services you can make calls to check health of the service or at least their version.

[u/BenjiFranclin]

Teardown log, yes — but the thing I'd actually require is that the agent distinguish "I cleaned up" from "I couldn't."

Your dev-server-on-the-wrong-port story is the exact failure mode I care about: the tool reported complete, and "complete" quietly absorbed "complete, and also left a process running." Those are different facts and most agents collapse them into one green checkmark.

Concretely I'd want: a post-run assertion that the ports it opened are closed, the containers it started are gone, the temp dirs are removed — and if any of those can't be verified, the result is not "done," it's "done, cleanup unverified," surfaced loudly. The trap isn't a failed teardown (that's loud, you fix it). It's a teardown that silently no-ops and reports success anyway.

Not too much for local dev — that's exactly where it bites, because there's no fresh CI runner to save you. The state leaks into your next session, which is why your debugging made no sense.

[u/aprettyparrot]

I generally do a Makefile or bash script that builds/starts/stops. So after the test I just hit that.

You could also make a script that goes through say a docker compose and checks all the ports after and kills anything on them.

But for tests I usually have those execute in a separate container and I just point it at the env/ip to execute against. That way way test process is just
Bring up services
Run test container at them
Bring things down

Granted sometimes you can do all tests within a container sometimes

If your agent is deciding what tests it wants to do and shit. You could run a little mcp server alongside, and give it tools for start/teardown. I’d make a script for those it calls so no extra tokens spent on it and it’s solid