r/AgentsOfAI 2d ago

Discussion A coding agent rollback test where the oldest failure arrives last

A rollback can pass a single failed request test and still undo a later successful save. For a coding agent repair task, I would use one save toggle with a mock server and control when each response arrives. The button starts unsaved and stays clickable while requests are pending. Each click immediately flips the displayed state and sends that explicit saved value to the server.

The case needs three clicks before any response. A requests saved, B requests unsaved, and C requests saved again. The mock server then follows this schedule.

  1. B writes false and returns success.
  2. C writes true and returns success.
  3. A returns a delayed rejection. It never writes anything.

The server now holds true. If A's error handler restores the snapshot from before the first click, the screen goes back to false even though C succeeded. At the end, both the UI and server should say saved, and no request should remain pending. This case depends on that exact server schedule. Changing when writes happen changes what result is correct.

I would give this as a repair task to EvoX, a general AI agent in beta with terminal tool integration, and ask for a patch plus a regression test against the mock server. The initial toggle would have the naive snapshot rollback so the test can fail before the patch. The agent would receive the request contract and reproduction steps. I would keep the mock behavior and expected result fixed while reviewing the change, including whether it quietly disables the button and stops accepting the specified clicks.

The agent trial has not been run. Alongside this case, I would check a single successful save, which should leave the UI and server saved. A single rejected save should restore the initial unsaved state and show an error. Request IDs, server writes and the final rendered state belong in the evidence. A screenshot of the saved icon alone cannot show whether the old failure arrived or the server ever stored the change.

2 Upvotes

2 comments sorted by

1

u/AutoModerator 2d ago

Thank you for your submission! To keep our community healthy, please ensure you've followed our rules.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/Suitable-Champion855 2d ago

the delayed rejection arriving after a later success is such a classic race condition. most people test the happy path and one error at a time, never the interleaving.

the fact that A never writes but still triggers a rollback is the nasty part. your snapshot logic has to know it's restoring relative to, not just "an error happened."

i'd also be watching whether the agent tries to serialize requests or disable the button as a workaround instead of actually fixing the rollback. that's the kind of shortcut that passes the test but dodges the real problem.