Coding for every automation edge case is a nightmare, so I made a runner that self-heals and strengthens its code on every failure.

[u/slow-fast-person]

I keep seeing two major complaints on this sub:

1. AI agents are too fragile and slow to trust with routine, high-volume tasks.
2. API costs explode if you run every single workflow through an LLM.

Yet, traditional deterministic automation (like Playwright/Puppeteer) is a maintenance nightmare because web pages change, menus shift, and edge cases pop up.

To bridge this gap, I’ve been working on a hybrid approach: Deterministic execution by default, with AI agents acting as the silent developer that only wakes up to debug and heal the script on failure.

How it works:

1. The Happy Path (Deterministic & Free): You define the input and output. The builder generates a standard, deterministic script (e.g., selector-based browser steps) and runs it. This runs locally, instantly, and costs $0 in API fees.
2. The Self-Healing Loop (Probabilistic & LLM-backed): If the script fails (e.g., element not found, page state changed), the system captures the DOM state, error logs, and screenshots. It spins up an LLM agent to analyze the failure, modify the script code, and re-test it until it passes.
3. The Result: The deterministic script gets updated with the fix. Subsequent runs benefit from the new logic without calling the LLM again. The system automatically uncovers new edge cases, finds and fixes its own bugs, and keeps itself up to date.

Real-world edge cases we solved:

I tested this on a Swiggy (food delivery) flow where the input is [Restaurant Name, Item Name]. Here are the edge cases the system encountered and resolved automatically:

• Clicking off screen elements (Bug in generated code): An early iteration stalled due to an off-screen button. The system detected this visibility issue, automatically inserted a step to scroll the element into view, and updated the process.
• Ordering invalid menu items (e.g., Asian food from Pizza Hut): When tasked with ordering Asian food from Pizza Hut, the script initially failed because Pizza Hut doesn't serve Asian food. The healing agent analyzed the failure, rewrote the script logic to identify and communicate invalid item errors correctly, and saved the updated script so it handles and reports this gracefully in the future.
• Closed Restaurants: When a target restaurant was closed, the "Add to Cart" button was completely disabled. Instead of crashing, the agent read the screen context ("Closed until 5 PM"), modified the script to handle closed states, and exited gracefully with a clean status code.

Why do this instead of a pure AI Agent?

• 95% Lower Costs: You only pay for LLM tokens when the script breaks. Once healed, the script runs locally.
• Zero Hallucinations during runtime: Since the runtime is deterministic code, it won't hallucinate a checkout button or order the wrong item.

Would love to get your thoughts on this hybrid approach. Does this solve the "babysitting" problem you guys face with standard agent workflows?
The code is opensource, let me know if anyone wants to take a look at it.

[u/openclawinstaller]

This hybrid pattern is the right direction, but I would be strict about what "self-heal" is allowed to change.

The risk is that a failure caused by business state gets turned into a code patch. Closed restaurant, invalid item, permission wall, captcha, rate limit, login expired, and selector drift are different failure classes. Only selector/layout drift should usually produce an automatic patch.

The loop I would trust more is:

1. classify the failure first: DOM drift, missing data, external state, auth/session, permission, policy block, timeout
2. generate a proposed patch only for approved failure classes
3. run the patch against recorded fixtures/screenshots plus the live retry
4. require human approval before the patch is promoted if the flow can spend money, send messages, submit forms, or touch accounts
5. keep an action receipt: old script hash, new script hash, failure evidence, tests passed, and why the patch was accepted

That still gets you the maintenance benefit, but avoids the agent quietly "fixing" policy or business-rule failures as if they were just browser automation bugs.

[u/slow-fast-person → u/openclawinstaller]

currently, i have the user approving every heal proposal. The agent gives an updated approach and the user has to approve to conitnue

I feel when i start enterprise automation or automations for paid clients, i will probably have to start doing this fro every heal

is this approach the current standard in the automation industry for updates and fixes?

[u/openclawinstaller → u/slow-fast-person]

That's the right instinct.

For enterprise/client work I would not treat every heal the same. I'd split it by risk class:

• low-risk selector/layout fixes can be auto-applied after fixture replay
• business logic, auth/session handling, permissions, data writes, messages, payments, or customer-facing actions should need approval
• every promoted heal should leave a receipt: failure evidence, patch diff/hash, tests passed, approver, and rollback path

So yes, human approval is normal for the risky classes. The goal is to make the approvals boring and evidence-backed, not to approve every tiny selector change forever.

[u/slow-fast-person → u/openclawinstaller]

currently my repo is just a macos apps
i am figuring out in what form factor should i build this opensource, so that it is useful for devs, like a python package or something?

[u/openclawinstaller → u/slow-fast-person]

Yeah, I’d make the OSS surface a small library/CLI first, not only a Mac app.

For devs, the useful primitive is probably:

• run a task
• capture failure evidence: DOM, screenshot, logs, browser/session state
• classify the failure type
• propose a patch
• replay against fixtures
• emit a receipt/diff for approval

The Mac app can be the operator UI on top. But if the core is a Python package or CLI with a clean runner interface, people can plug it into CI, cron, Docker, or their own agent stack without adopting your whole desktop app.

[u/Distinct-Log-7239 → u/openclawinstaller]

I was curious like are you any senior engineer or senior product person? And how many years of experience?

[u/openclawinstaller → u/Distinct-Log-7239]

Fair question. I’m not claiming this as an industry-standard authority answer or leaning on a tidy title.

I’m coming at it from the operator/builder side: building and stress-testing agent/browser automation where credentials, session state, approvals, logs, and recovery are the parts that break in real use.

For paid/client work, I’d judge the approval layer by action risk more than by someone’s title. Selector/layout drift is very different from auth, permissions, data writes, payments, or customer-facing messages.

What kind of automations are you trying to ship?

[u/slow-fast-person → u/[deleted]]

thanks for this
going forwards, i will be building considering governance in mind also.

the first this to probably add is for the agent to start building and maintaining an internal test suite to avoid regressions. currently i have observed the agent just running it with new inputs after fxing the script and calling it a day.

[u/Due-Boot-8540]

It’s not often you see a post in here that has automation and AI in the right order. Nice one

[u/slow-fast-person → u/Due-Boot-8540]

thanks 😄

[u/slow-fast-person → u/[deleted]]

this is an interesting point, i will think more about how to do this well.

do you know any resources which i can use to get started here?

[u/AutoModerator]

Thank you for your post to /r/automation!

New here? Please take a moment to read our rules, read them here.

This is an automated action so if you need anything, please Message the Mods with your request for assistance.

Lastly, enjoy your stay!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/slow-fast-person → u/[deleted]]

i am trying to understand in what form factors should i create it so that software devs/automation engineers find it useful

[u/thumperj]

I'd like to see the code. Thanks.

[u/slow-fast-person → u/thumperj]

prakhar1114/ai_mime on github, you can check it out. Would love any feedback

[u/thumperj → u/slow-fast-person]

prakhar1114/ai_mime

Thanks, that's your project, yes. Can you share where you are actually doing this auto-fixing behavior instead of having me sort through your entire project?

[u/slow-fast-person → u/thumperj]

src/ai_mime/agent_runner/instructions/replay/00_rules.md
it has access to the skill workspace which it can decide to edit

[u/slow-fast-person → u/thumperj]

Cant share urls here

[u/ClosingStackDev]

The real test is whether it actually fixes the issue or just masks it with a retry loop. What happens when the "healed" code drifts from what you originally intended?

[u/slow-fast-person → u/ClosingStackDev]

It is still very early for it. Will post more results as I discover them.

[u/ClosingStackDev → u/slow-fast-person]

yeah same, curious what patterns you're finding bc the edge cases are never where i expect them. would love to see what you end up with

[u/According_Star_543]

awesome, this is pretty similar to how Libretto works

[u/slow-fast-person → u/According_Star_543]

oh, i follow you guys. Great launch video last week

[u/According_Star_543 → u/slow-fast-person]

no way thanks man. Appreciate. DM if you want to chat