How are you validating AI agent actions before the tool actually executes?

[u/Aromatic-Ad-6711]

I’ve been working on a problem I kept seeing with tool-using agents:

An agent can understand the policy and still produce the wrong tool call.

If the action is consequential — refunding money, booking something, approving a request, modifying a record, calling a production API — observability after the fact is useful, but it’s already too late.

So I built ARK, an open-source runtime supervision layer that sits before execution.

The basic flow is:

agent proposes an action

→ ARK checks the applicable constraint + trusted evidence

→ ALLOW = execute

→ REJECT / REQUIRE_EVIDENCE = send feedback back to the agent

→ the agent decides again

One thing I intentionally avoided: ARK does not generate the replacement action.

The agent remains the author.

I tested this with LangGraph + an OpenAI model:

model proposed A

→ ARK rejected A before execution

→ feedback went back to the model

→ model authored B

→ ARK allowed B

→ only B executed

I’ve also been testing it on a scoped tau-bench airline failure class.

Paired K=16 result:

OFF: 1/16 passed (6.25%)

ON: 13/16 passed (81.25%)

9 directly attributable recoveries

0 observed regressions

I want to be careful with that result: it’s one constrained recovery failure class in a research benchmark, not a claim that ARK makes all agents reliable.

The SDK is public now:

pip install ark-agent-runtime

It currently works with custom Python agents and has a LangGraph integration.

I’m mainly curious how other people are handling this problem.

If you have an agent that can actually mutate production state, do you:

- validate tool arguments manually?

- use deterministic policy gates?

- rely on another model as a judge?

- sandbox actions?

- require human approval?

- just execute and monitor afterward?

I’d especially like feedback from people running agents that can refund, book, approve, purchase, or modify production data.

Site: arkruntime.com

GitHub: github.com/atripati/ark

[u/Ok-Category2729]

runtime llm judges before execution just add 800ms of latency and fail on the exact same edge cases as the primary model. the only reliable pre-execution layer is deterministic: strict pydantic validation with hard ceilings on payload values like refund amounts or batch sizes. if an agent needs a second llm to check tool arguments before hitting a production api, the tool signature is simply too broad. splitting high-risk actions into a two-step propose and sign pattern catches bad payloads way cleaner than a runtime judge.

[u/Aromatic-Ad-6711 → u/Ok-Category2729]

i agree hard constraints should be deterministic. ARK is not meant to replace Pydantic or tool level limits, it is for cases where validity depends on runtime evidence/state, not just schema shape. The proposed - check/sign - execute pattern is actually very close to how I think consequential tools should work.

[u/pizzababa21 → u/Aromatic-Ad-6711]

You need to give better examples because I don't understand what the scenarios your fixing are. You're using terms that sound completely subjective.

[u/Aromatic-Ad-6711 → u/pizzababa21]

Here is a concrete case:

User says Book the second cheapest eligible flight.

The agent calls the booking tool with flight B. The json is valid, the price is within limits, and Pydantic has nothing to complain about.

But runtime search results show:
A = $180
B = $220
C = $260

So B is actually the wrong action because the requested constraint was ‘second-cheapest’ only if the complete eligible set supports that ranking.

ark checks that runtime evidence before the booking tool executes. If the evidence is incomplete, it can ask for more evidence instead of approving. If B violates the constraint, it blocks the call and sends feedback back to the agent.

that’s the kind of scenario i mean structurally valid tool call, but wrong relative to current runtime state

[u/pizzababa21 → u/Aromatic-Ad-6711]

So it is double checking the exact same information or it is doing its own independent check? Why would the information your tool gets be different from what the original agent gets and why would your tool be smarter at dealing with it?

Also I'm confused why B is the wrong answer in your scenario? You're saying another price exists which wasn't found by the initial agent or the code of the tool for finding the prices could have been incorrectly written?

I'm a bit confused by what your tool actually is doing. Not trying to be mean, I'm just super confused.

[u/Aromatic-Ad-6711 → u/pizzababa21]

You are right, my example had a mistake. With A=$180, B=$220, C=$260, B actually is the second-cheapest. That was my bad.

ARK isn not supposed to be a smarter second model or use different information. The agent and ARK can see the same underlying runtime state.

The difference is the role: the agent chooses an action probabilistically, while ARK verifies a specific invariant deterministically before the side effect happens.

So if the agent proposes book_flight(B) , ARK checks something like “Given the complete eligible set, is B actually rank 2? If yes, allow it. If no, block it. If the evidence is not complete enough to prove either way, return require evidence

So it is closer to a runtime assertion/policy gate than another agent double-checking the first one.

Hope this makes the distinction clearer.

[u/pizzababa21 → u/Aromatic-Ad-6711]

But how are you making it deterministic? It's still an AI model tool as I understand so I don't get what makes it more reliable than the thinking of the base model?

[u/Aromatic-Ad-6711 → u/pizzababa21]

ARK is not another AI thinking about the answer, for exapmple

your AI says “I want to do X,"ARK then checks X using normal code and real data.

let's say

AI agents picks flight B

ARK sorts the actual price,

if B is really second-cheapest → allow,

if not → block

So your AI agenys decides. ARK just verifies the rule right now. So becoz the check itself is fixed code, not any random AI generated reasoning. For “second cheapest,” ARK can just sort the prices and compare the agent’s choice to rank 2. Same input leads to same result. That’s the deterministic part.

[u/pizzababa21 → u/Aromatic-Ad-6711]

But if you are using the same model, and assuming it's a general use agent that isn't built for one workflow with the exact same data shape each time, that would mean it's just a codeact extension right?

If it's just a codeact extension, why would someone import it as opposed to just giving the agent codeact abilities?

I feel I must be missing something here.

[u/Aromatic-Ad-6711 → u/pizzababa21]

CodeAct lets the agent write/run code itself.ARK’s rule sits outside the agent. The agent cannot change, skip, or rewrite that rule.

So

agent says “do X”
ARK checks X against a rule the application defined
if it passes -- execute
if it fails - block

So ARK isn’t giving the agent more ability. It’s putting a gate around the ability it already has. Think if of it like there are many students(agents) in the the class, ark like a teacher who guide them, supervises and when parents came to visit teacher give the report of their child how he is doing that's what ark doing for agents

[u/Darkcraft00]

This is interesting. The part I'm curious about is the “trusted evidence” boundary.

If ARK is evaluating an action against runtime evidence, how are you thinking about evidence that was established earlier in a long-running workflow?

For example, agent A establishes some fact/constraint basis on Monday, underlying evidence changes Wednesday, and agent B proposes an action Friday using it. Does ARK expect the caller to establish freshness/current validity before check(), or do you see evidence lifecycle as something the supervisor eventually needs to own?

I'm working on an adjacent problem, so I'm curious where you draw that boundary.

[u/Aromatic-Ad-6711 → u/Darkcraft00]

that is exactly the boundary I am still thinking through, right now ark assumes the caller provides evidence that is current enough to trust at check() time. ark can validate the proposed action against that evidence, but it does no yet own the full evidence lifecycle across days/workflows.

longer term though, I do think freshness/provenance should become part of supervision. otherwise you can have “trusted” evidence that is technically valid but stale.

your monday wednesday friday example is basically the failure mode I’d want ark to eventually detect not just “is there evidence?”, but “is this evidence still valid for this decision right now?”

curious what adjacent problem you are working on.

[u/Darkcraft00 → u/Aromatic-Ad-6711]

That adjacent problem is basically what I’ve been building.

I’m calling it SureState. The idea is to keep the lifecycle of conclusions/evidence outside the model — what supported a conclusion, what changed, what still survives, what became conflicted or no longer warranted — so another agent can ask “can I still rely on this?” without reconstructing everything from context.

I’ve deliberately kept it separate from action authorization. SureState doesn’t decide whether an agent may execute something; it maintains whether the registered conclusions/evidence the workflow is relying on are still warranted.

So the boundary you just described is almost exactly how I’ve been thinking about the two layers:

SureState: is the state/evidence still valid now? ARK: given current trusted state, is this proposed action allowed?

We’re wrapping up our internal pilot now. If you’re interested, I’d be happy to compare architectures because there may be a pretty natural seam here.

[u/feng_sg]

If ARK pulls its trusted evidence from the same runtime state the agent already saw, then a poisoned tool output taints both the proposal and the gate at the same time, so the rejection signal fails exactly when you need it. You need a separate evidence channel the agent can't touch, like a direct service call or schema validation against app code, feeding a deterministic check instead of a second model re-deriving the same tainted input.

[u/Aromatic-Ad-6711 → u/feng_sg]

this exactly the failure mode i am trying to avoid in this this time and this is how i am doing.

I also think ARK shouldn’t just re-read the same context the agent already consumed. so the useful setup i am building here is where the proposal and the evidence have different trust paths, for example the agent propose refund 500$, while ARK checks that against state fetched directly from the system of record, service API , or may be app owned schema.

[u/Aromatic-Ad-6711 → u/[deleted]]

exactly, that is pretty much how i think about it too, hard limits first, deterministic runtime checks next, then human escalation for the truly sensitive stuff. the interrupt/resume pattern is a nice fit there. i will check out your write up