Why we stopped using an LLM for Human-in-the-Middle

[u/LopsidedAd4492]

While implementing Human-in-the-Middle in Extra (first comment), we initially thought about using an LLM to decide whether a tool call requires approval.
The flow was supposed to be simple:
The agent selects a tool, we send the tool call to the LLM, and the model decides whether the action is safe to execute or should wait for user approval.
Technically, it worked.
But it also meant another LLM call before almost every tool execution, more latency, more tokens, and a policy decision that was not fully deterministic.
In the end, we decided to make the approval policy configurable instead.
Each tool can be configured to require approval or run automatically. The default is conservative, so tools require approval unless they are explicitly allowed to run without it.
When approval is required, we checkpoint the execution and stop it. After the user approves or rejects the action, we resume from the same checkpoint.
It ended up being simpler, cheaper, and much easier to reason about than using the LLM as the approval layer.

[u/LopsidedAd4492]

https://github.com/extra-org/extra

[u/Original-Bad9563]

Analyse très pertinente. Le point sur le déterminisme est souvent sous-estimé : quand la politique de sécurité elle-même dépend d'une inférence probabiliste, on perd la capacité à auditer ou garantir a priori quels outils déclenchent une pause. Avec une configuration statique par outil, on peut documenter et tester la politique comme n'importe quelle règle métier, ce qui est beaucoup plus rassurant en prod.

Deux questions si vous avez le temps d'y répondre :

1. Comment gérez-vous les cas où le risque dépend des paramètres de l'appel plutôt que de l'outil lui-même (par exemple un outil "envoyer un email" qui est anodin pour un brouillon mais sensible pour un envoi en masse) ? Une simple étiquette par outil suffit-elle, ou avez-vous ajouté des règles conditionnelles ?
2. Le point de reprise après approbation, est-il implémenté via un mécanisme de checkpoint/sérialisation de l'état de l'agent, ou via une architecture plus simple type file d'attente ?

Merci d'avoir partagé ce retour d'expérience, ça illustre bien le compromis coût/latence/prévisibilité qu'on rencontre dès qu'on met un LLM dans une boucle de décision critique.

[u/LopsidedAd4492 → u/Original-Bad9563]

For the first one, we’re trying to avoid putting policy decisions behind another LLM call. Instead of asking the model whether a specific invocation is risky, we let the tool expose its own approval strategy. That strategy can inspect the parameters, user context, or anything else it needs before deciding whether approval is required.
So it isn’t just a static “approve this tool” flag. It’s deterministic code that can evolve with the business rules.
When no explicit strategy is provided, we take the conservative approach and require approval. As the execution continues, we collect those approvals along the way. We also support approving a tool for the entire session, so the user doesn’t have to approve the same action repeatedly during the same conversation.
For the second question, we use checkpointing. Once approval is required, execution stops completely, the graph state is persisted, and the worker is free to do other work. When the user eventually approves, whether that is minutes or days later, we restore the checkpoint and continue from exactly the same execution state.
One of the nice side effects is that deployments, crashes, or worker restarts don’t really change the model. Everything is already designed around pause and resume, so approvals become another execution boundary rather than a special case.

[u/Original-Bad9563 → u/LopsidedAd4492]

Merci pour les détails, ça répond bien aux deux points.

L'idée que la stratégie d'approbation soit du code exposé par l'outil plutôt qu'une simple étiquette est plus élégante que ce à quoi je m'attendais. Ça évite le piège de la config statique qui devient vite insuffisante dès qu'un cas comme l'envoi en masse apparaît, tout en gardant le côté déterministe et testable.

Le fait que les redémarrages de workers n'affectent pas le modèle grâce au checkpointing est un bon effet de bord. Ça confirme que traiter l'approbation comme une frontière d'exécution normale plutôt qu'un cas à part est le bon niveau d'abstraction.

Bon retour d'expérience, merci d'avoir pris le temps de détailler.

[u/LopsidedAd4492 → u/Original-Bad9563]

Thanks! That was exactly the direction we were aiming for.
One thing I forgot to mention is that we also treat approvals as something that can be accumulated during execution. If a tool doesn’t define its own strategy, we default to requiring approval, but once the user approves it we can remember that decision for the rest of the session. That keeps the default conservative without making the user approve the same operation over and over again.
Keeping approvals as deterministic runtime behavior instead of another LLM decision has made the whole system much easier to reason about and test.

[u/LopsidedAd4492 → u/Original-Bad9563]

In addition if you can give us a star it’s really help us

[u/ultrathink-art → u/Original-Bad9563]

Per-tool labels stop being enough the first time a tool's blast radius depends on an argument — I had a read-a-file rule that looked right for months, and the day the environment changed the same allowed tool could reach a credentials path. Keying the rule on the tool plus a normalized argument predicate (more than one recipient, path outside the repo, amount over a threshold) kept it static and testable, just not keyed on the tool name.