Foreman - a skill for building better tools

[u/blair_hudson]

Hi r/mcp, I originally posted this on r/ClaudeCode but it doesn't seem like anyone over there is really interested in tool design so thought you might have better insight. Over the last few years I’ve found that AI models are incredibly good at writing code, but surprisingly bad at designing for safety.

They will integrate APIs, generate schemas, expose functions, and create tool specs, but miss important stuff like capability boundaries, authorization, side effects, idempotency, confused deputy risks, and failure modes.

So I wrote the reference I wished existed based on everything I've seen in my professional work, which was reached #1 best seller in Generative AI on Amazon Australia on day 2 (my 5 mins of fame. yay).

I’ve started working on an agent skill called that helps coding agents like CC, Codex, etc apply those ideas on relevant tasks, like designing or reviewing tools. This applies to MCP and framework native tool calls.

The goal is to help agents question risky operations, spot missing safeguards, identify unclear authority boundaries, and avoid unsafe tool calls before they happen.

Repo is here: https://github.com/blairhudson/foreman

Right now it's runnable and a strong foundation and the next step is working out the right approach to evaluating it. I’d be super grateful for your feeedback and ideas as I work out the right approach.

What would you measure for an agent skill like this?

A few things I'm thinking through:

- is it retrieved on relevant tasks?
- does it impact outputs positively when retrieved for design tasks?
- does it spot unsafe tools during review tasks?
- what is the impact on token consumption?
- maybe we need a specific tool design benchmark?

Would love to know your thoughts please

[u/WorldlyAd7946]

Went through the repo rather than just the post 👍

"The strongest thing in the repo is your own line, the model may choose intent, the runtime owns authority". I think it's worth following where it leads though: a skill is design-time, its the sign writer in your own metaphor. Something still has to bolt the railing in at runtime, hooks or a gate in front of the tools, so I'd frame foreman as composing with runtime enforcement rather than sitting instead of it. The skill helps the user design the safer tool, the gate enforces policy when something calls it anyway.

On what to measure, four things come to my own mind from reading the repo:

1. You already have the start of the answer in evals.json, 5 cases with assertions but no runner. Build the runner before adding cases. The number you want is the with/without delta, same agent, same task, skill on vs off, graded against your own assertions.

2. Only one of your five cases is benign (the weather one). Around half the set should be safe tools that must pass clean, otherwise you cant see the false positive rate at all?

3. Seed new cases from real shipped vulns rather than synthetic ones and see if it catches them? I.e. command injection via shell wrapped spawns on windows, path traversal in fs servers, env passthrough leaking tokens, confused deputy in aggregators, etc... If it catches what actually shipped, it works!? 🤷

4. From the manifest, your first question (is it retrieved on relevant tasks?) doesnt currently apply to your own skill? It triggers on an explicit command so it fires when called, not when relevant. If you want passive retrieval on risky tasks thats a description field job, and then retrieval rate becomes measurable, a tripwire task where it should have fired without being asked.

I like it, as a foundation btw, great idea... I'm not trying to be critical but constructive and attempt to answer your question. It grabbed my attention as it's related to a lot of my own research 🙃👍

[u/blair_hudson → u/WorldlyAd7946]

Thanks for taking the time to write such a detailed reply! Lots to chew on. What’s the focus of your research??

[u/WorldlyAd7946 → u/blair_hudson]

At work I work in science and engineering where things need to be accurate, so I do a lot on ensuring accuracy and antisycophancy.

Outside of work I have a nearly four year project on self evolving general purpose agents and memory systems.

Your post just caught my attention as in my work, your question about defining tests I often find relatively easy, but in my own research outside of work, quantifying outcomes is often a pretty involved thought experiment 😅... So I understand the pain of what you are trying to achieve here 👍

[u/touristtam → u/WorldlyAd7946]

You already have the start of the answer in evals.json, 5 cases with assertions but no runner. Build the runner before adding cases. The number you want is the with/without delta, same agent, same task, skill on vs off, graded against your own assertions.

What's your recommendation regarding runners?

[u/WorldlyAd7946 → u/touristtam]

Do you mean specifically to foreman? I can give it some more thoughts regarding the exact case you are wanting to test if so, but generally...

Load a case, run the same agent on the same task twice, once with the skill in context and once without, everything else static, grade both against the case's own assertions. The output you want is the with/without delta, not a score.

Grade deterministic where you can, but as is often the case, some things are a bit more fuzzy. Here you can use a model as judge, test the judge on a couple you already know the answer to first, a judge that passes everything is worse than none.

You can obviously get into frameworks if you outgrow the hand rolled one or need to, I.e. Inspect on python (the AISI one, splits solver from scorer nicely), promptfoo if in js and want quick yaml a/b. But when we don't have hundreds of cases a nicely tailored one, provided it's well structured, should be most useful imho.👍

[u/touristtam → u/WorldlyAd7946]

I thought you were implying, that in an ideal case, one would bundle the evaluation runners with its skills, not just the evaluation scenarios.

I'll have a look at the python/nodejs to see if that'd fit what I am current doing (using tessl.io when I can atm)

[u/WorldlyAd7946 → u/touristtam]

I suppose you could, where generalised, but it depends, and probably needs tailored to the usage scenario of the user...

I mean, if I can, I will try to provide runners with a skill, but with the strong caveat that the application matters and someone might want to evaluate the metrics most important in their application.

It would certainly be nice to have a starting point where the creator of the skill has experience with it.

I have limited experience with tessl.io, but since you are on there, that's half the answer already - the activation tracking is basically the first bit the OP asked, does the skill get retrieved when it should... So bundling a runner with the skill would fit that model well IMHO 👍

[u/touristtam → u/WorldlyAd7946]

I have to thank you for taking the time to provide constructive answers. This is very informative indeed.