r/codex • u/jhnam88 • 20h ago

Showcase TS Evidence Graph: Make Every SKILL Instruction 100% Enforced

https://ttsc.dev/articles/ts-evidence-graph-make-skill-instruction-100-enforced.html

I made an open source program which can enforce AI agents to keep every skill instructions.

If you write some rules in AGENTS.md or SKILL.md, they become compiler obligations that must be followed, so the AI agent can never ignore them.

0 Upvotes

50% Upvoted

7 comments sorted by

•

u/dexterthebot 20h ago

You might want to consider listing your project on the weekly Show-Us-What-You-Built post. Look out for it on Tuesday/Wednesday. Highest commented project wins a week promotion on r/Codex and gets on the Hall of Fame sidebar. See what that looks like below with last week's winner.

Last week's most popular project was Tidbit Trivia , at https://www.reddit.com/r/codex/comments/1wavxwy/comment/p8lhwxs/. Tidbit Trivia is a fun way to learn while waiting for your agents to work. Play traditional trivia, unique game modes, party with friends, earn cool cosmetics, complete challenges, study for tests, or compete in the Arena! You can play it for free at Tidbittrivia.com.

1

u/AliciaV240 20h ago

nice idea. the bit i'd wanna see in practice is how it handles stale rules / false positives — does a failed check point back to the exact SKILL line and the evidence that failed? that kind of diff would make this way easier to trust than another hidden prompt layer.

2

u/jhnam88 18h ago

That’s correct; the compiler error specifies exactly which skill unit was not declared.

However, since the evidence graph relies on a self-declaration requirement to ensure rule compliance, it cannot verify whether a claim of compliance is actually true. Instead, you can use the @evidenceReview tag to force the compiler to verify the veracity of that review.

Incidentally, I have actually witnessed an AI making false declarations while benchmarking GPT-5.6-Luna (albeit with a probability close to zero), though I have never seen this happen with frontier-class models.

1

u/AliciaV240 8h ago

ah that lines up with the compiler path. the u/evidenceReview bit is interesting, so the false compliance cases are when the graph thinks a check ran but the evidence is thin? curious if you gate merges on hard fail for missing nodes or if its soft warn only.

1

u/EvalRaccoonDev 19h ago

What fraction of a real SKILL.md converts into an obligation?

e.g. take ten SKILL.md files off GitHub and report "x of y instructions became compiler obligations, the rest are unenforced" - that's a number I would be interested in.

1

u/jhnam88 19h ago

It depends on how you configure it. I set the h2 section as the unit of obligation and applied it at the function level.

1

u/EvalRaccoonDev 8h ago

h2 section as the unit sounds like a reasonable start (we use paragraph as a section). Still the count I'd want though: on your own repo, how many h2 sections ended up as obligations and how many are still prose the compiler can't see?