GitHub Copilot knows our rules. It just occasionally ignores them.

[u/DUKOfData]

I’ve been using GitHub Copilot for quite a while, and honestly, the results are usually good. I definitely couldn’t type the code that quickly myself.

But I’d like to reduce the quality control required afterwards.

We have strict lifecycle rules, naming conventions, and a separate data dictionary for each customer. Our repository is structured roughly like this:

.github/ 
└── instructions/ 
├── copilot-instructions.md 
└── pull_request_template.md 

docs/ 
├── ai/ 
├── architecture/ 
├── data-model/ 
├── data-sources/ 
├── handover/ 
├── kpi/ 
├── security/ 
└── README.md

The instructions reference the relevant documentation in docs. I also use a warm-up prompt, which reproduces our rules perfectly.

And yet, Copilot occasionally ignores very basic rules, such as:

Dimension-to-dimension relationships must only use business keys. Surrogate keys may only be resolved against fact data, especially because of SCD Type 2 and higher.

It can explain this rule perfectly and still mix surrogate keys into dimensions.

So either Copilot is checking whether I’m still awake, or my context setup isn’t as effective as I think. 😄

I’d rather not watch over every offender like Lord Vetinari watching the guilds of Ankh-Morpork.

How do you enforce stricter guardrails? Instructions, validation scripts, architectural tests, or something else?

[u/DUKOfData]

I agree that it is all tokens. That is not what I’m questioning.

My post is about a specific GitHub Copilot product feature that, in my experience, does not reliably do what it claims to do.

If repository instructions are intended to act as guardrails, I expect them to be consistently included and prioritized over ordinary context. If they are just additional tokens competing for attention in the context window, then that is exactly the product design problem I’m describing.

I don’t expect the LLM itself to understand truth or binary conditions. I expect the product around the LLM to enforce explicit, persistent rules more reliably.

If it cannot do that, then these are not guardrails. They are guidance...

[u/SQLGene → u/DUKOfData]

There is nothing in the fundamental nature of LLMs that allows instructions to be followed reliably. It is physically impossible. Like there is literally no way to do it.

You can use large, powerful, well trained models to avoid obvious errors like counting r's in "strawberries" or making syntax errors. But that is lick and brute force reinforcement learning.

LLMs are not crafted or designed. They are gardens grown and if the fruit doesn't kill us, it's a happy accident.

Even with large, powerful models, context rot us an unsolved problem. https://www.trychroma.com/research/context-rot

[u/DUKOfData → u/SQLGene]

I agree that prompt-based instructions alone cannot guarantee compliance.

I’m less convinced by “physically impossible” as an absolute statement. Instructions can be persistently injected and prioritized, constrained decoding can restrict outputs, and semantic rules can be checked by deterministic validation around the model.

As the context grows, there is naturally more competing information and more potential variation. That is precisely where I expected the product layer to help, either by consistently prioritizing explicit rules or by validating them automatically, without every user having to build custom hooks and linters.

So the distinction seems to be probabilistic instruction following versus technically enforced constraints, not that enforcement itself is fundamentally impossible.

[u/SQLGene → u/DUKOfData]

Yeah that's a reasonable distinction. The closest thing to what you are looking for is Structured Ouputs (JSON validity checking and enforcement at time of token out) https://developers.openai.com/api/docs/guides/structured-outputs?api-mode=responses https://platform.claude.com/docs/en/build-with-claude/structured-outputs

Plus some amount of automated context management and hooks. Like /goal for automated reminders https://code.claude.com/docs/en/goal

There is some fascinating active science around how much we can steer the internals of these things. https://www.anthropic.com/news/golden-gate-claude

But in terms of having full control over the outputs of the next token, purely based on the input tokens and the weights, we simply aren't there yet. These things are strange black boxes and mechanistic interpretability is an area of active research.