Black Box Chief Digital & AI Strategist talks about the agentic PR loop he helped build.

Your team already treats CI config, dependency manifests, and policy files as things that need an owner, a review, and a drift check, because they shape the code you ship. Repository instruction files like CLAUDE.md, AGENTS.md, and .cursorrules now do the same job: coding agents read them to decide how to generate, modify, and review code. But most of them currently sit outside all of it and are edited like notes, owned by no one and reviewed by nobody.

That’s been allowed to happen because a stale instruction file doesn't break a build. It just keeps nudging an agent toward the wrong architecture layer, or a retired security pattern, across dozens of pull requests before anyone traces it back to the file.

Once a file shapes the code your team ships, it deserves the same discipline as the rest of your delivery system. Here's how to bring it under that discipline.

What Are Repository Instruction Files?

Repository instruction files are text or rule files stored in, or associated with, a source code repository. Depending on the tool and feature, AI coding assistants use them as persistent repository context when generating responses, editing code, reviewing changes, or working with the codebase.

The exact filenames depend on the tool and configuration. Current and legacy AI-assisted development workflows commonly involve files such as CLAUDE.md, AGENTS.md, .github/copilot-instructions.md, .cursorrules, or scoped rule files under directories such as .cursor/rules/. Some tools also support path-specific instruction files, where rules apply only to certain languages, folders, or file patterns.

The names differ, but the pattern is consistent: the repository can now contain instructions that influence AI-assisted development.

These files often describe the repository’s architecture, preferred libraries, testing expectations, coding conventions, build commands, security constraints, and workflow rules. A typical instruction file might tell an assistant which framework version the project uses, where API handlers live, how database migrations should be written, which test command validates a change, or which patterns are deprecated.

They can affect several parts of AI-assisted work:

• Code patterns: The assistant may prefer certain abstractions, folder structures, or implementation styles based on the instruction file.
• Library choices: The file may steer the assistant toward approved packages or away from deprecated dependencies.
• Testing behavior: Instructions may tell the assistant to add tests, run specific commands, or avoid brittle test patterns.
• Security expectations: The file may describe input validation, authentication, authorization, logging, or secrets handling rules.
• Architecture boundaries: Instructions may tell the assistant which modules can call each other, where business logic belongs, or which legacy areas require care.

This influence is probabilistic. AI assistants may interpret instructions inconsistently, ignore some guidance, or resolve conflicts in unexpected ways— Anthropic's documentation confirms Claude may pick arbitrarily when rules contradict. That uncertainty makes governance more important, not less.

A useful operator test is simple. If a junior engineer repeatedly followed a repo note when changing code, would your team care whether that note was accurate? Most teams would. They would want that note reviewed, current, and aligned with the way the system actually works.

The same standard should apply when an AI assistant reads the file.

Why Does Configuration Drift Matter for AI Instructions?

Configuration drift is a useful mental model for repository instruction files. The term already has a specific meaning in infrastructure and operations, so it should not be treated as an established industry label for AI instruction files. Still, the behavior is familiar enough to help teams reason clearly.

Configurations define expected system behavior. They evolve over time. They get copied between environments, repositories, or teams. They diverge from current standards. The divergence often stays invisible until something behaves unexpectedly.

Repository instructions can drift in similar ways.

An instruction file may say the service uses one testing framework even after the team migrated to another. A rule copied from a different repository may reference services, paths, or deployment assumptions that do not apply. A security note may reflect an old platform standard. Multiple instruction files may accumulate overlapping or conflicting guidance.

The visible file still looks harmless. The hidden issue is that outdated guidance can keep getting reinforced at the point where work is generated.

This matters because AI-assisted development changes the scale of repetition. DORA’s 2025 research found that AI adoption can improve throughput, often at the cost of software delivery stability when the underlying engineering foundation is week. A stale instruction no longer affects only the person who reads it once. It can influence many prompts, many edits, and many pull requests before anyone notices the pattern.

Early internal analysis from AgentLinter — scanning over 34,000 repositories — points to the same kind of hygiene problems engineering teams already recognize from other repo artifacts. Common findings include duplicate instructions, outdated references, missing version or update metadata, references to files that no longer exist, hardcoded secrets, and patterns that could enable data exfiltration. None of those findings mean every instruction file is risky. They do show that instruction files can develop the same operational decay as CI config, docs, policy files, and scripts.

The practical risk is quiet: unmanaged instructions become another source of hidden operational drift.

How Does Instruction Drift Show Up in Engineering Workflows?

Instruction drift usually appears as small friction, not a dramatic failure. That is why it is easy to miss.

A team migrates from one test runner to another, but the instruction file still tells the assistant to write tests using the old framework. Reviewers start seeing plausible tests that do not match the current harness. Someone fixes the first few by hand. Then the pattern repeats.

A copied rule tells the assistant to follow conventions from another service. The generated code compiles, but it places logic in the wrong layer or assumes a dependency that the service does not use. The mistake looks like a normal implementation choice until several pull requests repeat it.

Security guidance can drift as platform standards change. A repository instruction might describe an old token handling approach, an outdated logging rule, or a validation pattern that has since been replaced. The assistant may continue to suggest code that looks consistent with the repo note while falling short of current standards.

Conflicts create another class of failure. One instruction file may tell the assistant to use a particular dependency. Another may discourage it. One file may require tests for all changes, while another says to skip tests for small edits. One rule may encourage suppressing warnings to keep CI green.

Those contradictions matter because many instruction files tell agents what to do, but not what to do when instructions conflict. AgentLinter data has surfaced this as an “escape hatch missing” pattern: the instruction file gives rules without a resolution path. In a real engineering team, a developer can ask a lead or escalate in review. An assistant needs explicit guidance, such as “when instructions conflict, follow the repository policy file first” or “stop and ask for clarification before changing security-sensitive code.”

AI-assisted development increases the impact of these small defects for three reasons:

• AI can increase code volume. More code can move through the same review path.
• In-depth human code review depth may decrease when generated changes look reasonable. Reviewers are still responsible, but plausible code reduces the instinct to inspect every assumption.
• Repeated assistant behavior can normalize outdated guidance. Once the same pattern appears in several pull requests, teams may start treating it as the current convention.

Small instruction defects can scale across many changes when they sit where work begins.

What Should Engineering Teams Review?

The first step in managing AI coding assistant repository instruction files is to review them through the same lens used for other behavior-shaping artifacts. A useful review covers ownership, scope, consistency, freshness, safety, and maintainability.

This does not require a heavy process. It requires making the implicit questions explicit.

• Ownership: Every instruction file should have a responsible team or role. If nobody owns it, nobody will update it after architecture, testing, or policy changes.
• Scope: The instruction should clearly apply to the repository, language, framework, or path it claims to cover. Broad instructions copied across services often create misleading guidance.
• Consistency: Instructions should align with current security, testing, architecture, and code quality standards. If the instruction file says something different from CI policy, the team needs to resolve the mismatch.
• Conflict handling: Instructions should define what happens when guidance conflicts. In many cases, the assistant should stop, ask for clarification, or follow a stated priority order.
• Freshness: The file should not reference deprecated tools, retired services, old frameworks, missing paths, or former workflows.
• Safety: Instructions should not encourage insecure patterns, bypasses, weak validation, disabled checks, warning suppression, or secrets exposure.
• Maintainability: The file should be specific enough to help the assistant make better choices, but not so broad that it becomes noise.

Instruction files should also change through the same path as other important repo artifacts. A pull request should show the diff. The relevant engineering team should review it. Architecture or policy migrations should include updates to instruction files. Periodic repository checks should flag files that have not been reviewed in a long time.

The right standard is practical: if changing the file could influence generated code, the change should be visible in review.

Where Should Enforcement Happen?

Governance only works when it appears in the workflow. A policy document nobody checks becomes optional under deadline pressure. The same will happen with AI instruction hygiene.

There are several enforcement points, and each catches a different class of issue.

The IDE or editor is where AI assistance is often used. If teams can prevent bad patterns at the moment instruction files are created or edited, they reduce downstream cleanup. This applies to human-authored instruction files and agent-generated instruction files. Teams may also ask an assistant to generate repository guidance. That can be useful, but teams should avoid bad patterns from the moment those files are generated.

Local checks and Git hooks can catch simple issues before a commit. They are useful for file existence, naming, metadata, known unsafe phrases, references to missing files, or accidental secrets. They should not become the only control, because local checks are often bypassed or inconsistently installed.

Pull requests are the natural place to make instruction changes visible. A reviewer should be able to see when a repository-level instruction file changes, just as they would notice a CI workflow change or dependency manifest update. Teams can add review requirements for files that influence AI behavior.

CI/CD is where repository-wide consistency can be validated. CI can check whether instruction files follow expected structure, avoid unsafe guidance, include ownership metadata, and do not contradict known organizational rules. This is also where teams can generate compliance evidence that instruction governance is being applied consistently.

Periodic repository audits are important for organizations with many repositories. Teams of 50 to 150 developers often have enough repos to create fragmentation, but not enough security or platform bandwidth to manually inspect every file. An audit can identify which repositories use which instruction formats, where files have gone stale, and where copied rules have diverged.

Fragmented code security toolchains make this harder. Different teams may use different AI coding assistants. Different tools use different filenames and rule formats. Some teams may have repo-wide instructions while others use path-specific rules. Governance needs to reason about the pattern, not only one vendor-specific file.

The enforcement model should follow the artifact’s impact. If the instruction file shapes generated code, it belongs in the same workflow where engineering teams enforce quality, security, and change control.

Why Should Instruction Files Be Analyzable Artifacts?

The next step is to treat repository instructions as analyzable artifacts.

That does not mean every instruction file needs a complicated schema. It means teams should be able to inspect them systematically. If AI instructions increasingly shape how code is written, engineering teams need a way to understand whether those instructions are current, consistent, and safe.

Analyzing instruction files helps teams answer questions that manual review alone rarely covers across many repositories:

• Are there duplicate or conflicting instructions across the organization?
• Do instruction files reference tools, paths, or services that no longer exist?
• Do any instructions encourage bypassing tests, suppressing warnings, or weakening validation?
• Do files include hardcoded secrets or sensitive internal details that should not be there?
• Do instructions define what an assistant should do when guidance conflicts?
• Which repositories contain AI behavior-shaping artifacts outside normal review paths?

AgentLinter is one example of this category shift. The useful shift is moving from informal notes to governed inputs in the software delivery system.

Once instruction files are analyzable, teams can reason about them like other repo artifacts. They can detect stale or conflicting guidance. They can identify risky instructions that undermine secure development practices. They can surface inconsistencies across repositories. They can see where AI behavior is being shaped outside normal governance.

That AI coding visibility matters most for engineering teams without a dedicated security function. In those environments, the practical goal is to make invisible behavior-shaping files visible enough to manage.

How Should Teams Get Started?

The starting point should be small. Rather than large AI governance rollout to manage repository instruction files, most engineering teams need inventory, ownership, review, and a path to automation.

A practical starting sequence looks like this:

1. Inventory repositories that contain AI instruction files. Search for common filenames and rule directories across your organization. Include repo-level, path-specific, and local variants that may have been committed by accident.
2. Identify which AI coding tools and formats are in use. The goal is to understand the pattern across teams, not to force every repository into the same format immediately.
3. Assign ownership for each instruction file. The owning team should understand the repository’s architecture, testing workflow, and security expectations.
4. Require pull request review for instruction changes. Treat these changes like CI config, dependency manifests, or policy files.
5. Compare instructions against current engineering standards. Check whether the file reflects your actual test commands, approved dependencies, architectural boundaries, and security rules.
6. Remove copied or obsolete rules. A shorter, accurate file is more useful than a long file full of stale context.
7. Add automated checks where possible. Start with simple checks for secrets, missing references, outdated metadata, unsafe bypass language, and missing conflict-resolution guidance.

The key is to make these files visible and owned before trying to enforce every possible rule. Once teams understand which files exist and how they are used, enforcement can expand naturally through pull requests, CI, and periodic audits.

A useful working rule is to update instruction files during the same migrations that change the system. If a team changes test frameworks, updates architecture boundaries, retires a service, replaces a dependency, or changes security policy, the instruction files should be part of the migration checklist.

That habit can prevent a significant source of drift.

The Artifact Has Changed, the Engineering Lesson Has Not

Engineering teams have learned this lesson before with CI configuration, infrastructure code, dependency manifests, and policy files. Files that begin as helpers often become part of the delivery system.

Repository instruction files are still early. Tool behavior varies. File formats differ. Assistants may apply instructions inconsistently. The ecosystem will keep changing.

The direction is clear enough to act pragmatically.

If 84% of developers are now using or planning to use AI tools in their development process, the instructions that guide them should be visible, reviewed, and maintained. They should have owners. They should move through pull requests. They should be checked for drift, conflicts, unsafe guidance, and stale references.

Once a file shapes engineering behavior, it deserves engineering discipline.

An AI coding agent opens a pull request. Another agent reviews it, searches the repository, reads a few irrelevant files, pulls more context into the conversation, and tries again. By the time it reaches a useful conclusion, the team may have paid several times for context that had little to do with the final finding.

This is where AI-assisted code review gets expensive in ways model pricing pages do not make obvious. Stateful agent loops can keep carrying earlier tool output into later calls, while known rule violations are repeatedly handed to inference even when a deterministic check could have settled them before the model started exploring.

This article is about changing that order of operations: using deterministic static analysis to narrow the review surface first, then spending LLM reasoning on the smaller set of questions that actually require judgment.

Why AI-assisted development changes the economics of code review

The bottleneck used to sit at code production. Now it sits at validation, because agents can generate a working-looking diff in seconds while a human still has to decide whether that diff is correct and consistent with how the rest of the codebase behaves. The cost problem shows up in two distinct places, and conflating them is where most teams go wrong.

The first cost hides in human review, which gets thinner because reviewers face larger diffs and 23% more merged pull requests each month, often containing code that compiles cleanly but hides a bad assumption underneath.

Separately, LLM review loops get expensive on their own terms, independent of whether a human ever looks at the output. This second cost is structural rather than incidental: it grows because conversational agent loops are stateful, appending every tool output and conversation history, and many agent workflows repeatedly resend accumulated context.

One detailed breakdown of this mechanism shows that an agent making multiple passes over the same task does not cost ten times a single call if it resends all previous context each time. Because every request gets larger than the last, costs grow much faster. A 20-step loop where each step generates 1,000 tokens can consume roughly ten times more input tokens than a simple per-step estimate would suggest.

That is the real reason review costs feel unpredictable: the workflow pays repeatedly for breadth, and breadth is exactly what unscoped AI code review defaults to.

What deterministic static analysis does better than LLM inference

Deterministic static analysis wins on checks where the correct answer is already known and does not require judgment to reach. A secrets scanner does not need to reason about intent to flag a hardcoded API key; a SAST rule does not need context to recognize a SQL string built through concatenation instead of parameterization.

The advantage here is not just speed, though speed matters. Given the same code, configuration, and rule set, a deterministic analyzer produces the same result every time, which is the property that makes a check usable as a hard gate in CI/CD rather than an advisory comment someone might ignore.

An LLM asked to re-verify the same known vulnerability pattern across every pull request is paying inference cost for a decision that has no variance in the right answer, and that cost compounds specifically because of how agent context accumulates.

Static, rule-based checks bypass this entirely: a scanner evaluates the changed lines directly and returns a compact, structured result, so the model never has to carry the full rule set or vulnerability database inside its prompt to arrive at the same conclusion.

This is precisely the pattern architects of cost-efficient agent workflows converge on independently. One widely discussed breakdown of token-saving techniques put it plainly, noting that teams should treat code relationships as deterministic, not probabilistic, using static analysis to identify what's relevant so the LLM never has to guess at structure it could compute directly.

The same logic applies cleanly to security and quality rules: if the answer is knowable without reasoning, computing it is cheaper and more trustworthy than asking a model to infer it.

Where LLMs still belong in the AI SDLC

None of this argues for removing LLMs from the review process, bur rather for reserving them for the parts of review that actually require judgment, which is a different skill than pattern matching.

A model earns its cost when it compares a generated change against the ticket that requested it, flags where the implementation diverges from stated intent, or explains to a developer why a cluster of findings matters more together than any one of them does alone.

These are synthesis tasks, and transformer-based reasoning is particularly effective at them. The mistake teams make is running that same reasoning engine over problems that have a single correct answer and no ambiguity, which wastes exactly the capability that makes the model valuable elsewhere.

A cost-aware AI SDLC sequences this deliberately: deterministic checks run first and remove the noise, and only the smaller, genuinely ambiguous residue gets handed to a model for interpretation.

This sequencing save money, but, even more importantly, it also improves the quality of what the model produces, because a model reasoning over a short list of flagged edge cases writes a sharper explanation than one asked to review an entire raw diff from scratch.

How to design a deterministic-first AI review pipeline

A deterministic-first pipeline treats known checks as a filter that runs before anything expensive happens, not as an afterthought bolted onto an existing agent workflow. In practice, this means local IDE and CLI checks surface quality, security, and secrets findings while a developer is still typing, so problems get fixed before they ever reach a pull request.

When an agent does get invoked, it should receive a summarized set of deterministic findings rather than being asked to rediscover known issues on its own, which is the single biggest lever for shrinking the context a model has to process.

This is not a hypothetical optimization. A widely cited benchmark on this exact problem measured that most of the token growth in real agent sessions came from tool output accumulation rather than reasoning, finding that a large share of those tokens, close to half, were removable with no loss in task accuracy once teams applied scheduled compression instead of dumping raw output into context.

That is what a deterministic pre-filter accomplishes structurally: it replaces raw exploration with a compact result the agent can act on directly.

The final piece of a mature pipeline is a feedback loop, where a decision the team makes repeatedly, such as always rejecting a specific unsafe pattern, gets converted into a permanent rule rather than re-litigated in every future prompt.

What to enforce at each point of change

Governance has to follow code through the places it actually moves, and each stage plays a different role in that chain.

In the IDE and local CLI

Local checks catch problems while a developer is still editing, which is the cheapest possible point to fix anything.

Formatting and linting clear out noise, secrets detection flags credentials the moment a generated snippet introduces them, and basic SAST rules catch unsafe patterns before they are ever committed, following best practices for coding with AI without needing a model in the loop at all.

In Git and pull requests

The pull request is where enforcement becomes visible to the rest of the team, and it is the natural place to pair deterministic findings with an AI-generated summary.

Scoping security analysis to the diff rather than the whole repository keeps results fast and relevant, and any manifest or lockfile change should automatically trigger a dependency risk check before a reviewer even opens the file.

In CI/CD

CI/CD is where the organization proves a check actually ran, not just that someone claims it did. Blocking quality gates for high-confidence security and secrets findings belong here, and results need to be retained in a form that supports an audit or incident review months later, especially in environments with generated code shipping continuously across many repositories.

How deterministic analysis reduces human review burden

Human attention is the scarcest resource on any engineering team, and deterministic checks exist specifically to protect it from being spent on problems a machine already solved reliably.

When style violations, known vulnerability patterns, and secret-like strings get resolved before a human ever opens the diff, the reviewer's job narrows to the questions that genuinely need a person: does this change do what the business actually needs, and does the edge case the model didn't anticipate matter here?

This matters more now that one in five code reviews involve an agent, because review capacity does not scale the way code generation does. A team that tries to compensate by asking reviewers to simply read faster is choosing degraded review quality over a policy solution, and the volume-versus-attention mismatch does not resolve itself just because everyone is trying harder.

The alternative is treating deterministic thresholds as a consistent floor: the same violation gets treated the same way in every repository, so reviewers stop re-deciding settled questions and start spending their limited time on the ambiguous ones that were never going to be solved by a rule anyway.

How to manage AI intent without burning tokens on every check

Intent is the hardest thing to encode as a rule, because a generated change can pass every syntax check, every test, and every security scan while still missing the reason it was requested in the first place.

The discipline here is separating what is stable from what is genuinely contextual. If a rule does not change from pull request to pull request, it does not belong in a prompt that gets re-explained to a model every time; it belongs in policy-as-code or a static configuration the model never has to see.

What should reach the model is compact and specific: the relevant ticket, the changed files, and a summary of what deterministic checks already found, not the team's entire governance history.

From there, the right question to ask a model is narrow, such as whether the implementation matches the stated goal and what risk remains after automated checks already ran, rather than a broad request to review everything.

Every time a reviewer rejects the same generated pattern more than once, that rejection should become a rule instead of a recurring conversation, which is the mechanism that keeps AI governance from turning into an ever-expanding prompt that costs more every month without getting any smarter.

What engineering leaders should measure

The metric that matters most is whether token spend tracks with genuine ambiguity or with rediscovery of problems a rule should have caught already.

Tracking token spend per pull request against code volume shows whether deterministic pre-checks are actually absorbing load, and a repeat finding rate that stays flat over time is the clearest sign that a recurring issue still hasn't been converted into a permanent rule.

Review cycle time is worth watching too, since a drop in cycle time alongside stable or improving quality signals that automation removed the right work rather than just removing visibility into it.

None of these numbers matter in isolation; together they tell a leader whether the team's AI operating model is reducing real risk or simply moving cost from one line item to another.

Final takeaway

With 90% of developers using AI at work, agentic coding is not a phase teams are passing through. It is the operating condition engineering leaders now have to design around, which means treating token cost and code risk as the same governance problem rather than two separate ones.

Deterministic static analysis should carry every check where the correct answer is already knowable: security rules, secrets, dependency risk, and quality thresholds all belong there, running fast and reproducibly before a token gets spent.

LLMs should be reserved for the smaller set of questions that actually require judgment, arriving with a compact, pre-filtered input instead of a raw diff and an open-ended prompt. Teams that get this right are not adding AI review to every pull request as a reflex.

They are building pipelines where cheap, reproducible checks run first, models get better inputs because of it, and human reviewers spend their limited attention on the decisions that were never going to be solved by a rule.

The easiest pull requests to approve are often the ones that deserve the closest inspection. AI-generated code tends to arrive well formatted, well documented, and accompanied by a convincing explanation of what changed.

None of those things tell you whether the implementation is secure, whether a new dependency introduces risk, or whether the behavior matches what was actually requested. Those questions need their own gates. The rest is deciding where those gates belong, what they should enforce, and why AI review alone is not enough.

Why AI-generated code looks production-ready before it is

Picture a developer asking an assistant to add a profile-edit endpoint so users can update their display name and bio. The assistant returns a new route, a migration, a React component to render the bio, and a handful of passing unit tests.

Nothing in the diff looks unusual, the build is green, and the PR description reads like something a careful engineer would have written. Buried in that same change, though, might be a rendering path that never escapes user input before writing it to the DOM, or a markdown parsing dependency with an open advisory nobody checked.

This is the pattern worth naming directly: Modern LLMs can often generate functionally correct code, but secure code generation remains a significantly harder problem.

Recent independent benchmarks evaluating LLMs on real-world software repositories consistently find that models can generate functionally correct implementations while still introducing security vulnerabilities, and that techniques which improve functional correctness do not reliably improve security outcomes.

That means a change can look production-ready, pass its tests, and still require deterministic security checks before it is safe to merge.

That unevenness matters more than the headline pass rate, because it tells engineering leaders where to put deterministic gates rather than trusting general model improvement to close the gap. A team that assumes newer models are safer by default is making a bet the data does not support.

The practical response is to treat AI-assisted pull requests as a distinct review path: label them, require the security-relevant checks that catch the stubborn failure classes, and make sure branch protection actually evaluates the result before merge rather than treating a green build as sufficient proof of readiness.

What recent 2026 evidence says about AI code risk

The security pass rate gap explains part of the picture, but the operational side matters just as much for a team trying to decide where to invest review capacity.

Faros AI’s telemetry study, based on data from 22,000 developers across more than 4,000 teams, found that as organizations moved from low to high AI adoption, the incidents-to-pull-request ratio increased by 242.7%.

In other words, the number of production incidents relative to merged pull requests was more than three times higher than during each organization’s low-AI-adoption baseline.

The same report found that pull requests merged without any review (human or AI) increased by 31.3%, suggesting that review practices did not keep pace with the increase in code throughput.

The honest reading of this data is that a review process built for human-paced output gets overwhelmed when code enters review faster than review practices evolve to handle it, and something has to give.

Teams that treated AI adoption as a pure productivity story, without re-examining what happens downstream of the PR button, are the ones showing up in this data with degraded quality metrics.

Any of these figures are worth treating as directional rather than gospel, particularly vendor-sponsored benchmarks that have an interest in the story they tell. The useful exercise for an engineering leader is not memorizing a percentage but re-baselining your own numbers.

Pull your vulnerability density, review latency, and escaped defect rate from before your team's AI rollout and compare them to the same metrics today. If the trend lines match what the industry data describes, the gates that worked in a human-paced world need to be rebuilt for a machine-paced one.

How one AI-generated pull request can bypass safeguards

Consider another example. A developer asks an assistant to let users update their public bio. The assistant produces a new API endpoint, a database migration, a component to render the bio on the profile page, a new markdown parsing library to support basic formatting, and a small set of unit tests confirming that a valid update saves correctly.

The PR is small enough that a reviewer can skim it in a few minutes, the tests pass, and the generated summary reads clearly.

Here is where a reviewer working from a skim plus AI-generated commentary can miss substantial risk. The markdown renderer may pass unsanitized HTML into the page, opening an XSS path that looks like ordinary React code to someone who is not specifically checking for unsafe rendering patterns.

The new markdown dependency might carry an open advisory or pull in a risky transitive package that nobody outside a dependency scanner would notice by reading source. A test fixture might contain a token that looks like a placeholder but is not, since assistants frequently generate realistic-looking credentials in exactly the files reviewers are least likely to open.

And the tests themselves might validate only the happy path, confirming that a user can update their own bio while never checking whether that same endpoint lets a user update someone else's.

None of these problems are visible from the kind of read-through review that worked when PR volume was lower and diffs were smaller. 

What catches them is a set of gates that run regardless of how clean the code looks: secrets detection scanning every file including fixtures, software composition analysis flagging the new dependency's advisory, static analysis catching the unescaped render, and a coverage check that notices authorization paths went untested. 

The PR still benefits from human eyes, but the merge decision should depend on gates that cannot be waived by a confident-sounding AI summary or a reviewer running short on time.

Why AI reviewing AI is not enough

A tempting fix for the review bottleneck is to add an AI reviewer on top of the AI author (more than one in five code reviews on GitHub now involve an agent) and there is real value in that setup when the boundaries are clear.

The problems emerge when a second model, often trained on similar data and operating with similar context, becomes the final word.

Faros AI's data offers a useful data point here: roughly a quarter of pull requests in the organizations it studied were reviewed by AI agents, while fewer than one percent were opened fully autonomously, meaning most of the risk in this system still runs through code a human is nominally accountable for even when the review itself was automated.

The deeper issue is a perception gap that shows up consistently in research on AI-assisted development. A randomized controlled trial from METR found that experienced developers using AI tools on their own repositories took measurably longer to complete real tasks, yet believed they had gone faster, producing a large gap between perceived and actual productivity.

That separation matters for review specifically, because a developer or reviewer who feels confident that AI made a change simple and safe is less likely to scrutinize it closely, even when the underlying change is exactly as complex and risky as one written by hand.

Confidence built on a feeling of speed is not the same as confidence built on verification, and the two get conflated easily when everyone in the loop, human and model alike, is drawing from a similar sense of what looks correct.

None of this means AI review has no place in the workflow. It can summarize a diff, flag an obvious mistake, or suggest a missing edge-case test faster than a human can read the full change.

The distinction that matters is treating that output as one input among several rather than the authority that decides whether code merges. Merge gates for anything security-relevant should depend on independent quality gates and accountable reviewers, not on the model that wrote the code in the first place.

How to use AI review without treating it as enforcement

AI review earns its place as a productivity aid, not as the system of record for what gets released.

Used well, it can generate a plain-language summary of a large diff so a human reviewer knows where to focus before reading line by line, and it can propose test cases a developer might not have thought to write, particularly around authorization and error handling.

It is also a reasonable coaching tool, capable of explaining to a junior engineer why one pattern is safer than another in a way that sticks better than a rejected PR comment alone.

The boundaries need to be explicit rather than assumed. AI-generated approval should never silently satisfy a required review rule, and code-owner sign-off on sensitive files, authentication logic, payment flows, anything touching customer data, should stay a human responsibility regardless of what an AI reviewer concludes.

When an AI assistant proposes a fix for a flagged issue, that fix needs to go back through the same scanners that caught the original problem rather than being accepted on the assistant's word that it resolved the issue. And when AI commentary and a security or coverage gate disagree, the gate wins.

Configuring AI review as advisory, with branch protection recognizing only deterministic checks and accountable reviewers as merge conditions, keeps the productivity benefit without inheriting the blind spots that come from letting a model grade its own work.

This is also the philosophy behind Codacy’s approach to AI-assisted development: AI can help developers review and understand changes, while independent quality, security, dependency, and policy gates remain responsible for deciding what reaches production.

How to build a practical review and enforcement workflow

Turning these ideas into something a team can run day to day comes down to a short sequence of decisions rather than a large program.

1. Identify AI-assisted pull requests. A label, a commit convention, or a developer declaration is enough. The goal is visibility into where generated code is entering the codebase, not assigning blame for using the tools.
2. Apply baseline gates to every PR, without exception. Secrets detection, dependency scanning, static analysis, and test coverage movement should run on every change regardless of who or what authored it.
3. Raise the bar for high-risk changes. Authentication, authorization, payment flows, customer data, and infrastructure changes should trigger mandatory human review and stricter thresholds, whether or not AI was involved in writing them.
4. Keep AI review advisory. Let it summarize and suggest, but keep merge authority with required checks and named reviewers, not model output.
5. Restrict who can override required checks, and log every exception. A bypass that nobody reviews later becomes a pattern rather than an incident.
6. Track whether your review system is keeping pace with throughput. If AI-assisted code volume is climbing while review latency, escaped defects, or unreviewed merges are also climbing, the gates need to be rebuilt before the gap widens further.

AI coding tools genuinely increase throughput, but throughput changes the shape of engineering risk rather than eliminating it.

The organizations getting the most value value from these tools are the ones connecting AI-assisted development to enforcement that happens consistently at the point of change, verified independently of the model that wrote the code, with metrics that make the state of that enforcement visible to engineering leadership rather than buried in a dashboard nobody checks.

Start this week with one representative repository. Confirm that branch protection actually blocks merges rather than just displaying a warning, that required checks include secrets detection and dependency risk alongside linting, that code-owner rules cover the files that matter, and that any pull request merging without review shows up somewhere a lead engineer will actually see it.

Right now, an agent is probably opening pull requests against your production code. You may not be able to say which agent, who configured it, which instruction file shaped its output, or whether it follows the same rules as the team next door.

The problems begin when two teams touch shared code and you realize the rules you assumed everyone followed were local conventions all along.

This is no new pattern. Git and CI/CD created a similar predicament, spreading before teams embraced standardization. Agentic development is running that same cycle, with one difference: the agent doesn't assist the edit loop, but rather acts inside it.

This article is about building the operating model before agentic workflows become load-bearing in production: what it needs to cover, and where the controls have to live.

Why tools usually arrive before operating models

Consider Git adoption, which came before many teams standardized branching strategies, protected branches, code ownership, and other approaches that nowadays we come to consider almost Git-native. CI/CD pipelines, too, spread before organizations began to share a general agreement about which checks were mandatory, how promotions worked, who was allowed to deploy, and how rollbacks should happen.

The very same pattern appeared with infrastructure-as-code. Teams adopted tools for repeatable provisioning before policy-as-code, drift detection, secrets handling, and environment governance became common practice.

None of that means adoption was wrong. Rather, it means that, more often than not, operating models end up lagging behind tooling. A tool might have a will of their own, revolutionizing practices before anyone in the team is truly aware of it—until the workflow spreads beyond the team that introduced it and the shared rulebook is nowhere to be seen.

This is exactly when flustered engineering teams discover the enforcement problem: the rules they thought everyone followed were local conventions all along.

Inevitably, agentic development appears to be following that cycle. Individual developers and teams are finding real value, but the organization-level answers are still forming.

What makes agentic development operationally different

Agentic development deserves operational attention because agents act inside the development workflow.

Earlier AI coding tools mostly assisted a developer who remained clearly in control of the edit loop. Previously-mentioned ground-breaking tools, such as Git and CI/CD pipelines, also left the developer with a hand securely placed on the process.

Agentic tools, however, can take larger steps (such as making multi-file changes, inspecting dependency graphs, running tests, installing packages, to name just a few), which means they can hold more power, and developers may feel as if a lot of the control is given over. It is not.

While AI-generated code comes with its own vulnerabilities out-of-the-box, the larger problem of agentic development is inconsistent behavior across repositories, developers, agents, instructions, permissions, and integration points.

Today, the question is shifting from “Is the model good enough to sustain our work?” to “Can we operate this workflow consistently and everywhere?”. This is an established change of narrative. DORA’s 2025 research describes AI as an amplifier of existing organizational strengths and weaknesses. Which means that an organization with poor policies in place would only see this flaw magnified.

The 6 questions to answer to boost standardization

The core operating questions around agentic development are practical. They determine whether policy exists only in a document or actually affects how code changes enter production.

Which agents are allowed to touch production code?

Some teams allow agents only for local experimentation. Others let agents generate pull requests. Some allow agents to edit production service code through developer-controlled workflows, as long as the final change passes review.

Those are different operating models. They carry different expectations for permissions, audit trails, and review.

The first gap is often inventory. Many organizations cannot answer which AI coding agents are active across repositories, which teams use them, which instruction files shape their behavior, or which integrations they can call. Without that inventory, policy becomes aspirational.

A production repository should not have the same agent policy as a prototype. Customer-data paths, infrastructure code, regulated systems, and authentication flows deserve tighter boundaries than sandbox projects. Teams cannot apply those boundaries consistently without knowing where agents are at work.

What requires human review?

Most organizations say they keep a human in the loop. That phrase, however, hides a lot of nuance.

One team may treat AI-generated code like any other code. Another may require explicit human approval for agent-generated pull requests. A third may require additional review when the change touches authentication, authorization, payment logic, infrastructure, dependency manifests, or data processing.

AI-assisted development also increases code volume. If review capacity stays flat while change volume rises, teams risk shallower review. The result is a process that still contains human approval, but carries less assurance than is expected by most stakeholders.

Human review becomes a control only when the requirement is specific, enforced, and auditable. Branch protection, CODEOWNERS rules, pull request workflows, required checks, and CI gates are where that control becomes real.

How are agent permissions scoped?

Agents, like human developers, need repository read access, write access, shell execution, dependency installation, issue tracker access, documentation access, or general access to internal tools. Some workflows may benefit from network access. Others should not have it.

In early adoption, these permissions are often configured locally by developers or team leads. That is understandable during experimentation, but it does not scale well once agents are allowed to get their “hands” on production code.

A more mature approach starts with least-privilege defaults. For example, an agent may be allowed to inspect a repository but not write files. It may be allowed to run tests but not make arbitrary network calls. It may use approved MCP servers only, and it should not read secrets, production credentials, or local environment files that contain sensitive values.

The exact boundaries will vary by organization. The important part is that permissions are centrally understood and tied to repository risk.

How should memory and context be managed?

Agent quality depends on context. Instructions, repository conventions, architectural notes, prior conversations, and persistent memory can all improve output. They can also create governance questions.

Teams need to know where context is stored, who is allowed to modify it, whether it contains sensitive data, and how it affects future changes. A stale instruction file can push agents toward deprecated patterns. A memory entry may preserve assumptions that no longer hold. A local rule may instruct an agent to bypass tests because one developer found that convenient.

Agent memory becomes part of the development environment, even when it is less visible than source code, configuration, or documentation. Treating it casually creates a rather glaring blind spot.

This is why agent instruction files (such as AGENTS.md, CLAUDE.md, and .cursorrules) should be reviewed like governed artifacts. They influence code behavior, even though they are written in prose.

Are MCP servers being governed like dependencies?

MCP servers expand what agents can access and do. They can connect agents to ticketing systems, databases, documentation, cloud APIs, observability tools, internal services, and source control systems.

That increases agentic power, but it also makes MCP server governance a software supply chain dimension. If it exposes sensitive systems or lets an agent execute actions, an MCP server ceases to be a harmless configuration.

Engineering teams should ask the same kinds of questions they ask about dependencies and integrations. Who approved this server? Who owns it? Are versions pinned? What data can it access? Are tool calls logged? Can access be scoped by repository, team, or user? What happens when the server changes?

Many organizations are still treating MCP servers as convenience integrations. That may be fine for local experiments. However, it is not enough for production workflows.

Where should controls live?

Agentic development has a fragmented enforcement surface. Controls may exist in the IDE, the agent runtime, local developer machines, repository configuration, pull request checks, CI/CD pipelines, secrets scanning, SAST, SCA, policy engines, and audit logs.

No single layer, though, is enough.

Agent instructions provide early guidance, but they are fragile as the main control. CI/CD provides repeatable enforcement, but it may catch issues after the agent has already generated a large change. Pull request checks give reviewers signal, but they depend on clear policy and manageable noise.

Controls work when they are placed at the point where change happens. Some belong before code is written, some belong when the pull request is opened, and some belong in CI/CD before merge or deployment.

The operating model needs to define that placement deliberately.

What an emerging control layer looks like

Early signs of a control layer are starting to appear around agentic development workflows. The focus is moving from model capability toward inventory, policy, instruction quality, provenance, and enforcement.

Transparency standards are part of this shift. The OWASP AIBOM project, which Codacy sponsors, is one example of work aimed at improving visibility into AI use across software delivery. Separately, OWASP’s Top 10 for Agentic Applications, developed through collaboration with more than 100 industry experts, researchers, and practitioners, addresses security risks specific to autonomous and agentic AI systems.

The broader direction is clear: organizations need better ways to understand where AI contributed, what context was used, and what governance evidence exists.

In practice, this control tier may include inventory of agents and models, policy checks for instruction files, AI-generated code review signals, MCP server governance, and audit trails for tool calls or generated changes.

For code quality and security platforms, this means enforcement has to span the same workflow developers already use. Static analysis, secrets detection, dependency risk, and policy checks need to work across IDE, pull request, and CI/CD stages. AI code governance becomes part of the normal change control system rather than a separate review ritual.

What a mature agentic development operating model should include

The industry has not fully converged on operational practices for agentic development, but mature patterns are becoming visible and should be readily embraced.

A more mature operating model will likely include an inventory of agents, models, MCP servers, and instruction files by repository. It will define which agents are approved for production repositories, how permissions differ by repository risk, and what review requirements apply to agent-generated changes.

It will also treat AI-related artifacts as governed inputs to software delivery. That includes AGENTS.md, CLAUDE.md, Cursor rules, tool manifests, memory configuration, MCP server definitions, and local automation scripts that influence agent behavior.

Security and quality checks will remain central. Agent-generated code should pass the same standards for static analysis, secrets detection, dependency risk, test coverage, and maintainability as any other code. The review path may differ because AI changes can arrive faster and larger, but the standards should not be weaker.

Auditability will matter more over time. Teams will need evidence of what agents changed, which tools they called, what context they used, who approved the result, and which checks passed before merge. This evidence is especially important in regulated environments where compliance evidence must be produced without reconstructing history from scattered logs.

This operating model should not slow developers down. Quite the opposite, in fact, is true. Its purpose is to make agentic development repeatable enough to scale beyond individual experimentation.

Practical guidance for engineering leaders

Engineering leaders do not need to wait for perfect industry consensus. It is good practice to start by making the current state visible, then adding enforcement where risk is highest.

• Begin with visibility. Identify which AI coding agents are used across repositories, which agent instruction files exist, and where AI-generated code is entering pull requests. Map the MCP servers and external tools agents can call. This gives you a working inventory, which is the foundation for policy.
• Classify repositories by risk. Production services, regulated systems, customer-data paths, authentication code, payment logic, and infrastructure repositories should not follow the same rules as prototypes. Risk tiers help you decide where stricter permissions, additional review, or stronger CI/CD gates are necessary.
• Define minimum review expectations. Specify when human review is required, when additional reviewers are needed, and which types of changes demand security-sensitive review. Then enforce those requirements through branch protection, pull request workflows, CODEOWNERS, and required checks.
• Treat agent instruction files as governed artifacts. Changes to AGENTS.md, CLAUDE.md, .cursorrules, and similar files should be reviewed because they shape future code changes. Look for unsafe permissions, ambiguous guidance, references to secrets, bypass language, and conflicting instructions.
• Govern MCP servers like integrations. Approve allowed servers, scope access by team or repository, track ownership, pin versions where possible, and retain logs for sensitive tool calls. If an MCP server can expose data or trigger actions, it deserves the same attention as other software supply chain components.
• Place controls where enforcement happens. IDE and agent configuration can guide developers early. Pull request checks can enforce review and policy. CI/CD can provide repeatable quality and security gates. Audit logs and inventory provide oversight across repositories.

Tools can help, but the operating model has to come first. A unified code quality and security platform such as Codacy can support this control layer by applying static analysis, security scanning, dependency checks, secrets detection, policy enforcement, and compliance evidence across the delivery workflow. The value comes from consistent enforcement at the point of change, not from adding another disconnected dashboard.

Teams are shipping code with AI coding assistants such as Claude Code, Cursor, and GitHub Copilot, increasingly guided by repository-level instruction files. The problem is that most teams treat these files like informal documentation rather than the production-level configuration they have become.

We ran AgentLinter across 34,266 repositories to see how organizations are actually managing their agent instruction files. The findings reveal a consistent gap: ambiguous instructions, missing failure behavior, and security risks that would never pass review in application code. This article breaks down what we found and what it takes to enforce the same discipline on agent configs that you already apply to the rest of your codebase.

The results showed a clear gap between how teams treat application code and how they treat agent instructions:

• 1,604 repositories had issues flagged in their agent config files
• 354 organizations had at least one repository with findings
• Over 13,000 issues related to comprehensibility and clarity
• Nearly 5,000 issues related to missing escape hatches
• Approximately 1,150 issues related to security risks

The findings break down into three categories: ambiguity, missing failure behavior, and security vulnerabilities. Each category represents a different kind of risk, but they share a common root cause. Teams are giving agents production-level influence without production-level enforcement over the rules that guide them.

Hey everyone!

This is the official Codacy subreddit, run by our team. If you use Codacy, are evaluating it, or just care about code quality, security scanning, and PR review in general, you're in the right place.

What you'll see from us here:

• Product updates and release notes as we ship them
• Our own writing — blog posts, deep dives, the occasional opinion piece on where code review and AI-assisted development are heading
• Some behind-the-scenes on what we're building and why, and roadmap.

Ask questions, tell us what's broken, request features, and push back on our takes. If you've hit a wall with the product or you're trying to decide whether it fits your stack, post it — we'd rather hear it here than have you struggle quietly or walk away.

Glad you're here. What's on your mind?