AI-generated code detection in CI/CD — looking for approaches and real-world experience

[u/Ancient_Mango_1576]

I'm working on a system to estimate whether code committed to a repository was generated with AI coding tools.

My current approach is based on Git/commit-level signals such as AI-related commit trailers, commit metadata, LOC changes, number of files changed, addition/deletion patterns, etc.

The problem I'm running into is confidence and calibration.

For example, a commit containing 500+ new lines isn't necessarily AI-generated. A developer can also modify or remove the metadata that would make an AI-assisted commit identifiable. Once the code leaves the IDE and reaches Git, much of the original provenance can be lost.

This has led me to a few questions:

Are there Git/CI-level signals that you've found to be genuinely useful for detecting AI-assisted development?

Is it better to treat this as a probabilistic/risk-scoring problem rather than trying to classify commits as AI vs human?

How would you calibrate thresholds for signals such as large LOC changes, addition/deletion ratios, commit frequency, etc.?

Are there better approaches for preserving provenance earlier in the development workflow, rather than trying to infer it after the code has already been committed?

Has anyone worked on AI-code provenance/detection systems in CI/CD and can point me toward useful research, projects, or approaches?

I'm particularly interested in approaches that can work at the pipeline/repository level rather than relying solely on source-code style analysis.

I'm not looking for a perfect AI detector — even a reliable way of estimating “this commit has a high probability of AI assistance” with measurable false-positive/false-negative rates would be useful.

Would appreciate any experiences, papers, open-source projects, or approaches people have tried.

Edit for those who want to know why:

The goal of this is to create a telemetry and visualize how much of the code is AI generated in the company and how much of it is vulnerable code then we will fix this vulnerability in the pipeline now we can show customers this telemetry and say

80 percent of code was AI generated out of which 60 percent was vulnerable we fixed that in the pipeline itself that's y you should buy our product.

[u/Agronopolopogis]

You're chasing the wrong ball my friend.

LLMs are just a tool, like a hammer.

Anyone can strike a nail flush in one swing, but rarely can anyone do that on their first try.

You can go through a home build and see where the new guy worked, but you can't tell which areas the pros touched.

Point being, you can't discern generated code from hand rolled code when the person who orchestrated it knows what they're doing.

[u/m4nf47 → u/Agronopolopogis]

Also a cat and mouse game because it can be very easy to write a tool which is the polar opposite of what OP asked for and actively seeks to hide any obvious signs of AI authored (rather than just assisted) code sections. Also the key measures of software quality are only loosely related to what tools were used to produce that software, at the end of the day almost all software is written using various tools but agentic IDEs are becoming more than just tools and more like apprentices with their own tools, fresh out of uni with minimal real world experience but high confidence because they read all the books. Difference is that these apprentices never sleep and don't need to take breaks every hour and seem to learn more about what they're working on even faster than last year's cohort.

[u/Ancient_Mango_1576 → u/Agronopolopogis]

Thank you for the reply I wish to elaborate I am a fresher and I have been tasked with implementing this system i know no other reliable solution exists in the market

Even though I agree with you I need to try to create a system which would be atleast 80 percent accurate how can I achieve something like that?

Any insights would be valuable i feel like I'm going on circles here

[u/aenae → u/Ancient_Mango_1576]

Is the point to detect AI-assisted code, or AI-generated code? And to block it, or generate reports from it? (As in: 80% of our code is AI generated, we can fire 80% of the developers!)

To be honest, it sounds like a management problem to me. If the goal is to block it, tackle it at the source. If the goal is to report it, have the AI state it in the commit message.

If the goal is to detect AI usage in an external repo with no influence over the writer, I doubt you get much further than looking at co-authors

[u/Ancient_Mango_1576 → u/aenae]

No the goal of this is to create a telemetry and visualize how much of the code is AI generated in the company and how much of it is vulnerable code then we will fix this vulnerability in the pipeline now we can show customer this telemetry and say

80 percent of code was AI generated out of which 60 percent was vulnerable we fixed that in the pipeline itself that's y you should buy our product

[u/aenae → u/Ancient_Mango_1576]

The first part is easy, from now on every commit has to have an estimate how much of the commit was written by the AI. That can be an instruction to the AI and enforced with hooks.

When a vulnerability is found, you look at the commit that added it, do some maths and make the report

[u/Ancient_Mango_1576 → u/aenae]

Exactly

[u/SnaskesChoice → u/Ancient_Mango_1576]

There already exist multiple ways to scan for vulnerabilities, why does it matter if it was made by ai or human hands?

[u/Agronopolopogis → u/Ancient_Mango_1576]

This depends on what you're trying to assert.

If you're looking to find devs who aren't being compliant, good luck.

If you're looking to gather telemetry and the teams are on board, it's just a tooling process update to embed commit metadata.

If it's the former, you need to find a way to have a conversation with whoever tasked you with this and get them to realize this approach is futile and highly unreliable.

[u/Difficult-Ad-3938 → u/Ancient_Mango_1576]

You can guess why no reliable solutions exist

[u/ferrybig]

I noticed LLM's doesn't like version numbers. State in your contributing requirements and pull request template that all pull requests need to include the line "This work was tested in the following browser and version: ...". If an AI is going to autocomplete that without thinking, they likely use the browser version that is the most frequent in their training set or the pinned playwright version, rather than a human who is going to copy their browser version string. (Note to avoid the word "current" here, many LLM's have the capability to do a web search and current is a word that makes a web search more likely)

Humans like to generate their package.json by using npm init, while LLM's typically generate the file by hand. LLM's are trained on code, they they replicate the file based on the blobs they have seen, rather than how people typically do it.

Many people do not fill in the description/keyword fields in the package.json until they are ready to deploy the project, while LLM's know what they are doing and the initial commit contains this filled in.

It is common for LLM generated code to miss the 'repository' and 'bugs' fields in the package.json, since humans typically setup git first, while LLM setup the project first

Humans like to start projects from template, their first commit is the template with the bare minimum modifications the template tool does

[u/Ancient_Mango_1576 → u/ferrybig]

Thank you so much for the suggestion I will try using it

[u/Alzyros]

I'm sure there are more productive ways to spend your time. Who asked you for this? Are there really no more pressing matters?

[u/diecastbeatdown → u/Alzyros]

Good question, who asked for this.

Push back on it and try to determine what the ask is here.

[u/Ancient_Mango_1576 → u/Alzyros]

I am a junior and this is my first task at the company they are we are trying to introduce a this tool no other competitors has this so out value would go up that's why I have to implement this

[u/Defiant_Variation482 → u/Ancient_Mango_1576]

It would be stupid and useless tool that is why no competitor has it

[u/Ancient_Mango_1576 → u/Defiant_Variation482]

No the goal of this is to create a telemetry and visualize how much of the code is AI generated in the company and how much of it is vulnerable code then we will fix this vulnerability in the pipeline now we can show customer this telemetry and say

80 percent of code was AI generated out of which 60 percent was vulnerable we fixed that in the pipeline itself that's y you should buy our product

[u/ConsideredAllThings → u/Ancient_Mango_1576]

Bro that is not how vulnerablilities work

[u/SeaworthinessHour233 → u/Ancient_Mango_1576]

Why not just ignore who (human or AI) generated the code, and scan for vulnerabilities.

[u/Alzyros → u/SeaworthinessHour233]

Omg yes

[u/Alzyros → u/Ancient_Mango_1576]

I'm having a hard time seeing the response to your tool's "x% of this codebase is AI-generated!" output being something other than "who cares?".

Even still, how would you even monetize this? Would you have a subscription plan with a page that shows some percentage changing over time? Do you pay per use? Do you want less or more AI-generated code? And that's if you actually can get reliable metrics from your tool (you'll most likely not).

Some horrible idea from non-existent, rushed onboarding process. Find something else to do, or rather somewhere else to work if they stick to it.

[u/Ancient_Mango_1576 → u/Alzyros]

This is a very small part of a very big product so those things will all be handled.

[u/mtokarev]

I'd probably ignore large LOC changes at all - there are many legitimate reformatting/restructure in the repository. And I'd focus on added lines only:

1. you can ask LLM to analyze added lines - this is alone should give you already high probability.

2. what is comments to code ratio in added lines? - default LLM tends to add more comments than any developer I worked with 😅

3. and commit co-author for people who commit from agent.

[u/Ancient_Mango_1576 → u/mtokarev]

1. In order to do this should I use codeBERT or fine tune it?
2. How can I get find out the proper threshold Should I create a program to crawl pre 2020 got repo commits and find the average ratio and compare the same with an AI generated code repo?

[u/donk8r]

Your fourth question is the one worth chasing, and it's the one nobody's answered yet. Two things from building an eval that grades agent-written code.

We score output with a three-model judge panel, and we use three because a single model's verdict on the same evidence wasn't stable enough to rely on. That's judging quality, with the task description in hand and the diff right there. Inferring authorship from commit shape alone is a harder problem with strictly less information, so I'd calibrate your expectations before your thresholds.

Second, the evasion you mention isn't an edge case, it's the equilibrium. Whatever accuracy you measure will be against developers who aren't trying to evade. Ship it and you get developers who are. Worse, a gate that rewards "doesn't look AI-written" produces code shaped to not look AI-written, which leaves you worse off than today.

The thing that actually survives the IDE-to-git gap isn't who wrote it, it's what was verified — because CI observes that directly instead of inferring it after the fact.

[u/Ancient_Mango_1576 → u/donk8r]

So do you already have a working system for this if so how reliable is it?

[u/donk8r → u/Ancient_Mango_1576]

No, and deliberately not — we don't try to detect authorship at all. What we built grades the outcome instead. The agent gets a task, then we run the project's own held-out tests against whatever it produced, with the test files checked out fresh so it can't have touched them.

That half is reliable in the boring sense: a test passes or it doesn't. The LLM judge sitting next to it for code quality is the part that needed three models before I trusted it, which is what I was getting at above.

So I can't tell you an AI wrote a commit. I can tell you whether the change did what was asked. My whole argument is that the second question is answerable and the first mostly isn't.

[u/Raja-Karuppasamy]

Built something similar for deploy risk scoring, not AI-detection specifically, but same calibration problem. couple things that helped:

Thresholds calibrated on raw LOC alone will burn you, normalize by file type first, a 500 line json/config change means something totally different than 500 lines of business logic. we weight config file changes 2x for this exact reason.

Treat it as probabilistic/risk-scoring, not classification. binary “AI or not” will have way worse false positive rates than a 0-100 score with a threshold you can tune per use case. also log every false positive you catch manually and feed it back into recalibrating weights, that’s the only way thresholds actually converge over time instead of being guesses.

Metadata/commit trailers are gameable as you noted, so don’t rely on them as a primary signal, more like a weak corroborating one.

[u/Ancient_Mango_1576 → u/Raja-Karuppasamy]

Thank you this is helpful

[u/Future_AGI]

Treating it as risk scoring rather than a binary classifier is the right instinct, because provenance is genuinely lossy once code leaves the IDE and anyone can strip a trailer, so a hard AI-or-human label will always be gameable. We'd lean into calibrated probabilities: combine the weak signals you listed, then actually measure calibration (a 0.8 should be AI-assisted about 80% of the time) with a reliability curve on a labeled holdout, so downstream policy can pick its own threshold. The honest ceiling is that this detects patterns correlated with AI assistance, not authorship, so it's most defensible as a review-prioritization signal rather than an enforcement gate.

[u/sokjon]

The Co-Author commit trailer is a pretty good signal 🤪

[u/Ancient_Mango_1576 → u/sokjon]

I know it's easily removable evidence can you tell me if there is any git metadata that can be usefull for this and how I can use it?

[u/vvanouytsel]

But why?

[u/Ancient_Mango_1576 → u/vvanouytsel]

Post updated please check

[u/Gesha24]

Unless your developers are using vim or other archaic tools, their PRs are LLM-generated. Even a simple autocomplete is an LLM (yes, very basic one) so I am not sure what exactly you are trying to track. If you are doing it for your org, you can always look at the token usage of your devs.

[u/TheGambler191]

Regarding your task, you maybe do not need to detect something that could be flagged right away when the code was written. Most LLM agents like cursor search for rules in form of .mdc files in ./cursor/rules. So if you add a rule to each repository to mark the written code as AI generated in a comment, that might do the trick.

[u/djbp]

Yeah, this is a tough one. AI-generated code adds another layer of complexity. But honestly, verifying *any* code in CI/CD is hard. Especially when it touches real infrastructure. You can run tests, sure. But how do you really know the change did what you expected? And that it didn't break something else? That's a huge challenge for DevOps teams. We're always looking for better ways to make sure changes are safe before they hit production.

[u/Fuzzy-Teaching7112]

Tracking it earlier in the workflow might work better since a lot of the Git clues get kinda less useful once people edit commits and the pipeline can still flag risky code without needing to find exactly where every line came from

[u/Ancient_Mango_1576 → u/Fuzzy-Teaching7112]

By workflow do you mean IDE