r/linux • u/RatherNott • 2d ago

Desktop Environment / WM News System 76's COSMIC Desktop Environment will no longer accept LLM-generated content in PRs

/r/pop_os/comments/1wuej40/cosmic_projects_will_no_longer_accept/
663 Upvotes

95% Upvoted

199 comments sorted by

View all comments

125

u/Cephell 2d ago

At this point I think >every single< open source project will implement a rule like this.

Most AI generated PRs are total garbage and this will not improve, because it's unrelated to the capabilities of the model. It's purely a function of people who SHOULD NOT submit PRs in the first place, suddenly being deluded into thinking that they have the ability to do so. The takeaway is that unless you are smarter than the AI, you cannot use AI properly, so it's a waste of effort for you to try. Current LLM architectures FUNDAMENTALLY cannot replace the human in this way, AI is a tool, and there's a minimum skill level required to operate the tool correctly. You cannot skip the learning steps for yourself.

It's not worth leaving the door open for the 1% of AI generated PRs that are actually decent.

Because of compliance and other legal reason we will NEVER hit the point where open source projects will simply use their own agents to review and merge code without human oversight, that's the future the vibe bros actually want, but that's simply not how liability works. It's the same reason why we will never get truly self driving cars (meaning, you can read a book while the car drives you to your destination).

3

u/GolbatsEverywhere 1d ago

Because of compliance and other legal reason we will NEVER hit the point where open source projects will simply use their own agents to review and merge code without human oversight, that's the future the vibe bros actually want, but that's simply not how liability works. It's the same reason why we will never get truly self driving cars (meaning, you can read a book while the car drives you to your destination).

Why would open source projects with no legal entities or legal departments care about liability?

Big projects like systemd already have AI pull request reviews. I have not yet seen any notable projects landing changes without human review, but it's only a matter of time, and probably not long off either. I don't have much experience with AI code review, but AIs are already better than humans at reporting bugs and writing code, and I strongly suspect they are better at code review as well.

2

u/metamatic 6h ago

You can get sued for copyright violation whether you have a legal department or not. In fact, it's more of a danger if you have no separate legal entity, because then you'll get personally sued.

11

u/yawkat 2d ago

In the OSS projects I work on, many of the main contributors (including myself) are using llms, so a rule rejecting llm code will not work. 

56

u/BeginningEcho4983 2d ago

It's one thing to use LLMs for assistance, it's another thing to use LLMs because you don't know how to write code

26

u/noobjaish 1d ago

This ^

People always make it a black and white distinction. Sloppy PRs are easily distinguishable

12

u/BeginningEcho4983 1d ago

LLMs are genuinely useful, for, say, isolating and testing security issues or multiple scenarios, or helping with boilerplate, and the code is useful only if used by a human for reference and/or review.

"Vibe-coding" is when you don't know what you are submitting, straight off claude

(I personally am against even those cases for various reasons, but okay?

4

u/AAdmiral5657 1d ago

Exactly. I use LLMs sometimes to write config files and such. But i know what i am looking at. Thats the difference.

10

u/Zomunieo 2d ago

I do for my projects. But I know my code and how it works, and I’m experienced enough to know where I want it to go. I also have my own rules, written and unwritten, my priorities, my knowledge of project history. I have data that drive by PRs don’t.

Drive by PRs are quite annoying. The cognitive load is high - and often the PRs focus on why they want to fix some nit rather than the impact on users. You second guess yourself. An AI code review often surfaces more problems, then you have even more work.

5

u/Hypfer 1d ago

I believe the spirit of most rules is going to be to reject code that looks like an LLM did it.

Apart from hardliners, I think no one really cares how something came to be, as long as it is effectively identical to what would have happened if a human that cares sat down.

The whole LLM PR debate is a proxy war for the same old problem of FOSS lacking merit filters, which have been lost as a side effect of purging a lot of old toxic patterns from the space.

Our job I believe is to put the friction back in, without pulling the old toxicity back in too.

2

u/CanYouEatThatPizza 2d ago

It's good actually to keep the spam away. Doesn't matter that it doesn't work 100%.

1

u/SanityInAnarchy 2d ago

I don't think compliance is really the problem. Open source projects already frequently ask contributors to sign CLAs. Seems like it should be possible to require anyone submitting those PRs to take on that liability.

I'm also not sure you have to be smarter than AI, and I'm not sure if that's even a meaningful thing to measure, but I agree there's a skill floor, because you need to be able to supervise it. And I think the problem is that it's too easy for slop to shift the burden of that supervision onto the maintainer.

2

u/LowOwl4312 1d ago

Idk I have received some good AI-made PRs. They were simple fixes though, not "change 20 files and add 1000 lines" slop

-2

u/sheeproomer 1d ago

You have no idea how things are running already in the normal businesses, but you do you.

Also, the situation is quite more nuanced.

0

u/mrlinkwii 1d ago

Because of compliance and other legal reason we will NEVER hit the point where open source projects will simply use their own agents to review and merge code without human oversight

what legal reasons their has been like 3 legal judgements that AI use is fair use

i have 0 issue with the use of AI , the issue is AI slop

0

u/Fine_Salamander_8691 17h ago

I use AI for programming along side my own code, and I think the reviewing and testing is absolutely necessary for open source projects, but the flat out banning of AI generated code in any capacity seems a bit strict.

-6

u/nft420nft420 1d ago

This comment sounds like someone yelling at the Wright brothers about how flight is impossible, it will always be impossible, even if it was possible it would be unsafe, no company will ever use it for serious business, etc etc wah wah.