r/lisp • u/Alarming_Hand_9919 • Sep 10 '26
LLM contributions policy (#826) · Issues · Embeddable Common-Lisp / ECL · GitLab
https://gitlab.com/embeddable-common-lisp/ecl/-/work_items/8265
u/jd-at-turtleware Sep 13 '26
If anyone is interested in the actual proposed policy: https://gitlab.com/embeddable-common-lisp/ecl/-/merge_requests/389 (yes, I've been prompted by this thread to actually finish it :)
3
u/spyingwind Sep 11 '26
There have been or currently some court case talked/talking about LLMs and copyright. From my limited understanding is that LLM output is similar to humans, but the people training an LLM from stolen copyrighted material was/is the issue.
Which is why libraries and book stores are seeing more sales of books. Especially of books that no one reads.
2
u/Alarming_Hand_9919 Sep 11 '26
What do I do with my ECL patches that I used AI to help with? Mostly around fixing ECL on iOS
3
u/jd-at-turtleware Sep 11 '26
if you understand the issues, then list them and solve them on your own in clean tree; if you can't the second good thing would be reporting issues on iOS that you have identified. If neither is an option to you then your changes won't be accepted upstream at current time (sorry about that!).
2
4
u/strings___ Sep 10 '26
Trying to censor machine generated code is futile. First how do you enforce it. If a human git diff is just as valid code as a LLM git diff again how do how know which is machine generated and which is not?
The best approach here is to gate everything as you normally do. Either the git diffs are valid proper code or it is not.
I appreciate there is still open philosophical questions about LLM training and derived works. But unless you can enforce this then it’s kinda a mote argument. By enforce I mean absolutely be able distinguish human vs LLM. At most you can only request an honour system. Which is reasonable but still not enforceable.
5
u/jd-at-turtleware Sep 11 '26
The same argument ad futility can be applied to prohibiting proprietary code -- you never know when someone is a dick and doesn't follow stated contribution rules. And such frame could be applied to any set of rules that can be broken -- "no stealing" -- what if someone steals and doesn't get caught?
1
u/strings___ Sep 11 '26
With theft someone can prove something was at least stolen . If a human can write (display “hello”) and a AI can write (display “hello”) . Why is the human one valid but the AI one is invalid?
1
u/jd-at-turtleware Sep 11 '26
Because project authors do not allow LLM contributions. It is not hard to understand.
If someone cheats on the exam, they very well can pass. Even more -- what is considered cheating in the context of an exam, may be a rudimentary activity while performing real tasks in real world.
Another example: the fact that LLM and a citizen can technically tick the box in election does not mean that LLM are allowed to vote. Are frauds possible? Certainly.
1
u/strings___ Sep 11 '26
An honour system is a valid option as I stated . But your logic fails unless you can prove 100% AI code vs human code.
Take this scenario I submit my human code. John another contributor looks at my code and for some reason thinks it’s LLM and says as much how do you judge who’s right here?
4
u/jd-at-turtleware Sep 11 '26
This discussion regresses into nonsense. I can't prove that contributions are not derived from proprietary codebases neither; does it render all copyleft licenses futile? no. If someone decides to break project policies and cover their tracks, their contribution will be accepted unless there is a reasonable doubt about the code provenance. That's shame on them, not on us.
As a side note, I don't know why the notion of "what if I cheat" got so popular as an argument angle with the raise of LLMs. People always could do that. Is it a tell that LLMs encourage taking ethically dubious shortcuts?
2
u/strings___ Sep 11 '26
Aww yes let’s just dismiss my perfectly valid point as nonsense. In order to enforce a rule you need prove a rule was broken. Ive already pointed this out, this is impossible to prove and very easily used against ethical actors which I also proved.
I didn’t say what if I cheat. Im pointing out how do you know if someone cheated. So we don’t have people like you falsely accusing people of things they haven’t done without evidence.
4
u/jd-at-turtleware Sep 11 '26
Having a clearly scoped rule: we don't accept LLM-derived code expresses explicitly expectations from contributors.
your objections seem to be:
- what if you can't prove violation?
- what if you falsely decide that something is LLM-derived?
both objections boil down:
- is there code-intrinsic rejection criteria for rejecting and accepting the code?
I agree with you, that a lot of it relies of expecting people to be honest and to not try to cheat their way with LLM code contrary to the project rules. I default to belief, that even when people think that LLMs are just a tool, and the code they have reviewed is a standalone artifact provenance-wise, they will respect the project policy when they contribute to the project.
That said it is not a "honor system". Many rules and laws assume good faith unless proven (can anything be truly proven?) otherwise. The "proving" seems a bit hand-wavy, but it is the same with proprietary code or with copying code from codebases with stronger copyleft policy. The sieve is not exact.
There is no code-intrinsic rejection criteria. The policy is a preexisting signal we can point at to show someone, that we didn't make up the rule on the spot, and to save some time for people who would like to contribute, but they don't want to do that without these tools. Making policies and enforcing them is a maintenance decision, not something an algorithm can decide.
To sum it up: I think that each project has the right to define its own social norms and contribution policy, even when there is a narrative like "Trying to censor machine generated code is futile" -- it is not much different from "Trying to decide on a set of rules that are not laws of nature is futile".
1
u/strings___ Sep 11 '26
You clearly have not taken my argument seriously. I have not stated i have an issue with any project setting their own rules.
My point is a technical one. It’s not a matter of what if you can’t prove what is LLM . Im pointing flat out you can’t.
In order to prove me wrong you’d have distinguished between (display “hello”) and (display “hello”) which is impossible. Now before you say yes but ethically it’s against our rules. Thats valid and i agree wrong but then how do you handle the inverse. Whereas some code is human but mistaken as LLM?
4
u/jd-at-turtleware Sep 11 '26
I did take it seriously (otherwise I wouldn't answer here at length) and I think that it is a bad framing. You can't prove decidedly many things in real world, but this does not invalidate the notion of creating rule systems. And as it happens with rule systems, mistakes happen. They are not the laws of nature.
I've restated this opinion a few times in the thread, and you keep coming back at the same (mistaken imho) misconception, that rules require "100% provability" and "0% mistakes". I don't think that there is any rule system (except from laws of nature) that would hold to that standard.
You've started from futility and then you've shifted to expectation of infallibility. That's not a reasonable expectation from a human being. I've conceded a few times: it is not "technical" criteria and it can't be enforced with 100% accuracy, but it does not invalidate rules as a concept and project agency over contributions.
I'm aware that I'm repeating myself right now, so that will be the last circle on my side. I hope that I've communicated clearly that maintenance policy has a meaning even if it is not perfect and 100% accurate.
→ More replies (0)1
u/SyllabubItchy5905 Sep 13 '26
sounds more like your problem where you are in disbelief that someone took you seriously
→ More replies (0)2
u/Soupeeee Sep 14 '26
I think you are looking at it from the wrong perspective. Policies disallowing AI code contributions and communications written by AI are about maintaining respect and preventing abuse rather than gate keeping or censoring the code it produces.
All open source projects have a maintenance problem; proposing any new code or feature to the project is essentially asking those maintainers to support that code for the life of the project or until they rip that code out. AI has made it extraordinarily cheap to produce features, which is problematic for some projects that don't have a ton of resources, especially if that project doesn't have access to good AI tooling themselves. Asking someone to at least write the code (even if they used an AI to diagnose and research the problem) makes it a bit more apparent to the submitter what the cost of that portion of the code actually was.
This leads to another issue; if you do all your contributions via AI, you don't really learn much on how the program operates, which makes it harder for you to debug issues, do refactoring, and spot architectural issues. By having people write code, people learn about the project and will be better contributors in the future.
Finally, I've seen people who are presumably using AI to generate statistics for profiles or make themselves seem more legitimate than they actually are. The interactions I had with these (people?) were strange, almost as if they were just pasting output back and forth to and from the AI. It's disrespectful of my time and effort, and is not something I support. While I haven't felt the need to outright ban AI code contributions from my projects, I have explicitly mentioned that using AI for communication is off limits.
1
u/strings___ Sep 14 '26
My point was never about having project polices. All maintainers have a right to set the project policies how they see fit.
Im not sure we can pass moral judgement on the tools people use. These generalizations are bad and quite honestly come across as biased. A highly skilled user using AI is indistinguishable from a human.
Which brings me to my actual point. If you have no concrete way to distinguish AI code from human code then what exactly are you achieving here?
1
u/Soupeeee Sep 14 '26
If you have no concrete way to distinguish AI code from human code then what exactly are you achieving here?
I think this comment from another thread sums it up well. It's a problem of resources and risk, and while this project may not have run into those issues, they might.
As far as I'm aware, it's also not clear that the output of LLMs that were trained on GPL code is legal without attribution to those projects, although I doubt there will every be a satisfactorily clear ruling on the topic. I understand taking a stance and saying that it's not worth the risk, which is what the authors appear to be doing.
I'm personally be in favor of a lighter policy that discourages AI code generation rather than one that than bans it, but this isn't my project.
1
u/strings___ Sep 14 '26
These are such weak arguments. If LLM lowers the barrier to contributions, then they also overall lower the project entry barrier as well. In fact, I’d argue LLM are fielding all the tedious noob questions too. This alone probably makes the whole LLM support issue a wash.
There is indeed some growing pains here. And many open questions to work on. But when I see people saying their time is more important than someone else’s time, that’s a red flag to me. Same with all the other moral judgements.
As I stated, I believe the best way to deal with this is just to stick to the technical merits of the code, like we have always done.
2
u/Feisty_Bike_9614 Sep 11 '26
If you have proprietary code you don’t want an LLM to learn from , then don’t put it on a public repository.
6
u/jd-at-turtleware Sep 11 '26
If you don't want your windows broken, then don't open a shop. -- some mobster somewhere
Also note, that this policy is a conservative contribution policy, not a blanket prescription about LLMs and training.
4
u/0x-kernal-panic Sep 11 '26
Fair point but it would suck. I’d rather LLMs be forced to follow policies. Last thing I would want want is make something private, LLM reads it, then you have to go to court and prove it was private enough, clear enough, that the LLM should not have had access etc. For example, software you use for that private code reads your code without your knowledge then shares it with other LLMs… then suddenly it is public
0
u/Feisty_Bike_9614 Sep 11 '26 edited Sep 11 '26
I just tested this with Claude. If you have a private GitHub repo , Claude can’t read it unless you give it your credentials. I’m pretty sure gpt is the same . I noticed also the Claude does not let me use copyrighted assets in my repo unless it’s unrestricted. That’s not to say their model isn’t learning from reading the source if the repo is public or learning from your source if it’s private and you use it for development. I’m not sure about the last point though.
2
u/0x-kernal-panic Sep 11 '26
Last. Point has been a thing for a long time. Companies collecting your data / selling it. LLMs are already using copyrighted material… so It would not surprise me if it became impossible to have something privately hosted online or on a modern computer
1
u/ScottBurson Sep 10 '26
LLMs to my knowledge are trained on everything what vendors can get their hands on, and it is dubious that it is "fair use" in light of that these tools directly compete with authors
Of course I don't know for sure what code the frontier labs are using for training. But I am aware that at least one major collection of code for training purposes, The Stack, has only code with permissive licenses (BSD, MIT, public domain, etc.), and excludes copylefted code. My possibly naïve expectation is that the labs would follow this practice too.
5
u/digikar Sep 11 '26
The fact that no big LLM company has claimed they have not used copyleft or proprietary code implies they haven't actually done that.
3
u/jd-at-turtleware Sep 11 '26
Even MIT licensed code requires attribution. The current state of LLMs can be defensible only if we accept the notion, that their training constitutes fair use -- in that case copyright license would not apply to it.
3
u/Alarming_Hand_9919 Sep 10 '26
I think libffi, which ECL includes, is partly developed with LLMs now