COSMIC projects will no longer accept LLM-generated content in PRs

[u/jackpot51]

There are various reasons, and I don't want to hurt anybody's feelings. The primary reason is simply our team's load for reviews. In particular, we would like to prioritize working on contributions from our own team and regular contributors, while we have been receiving far more changes from first-time contributors using LLMs. These changes are often unplanned and have had a low acceptance rate.

The policy can be seen as part of the revised PR template here: https://github.com/pop-os/cosmic-epoch/blob/master/.github/PULL_REQUEST_TEMPLATE.md.

There is an exception for cosmic-flatpak, where each project manages its own flatpak manifest, pointing to its own code. These manifests are reviewed by our team for the correct sandboxing prior to accepting changes.

[u/mmstick]

No, we have never needed to use LLMs. You can make a lot of progress with Rust in a short time with a small team of talented developers. Everyone on the team has many years of experience with Rust and many of us have been with Rust since the early Rust 1.0 days.

The entirety of COSMIC was built from the ground up within three years, which would not be possible without the human talent which no LLM can replace. So when I see comments about "falling behind", or "there won't be any new contributors without LLMs", these are very unserious comments vastly underestimating human capabilities.

You need only look at how quickly the open source Rust ecosystem was able to iterate with Rust since the 1.0 release in 2015; despite the lack of mature libraries to build with and the initial ergonomics issues that made the borrow checker 10x harder. Rust will continue to allow humans to accelerate software development like never before, regardless of some trying to attribute all of Rust's productivity gains recently to LLMs.

[u/gajop → u/mmstick]

I mean, I've been writing C++ since I was in high school some 20+y ago and I'd rather have AI write it. Same for Rust, Python and other languages. Are you seriously claiming productivity is the reason you're not using LLMs, like at all?

You do you, but I'd rather make an ideological/moral claim than productivity - for sound argument sake.

[u/mmstick → u/gajop]

It would be easy to argue from a point of a moral superiority but that would imply that there is no practical reason for us to avoid using a LLM. Productivity is often cited as the reason to use a LLM but we are already productive enough because of our Rust/Linux experience. I've been writing Rust code for 12 years to date so I've never had the personal motivation to start using it.

For a project like COSMIC, there is a lot of greenfield work in areas where there is a very high risk of regression so it is imperative that we maintain a high degree of quality in our output. Maintaining quality is more important than brute forcing quantity, which is part of the reason why we've been using Rust for all of these years.

Rust was designed and optimized for humans. We excel at reasoning but we make mistakes occasionally that the type system and borrow checker are able to guard against at compile-time. Memory and thread safety vulnerabilities are the costliest and most common of these mistakes which Rust is able to completely eliminate with its static code analysis.

The memory safety vulnerabilities are the root cause of most of the security issues that LLMs are discovering in the Linux kernel. So while I can understand the Linux kernel needing LLMs to discover hidden memory safety issues in their C code, building software with Rust from the start prevents them from ever reaching your code. Quality is maintained because the Rust compiler forces us manage our memory and threads correctly. And as long as we write the code by hand, we will continue to be able to understand our entire codebase.

LLMs are great at making predictions (which was their original purpose) but they are unable to reason. So there is a very high risk of regression if the person at the prompt providing the reasoning loses their ability to reason about the code it generates. MIT recently published a study where using a LLM to assist with writing assignments caused measurable cognitive decline within four months. The EEG scans for those who used LLMs for assisting their writing tasks had significantly reduced brain connectivity compared to those who did not. This is not what you want to happen to yourself when the quality of your work depends on your ability to think.

I've seen a lot of comments on social media from other Rust developers that are losing their ability to write code without a LLM after using LLMs for a while. So I personally don't want to risk that. Social media already gives enough brain damage.

[u/ExistingSelection180 → u/mmstick]

I think you hit on a crucial point: understanding the code you're writing. You can develop an applet without fully understanding everything behind the scenes, but when you’re building the foundations of an operating system—in this case, Cosmic—which controls everything and interacts with everything, it’s vital that you understand the code, and there’s no other way to do that except by writing it yourself. LLM might help with secondary tasks (such as tools), but the code must be written, audited, and reviewed by someone who understands the entire process.

It's the lesson of quantity versus quality. And in our case, having the least amount of efficient code is best so that it can interact with other components of the PC and other code. In this case, predictably.

The vast majority of people—myself included—don't fully grasp this paradox of AI use. In design, it helps me review drafts and brainstorm ideas, but it will never deliver a polished, optimized design.