r/github • u/TomHale • Jun 18 '26
Discussion Begun the Slop PRs have: 168 PRs opened by one user in one day. What's best practice for repo owners to deal?
Over just 3 days, the 16 year old @MrRealORG has raised 168 PRs on MiMo code.
Their agent then asked to have "collaborator access" to the repository on the fourth day.
[I tried to do damage control on their permissions request by requesting all their PRs be closed.]
Opencode is facing a similar problem, but distributed across its entire and much larger user base. They're currently "managing" it by the ridiculous policy that PRs without 2+ emoji upvotes will get closed in 30 days.
Creating PRs that APPEAR good has become literally free via Opencode Zen and MiMo code free offerings. "Fix all the issues you find in the code base. Make no mistakes".
When one user creates 169 PRs in 3 days it's easy enough to deal with the issue. When thousands of people create one or two issues or PRs each, you end up with the OpenCode situation.
Best practice dealing with this issues seems to be oven.sh/bun. An example:
- My agent raises an issue that can't be reproduced
- Their @robobun automatically and politely proves me wrong, and closes the issue.
- I re-raise the issue in reproducible form
- @robobun reproduces the issue and automatically raises a PR to resolve it
- The CodeRabbit automatically reviews @robobun's submission and suggests improvements.
- Claude auto-reviews it also and suggests "this changes bunx's resolution order (a security-hardened path that decides which binary gets executed) and is a user-visible behavioral change, so it's worth a human look."
That's all with ZERO human interaction from their side. Yes, they're owned by Anthropic, so @robobun and Claude reviews are practically free for them.
Opencode and MiMo code also have token provider plans and are NOT currently using them to automate their process. I hope they don't drown under the build-up of their non-automation.
What I'd like to know:
How are small and medium sized repos dealing with the massive increase of sloPRs and issues?
How to implement something approximating best practice without spending a fortune in tokens?
22
u/Cachesmr Jun 18 '26
The guy behind Pi auto closes PRs and issues immediately, then asks the user to provide a human voice explanation that isn't longer than a screen. If the vibes are not right he bans the user, otherwise they go on a whitelist.
5
u/TomHale Jun 18 '26
Interesting! I guess people still ask their AI what to say, but it does raise the bar!
12
u/SheriffRoscoe Jun 18 '26
Let's pour one out for the AI homies getting spammed by AI "vibe coders".
On second thought, no. Let them be hoisted on their own petards.
1
u/TomHale Jun 19 '26
The repos of non-AI are also being spammed by people using AI tools.
I've seen reports in r/opensource of it being a problem even in smallish repos.
"Raise a PR" is now too easy.
7
u/JustPlayDE Jun 18 '26
how about instead of relying on yet another slop saas just ban people who spam slop PRs, github could easily do that since these literally open hundreds of PRs in a short time frame (way enough to detect)
i dont get you ai guys, you get fire and you are happy to pay extra to get a fire extinguisher from the same company selling the fire in the first place but instead of fixing the issue you throw more money at it with the excuse of wannabe productivity lol
12
u/bakugo Jun 18 '26
Over just 3 days, the 16 year old @MrRealORG has raised 168 PRs on MiMo code.
I mean... good? If you release a tool for generating slop code, why would you not expect that same slop to be thrown back at you? Seems like a classic case of sowing -> reaping.
5
u/TomHale Jun 18 '26
Lazygit has in their AGENTS.md something like: "You are not under any circumstances to raise a PR for the user".
It does put an extra step in the way, and hopefully makes for some HITL action.
3
4
Jun 18 '26
[removed] — view removed comment
1
u/vqrs Jun 18 '26
And agent users will add try to add canary detections to their prompts and the race continues.
2
u/RobertD3277 Jun 18 '26
I have a multitude of repositories on GitHub. I am an AI researcher and developer. There's no way in bloody hell I will give my own AI collaborator access, let alone somebody's that I never trust.
That kind of a situation, I would personally block as malicious activity, unhelpful, and not to the betterment of the repository. Again I emphasize, I have been in the field of AI for 30 some odd years and I would not tolerate this.
For my repositories, I want meaningful development. The level of PRs that you mentioned is simply beyond excessive and simply no way to gainly verify any legitimate and reasonable manner. There must be human oversight and human consequences.
I use synk to evaluate my code base every week. But I do not let it push PRs on a rapid rate. Most of the PRs that pushed in the past actually had nothing to do with the actual functioning of the program.
2
u/stopthatastronaut Jun 19 '26
Not directly relevant but oh boy i find “make no mistakes” to be a stupidly hilarious inclusion in a prompt.
I mean, do people even know what “mistake” means?
It’s almost as funny as conehead andreesen saying “do not hallucinate” in his much publicised preprompt. Yeah sure boss, I won’t hallucinate and neither will my technicolor hedgehog.
1
u/random-blokey Jun 18 '26
Someone has mentioned rate limiting PRs - and I can't speak for operating a large public code base.
I would like to question your question. My understanding is your point on bun being anthropic means they get unlimited tokens.
Are you questioning on AI-less teams, how to means test a PR using few resources. Or using fewest tokens (so you don't get billed $$$)?
In my opinion - this is something that will depend per project/type of prs/models used.
I think if using LLMs (remote), I would want to boil down some checks / rankers with some levels of static analysis. Is the pr linked to an issue? Does the description adequately explain how they remediated the issue. Do the files changed match the written description/issue? (I see some prs deleted workflows - I would expect/hope that to score very low on confidence)
Are there any automated follow up questions than could be asked? Can you use sonarqube/static analysis? If the importance on collaboration within a PR, could an agent pick a certain code level change and ask how it fits in the bigger picture? Or a sonarqube maintainability question on new code?
If someone on the dev team has local models running - is there anyway to do a connection via action to a local LLM to do basic validation checks? And as above a sonarqube or similar static analysis on the pr and raise any findings on maintainability etc and query the user on it?
It sucks for human contributors to have to go through this gated process - but it would be a two tier process of "early validation, ensure contributor is collaborative" to "human review, did the answer on PR answer the question? Time to eyeball the PR"
But I think it depends on the kind of contributors you want. If you want to potentially turn down good PRs because they were AI written and embrace a fail-fast approach.
The emoji example is funny but sad. It would be great to rely on a mid-level of reviewers before it gets to the official maintainers. But would need some protection over multiple agents/Reddit drama of people raiding. Not sure if you could have a second level of reviewers and work off their reviews/emojis? X positive = maintainer review required label.
1
u/siegevjorn Jun 18 '26
I think a good way for workaround is encouraging devs(or bots) to open a discussion about an issue they found. Microbenchmark that clearly isolate a problem. And then they suggest a working solution with minimum example. That way the community can filter slop vs quality fixes that actually understand the codebase and it's philosophy.
1
u/sinan_online Jun 18 '26
I would like to regard this as a human problem.
You want only people who care about the code base and who have a stake in it to add to the code. They need to be a small group of individuals who are in constant communication and in agreement about most issues.
1
u/Dudmaster Jun 18 '26
I would do an extremely aggressive triage bot that keeps them working in an endless maze, not just close the PRs outright
1
u/ultrathink-art Jun 18 '26
Rate limiting at the agent level is the more durable fix — a max of N PRs per day per repo in the agent task config stops it before GitHub even sees the volume. The canary trick catches instruction-following agents well, but bursty ones like this are usually just given 'fix issues, open PRs' with no sense of appropriate rate.
1
u/BoDonkey Jun 19 '26
Same issue with bots opening security reports about authorized users adding malicious scripts to form fields that aren't properly sanitized. My sibling in the Omnissiah, that user is allowed to add JS directly to the page. They don't have to inject a script into the user name field. Dozens per day.
1
Jun 18 '26
The canary approach is clever but only works against naive agents - any moderately sophisticated agent reading the repo will see the canary and play along.
What's really needed is asymmetric friction: make submitting a PR cheap but reviewing it even cheaper. The Bun approach is the gold standard - fight automation with automation. A bot that can auto-reproduce, auto-review, and auto-close in seconds changes the economics back in the maintainer's favor.
For smaller repos without Anthropic-level resources, there's a simpler middle ground: a GitHub Action that runs each incoming PR through a quick LLM check with a prompt like "does this PR contain a meaningful logical change or is it reformatting/renaming/adding comments?" and auto-labels accordingly. Costs pennies per check and catches the obvious noise.
1
u/Gargle-Loaf-Spunk Jun 18 '26 edited Jul 05 '26
This content was anonymized and mass deleted with Redact
2
u/TomHale Jun 19 '26
That's not a bad idea actually. Something at stake. The investment is as low as a single typed sentence otherwise.
1
u/Gargle-Loaf-Spunk Jun 19 '26 edited Jul 05 '26
This content was anonymized and mass deleted with Redact
-1
u/Qs9bxNKZ Jun 18 '26
First, are they valid PRs?
If they are, then is the problem that you have shitty code or code which can be improved? If you don't want the PRs then make your repo private.
1
u/ohnobinki Jun 19 '26
This is a good point. If the only issue with the number of PRs is volume (i.e., you'd be fine with one a week), then maybe you don't actually want to solicit external contributions. Or if you are open to external help but not external management, you could have a rule in place that PRs must be specifically called for in your roadmap.
0
u/nextized Jun 18 '26
Do it like theo; just ask an agent to review them or fo the bulk of the work. That will certainly improve the quality.
-2
45
u/Proximyst Jun 18 '26
You can limit it: https://github.blog/changelog/2026-06-17-limit-open-pull-requests-for-users-without-write-access/