It's a fair policy. Getting those verbose, AI-authored walls of text is very annoying, especially when you're expected to thoroughly review it. It's like a denial-of-service attack on the human mind. I can only imagine how frustrating this can get in open projects that get a lot of contributions.
However, I don't think this will discourage AI-based coding at all. In fact, I see two potential outcomes of these policies:
- Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.
- Positive: Submitters actually provide to-the-point, no-bullshit commits and comments - "here's the code, here's why I made that change, here are the effects of that change". Even if AI-generated, these small contributions may become much easier to verify & validate. We may even see some standardization in terms of what qualifies as an appropriately sized contribution, what requires more thorough review (e.g., adding unverified dependencies), etc.
I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
> - Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.
From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.
> Positive: Submitters actually provide to-the-point, no-bullshit commits and comments
That would be a dream.
> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.
The policy allows the reviewer to reject it on the "AI" grounds.
> allows the reviewer to reject it on the "AI" grounds
… but still unfortunately leaves reviewers having to spend time checking submissions and rejecting them.
oh but it'll be very2 helpful and the time spent will be short. It's easy to verify:
* new contributor?
* more than 10 files affected (higher count are more valid)?
* wall of text on description without screenshots, etc?
just close the PR as AI, and then the contributor can challenge it if they feel it should not.
A contributor in good faith is going to accept criticism and resubmit an improved change: less files modified, more explanation, more focus, references to actual tickets and discussion with actual developers.
At least half the people firing off LLM generated PRs will have left the "Coauthored-by: Claude" line on it allowing automated rejections.
Unfortunately, only a single PR like this comes to mind. Most AI authors we've seen were identifiable mainly by overly verbose PR descriptions, meaningless code changes and copy-pasting more AI output when questioned.
> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.
True. At least with a policy about it, the project maintainers can unilaterally close such PRs without further internal or external discussion on any case-by-case basis.
Dingdingding, we have a winner. The main use of such a policy is to be able to just close those giant wall-of-text PRs and have something to point to when people start to scream it's not fair.
> when people start to scream it's not fair
Or LLMs, as we have seen.
Or, who knows, the "AI" might gain sufficient intelligence to read the policy...
Then we'll amend the policy to instruct LLMs to author PRs as Fred Flintstone, yabba dabba doo!
Why is a policy necessary. you were never entitled to have your pr merged in the first place? If pr wasn't reviewable pre AI I'd expect it to be closed or ignored too
The policy isn't necessary to close the PR. The policy just helps to shut down the ensuing discussion after closing the PR. It helps in quickly dealing with well-meaning onlookers asking for clarification when you block PRs from the account.
Still, your not even entitled to a discussion though.
Before AI a large pull request was enough effort to make that you could assume good faith work on the part of everyone. Likely even if it is bad for architectural reasons it is solving an itch other users have and it is reasonable for them to want an explanation why you refused someone who made this much effort. And since the effort required meant it didn't happen often it wasn't a big deal to provide that.
These days large PRs are easy to create and so humans need to shut them down.
Whether or not someone is entitled to something has very little bearing on whether someone believes they are entitled to something (and are willing to waste everyone's time to make a stink about it). Having clear rules to point to, even after the fact, is surprisingly effective in mitigating that.
Or, put more bluntly, your belief in what people ought to feel entitled to has no bearing on what they do believe, and policy needs to address the latter, not the former.
Prior to AI making a PR involved considerable effort from a human. So the default position for many open source projects was that it deserved some level of attention for the effort. Even if many projects in practice would struggle to review every PR. But with AI tools this dynamic has shifted dramatically - many PRs have basically zero effort been put into it. Additionally there are many more of them, and often way bigger also.
But now with AI, this should be "easier" for some definition of easy. In the sense that in the past, this might have taken 15 minutes to write, now with AI, this can take 5 minutes to write by first getting AI to produce a summary and then using human judgement to make it better. So, it's a good idea now to actually demand the dream.
If people knew how to get AI to write terse, focused summaries, sure, that might help. I haven't seen many that do (well, ignoring the toupee fallacy).
Though the most important aspect is that we need to know the motivation and thought process, and all AI can do is fabricate a 'plausible' one.
Reading AI PRs reminds me of Monty Python's holy grenade:
"And the Lord spake, saying, ''First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.'
I wouldn't mind reading that and having a good chuckle while processing an MR, as long as the comment had been crafted by a person. But now I think writing in grunts is gonna become the thing. "pete? you good boy? we won't hire you/ paper you gave HR girl with older gigs? remember it pete my boy? too many words/ words were too long/ dots you used dots/ you scratched long dash with knife, but baby saw scratch/we don't ai pete/ask if they have job in next cave/good luck."
I've found that the instructions "be extremely concise" gets me much closer to output that's actually sensible/helpful rather than another wall of text.
> If people knew how to get AI to write terse, focused summaries...
... then flooded maintainers would be doing it.
I have spent many years reviewing and editing design and product documentation from more Jr engineers and product managers.
It was a constant struggle to get them to be concise, one I mostly lost.
They could allow AI PRs, but then have another AI PR reviewer reject them if they do not match the definitions for `to-the-point` `no-bullshit` commits.
And who pays for the (likely significant, and controllable by everyone) tokens such a system would use?
A small 4-9B model would be able to run cheaply for this sort of work.
What question do you think you're answering?
The Godot Foundation pays for it if it furthers their mission.
Does it?
Please provide 3 examples where layering on MORE of the offending technology has solved the problem. Spam? Malware & Viruses? Customer Service? Hiring & Recruitment?
> That would be a dream.
“Mission. Fucking. Accomplished.”
https://xkcd.com/810/
> That would be a dream.
We just recently started that policy so we'll see how it goes. If anything, having it stated as policy lets us filter out these requests without spending brain tokens on them.
I've always instructed Claude to check the policies first, frankly I'm surprised it's not smart enough to do that already. Would be easy to add to a system prompt. But usually it doesn't matter because many projects have no policies, or maybe they exist but only in hidden forum posts issues or something.
> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
The problem is that a lot of AI contributions are lazily produced without review. Those that have been properly reviewed for correctness (tested to ensure actually working with no obvious undesirable side effects, tweaked where needed to be readable and understandable, fitting the other guidelines of the project, etc) will be indiscernible from human-only contributions, but there are a lot of people who make no such effort so the majority are not nearly this good.
> The problem is that a lot of AI contributions are lazily produced without review.
That sounds like a contributor problem. Not an AI problem.
I still don't understand a "no AI" policy whose only purpose is to weed out bad PRs. You should be weeding out bad PR's regardless of their source. I don't see why treating a purely human-authored, but bad, piece of code should be treated any differently than an AI-authored one.
All they've accomplished is creaking an environment where good code can't be submitted unless the submitter lies.
Probably because a human authored contribution, no matter how bad, can be trained to make it good and also improves the community.
> can be trained to make it good and also improves the community.
AI can be trained. Also, AI can create code that improves the community. It's replies like this that leave me even more confused.
Human being trained is already proven (that’s how most maintainers came to be). Can you explain how AI can be trained in the above context?
lol Human beings are famously not already trained.
> AI can be trained.
I don't have access to a data center's worth of GPUs, unfortunately. You offering yours?
So when you said "AI cant be trained" you didn't mean "AI cant be trained" you meant that you personally can't? lol.
I'm saying that most people, even most programmers, don't have the facilities or inclination to train new models man. I've done it plenty of times and I absolutely don't recommend it.
Frankly I don't think you even know what the phrase entails.
You should not be weeding out bad PRs regardless of their source! A pull request is a social artifact whose value and meaning is dependent on its author; bad PRs from a human author often mean things such as "I'd like to learn how this works and join your community". So it can be both satisfying and worthwhile to spend your effort on cleaning it up, even if it starts to take as much or even more effort than doing it yourself would have.
You're not the first person I've seen argue that authorship doesn't matter, so I don't want to blame you for it, but I really don't understand where that idea is coming from. To me it seems obviously wrong.
> You should not be weeding out bad PRs regardless of their source! A pull request is a social artifact whose value and meaning is dependent on its author; bad PRs from a human author often mean things such as "I'd like to learn how this works and join your community".
I think the difference in perspective might come from the fact that to many people the code and features matters more than any community or the idea of participating in it. If it works, it works.
Or maybe they’re not even indifferent about the community, just upset at people throwing away working code.
The Godot maintainers have decided they don't support that perspective. In their words, they value "being cautious about feature creep" and "dedicated to high code quality"; they don't accept that all working code should be merged.
Aspiring contributors who'd like to make a different tradeoff are of course free to make a fork. But then all of the stuff in their fork won't benefit from the participation of the community, which I suspect most such people do value even if they identify as a "code first" person.
> they don't accept that all working code should be merged.
That’s perfectly within their rights to do!
> Aspiring contributors who'd like to make a different tradeoff are of course free to make a fork.
Too many of those do fragment the development effort and hurt any project.
Here’s hoping that Godot doesn’t struggle too much with people who don’t care about their rules and spam PRs regardless and that the people who want to commit AI code regardless because it works and is good in their eyes at least demonstrate enough initiative to cheat convincingly (maybe actually read the code and make it their own). Godot is a pretty cool project!
Wonder what the middle ground would look like - a project with super high test coverage and tooling, that also requires at least 20 USD in Opus tokens spent on review on the behalf of the author or something, before an actual human being is bothered with it? Heh.
> A pull request is a social artifact whose value and meaning is dependent on its author;
Says who? How can you say I'm categorically wrong when your entire point rests upon an opinion?
Says the definition? I don't really understand your response. A pull request is a request from Alice (the author) that one of Barbara, Chris, Daniel (the maintainers) should pull her code into a particular branch.
Many communities do have a norm that all authors are to be presumed equal, as long as they're prepared to take advice and learn from it. (That's where all experts start, after all.) That's the norm that Godot are trying to protect here. If they don't stop accepting AI-authored contributions, they worry, reviewers will start to implicitly load-shed by not reading PRs from people they don't recognize.
A social contract? You invented that out of thin air. They're a means to an end not some discussion on philosophy.
So you can run your project that way.
You don't get to dictate that other people run their projects that way.
> A pull request is a social artifact whose value and meaning is dependent on its author
...and the project to which it is submitted.
SpicyLemonZest is not the sole arbiter of what PRs mean and stand for.
I was explaining why the Godot maintainers have chosen to enact the policy described in the source article, in response to a comment saying that they should not have enacted it. I don't understand why you think I'm dictating or arbitrating anything.
It’s just a social thing. Two identically bad submissions have different social contexts.
> > The problem is that a lot of AI contributions are lazily produced without review.
That sounds like a contributor problem. Not an AI problem.</i>
Ish. The tool is not doing the job fully, the contributor is not doing their task properly, checking for that, and fixing the issues.
> I still don't understand a "no AI" policy whose only purpose is to weed out bad PRs. You should be weeding out bad PR's regardless of their source.
Think of it like offering a job, and getting 100s of CVs. You don't have time to review each in detail, so you weed out a chunk of them using superficial cues that have been indicators of issues in the past. They are filtering the other contributions too, the no AI rule is just part initial triage to save a lot of time.
You have missed a key point in what I wrote: "Those that have been properly reviewed for correctness [before being submitted] will be indiscernible from human-only contributions". It is like a Turing test: if you take the effort to make your AI aided contributions good enough to be indiscernible from good human-only contributions then the no-AI filter won't bother you because the people or automations enforcing it won't be able to discern that your contribution was assisted.
> where good code can't be submitted unless the submitter lies.
No. Unless it looks like a good contribution looks.
The problem a lot of projects are facing right now is being inundated by bad PRs from people using AI without the effort to, or likely the knowledge of how to, properly understand and explain & document the change. People have a limited amount of time and if the filter saves a lot of time then maybe losing one or two good contributions due to the filter is worth it - better that than the project stalling under the weight of managing all the PRs.
This is one of the reasons, even pre-AI, some projects declared themselves "open source but not open contribution".
Saying you didn't use AI when you did, regardless of the end product, is lying. What definition of the word lie wouldn't cover that?
If they are simply filtering out based on signs of AI, then there is no lie.
If they are explicitly asking “was AI used” then yes, there is a direct lie involved.
If there is no explicit question but you are including “lies by omission”, then be aware that there are many things people choose not to say that they would be unfairly judged for if they did state (gender details, certain parts of their history, etc) so that might not be a line we want to draw.
I'm not exactly pro-AI, in fact I'm avoiding it in DayJob to the point where I expect at some point I'll be given the choice “get with the program or leave” (I plan to leave at that point, preferably by my own choice though we'll see how heated the discussion gets!), but if there is a at least one human at the end of the process that is properly testing and cleaning everything up so there can be nothing problematic in the AI output that survives, is that really practically any different to a fully human created contribution?
The problem with AI contributions in the vast majority of cases is the lazy ones, especially considering the sheer number of them. Using signs of AI as a code/documentation smell when it comes to filtering contributions is fine, especially if you have up-front requested no such contributions be sent. Permanently blocking people who have sent obviously AI-generated contributions previously despite you asking them not to, is also fine just as blocking people who have sent in entirely human made contributions that break your other requests/rules is. But banning all AI use when if you can't tell if a good contribution did or didn't use AI aid somewhere in the process, is both a daft extreme and actually impossible (if you can't tell, how do you tell?!).
A tool that encourages bad behavior at scale is a bad tool. Blaming individuals for a collective problem leads to no improvements.
> It's like a denial-of-service attack on the human mind.
I think this may be an example of deliberate hostile design, attempting to force users to adopt LLM based solutions to then summarise the vast output. Pushing back against AI contributions as such in this context makes sense, especially in software with an existing proven track record of great value delivery like Godot.
There's no chance that anyone saw that far ahead in the future and planned it. It's emergent behaviour.
Who says anything about „this far in the future“? It’s enough for Anthropic et al to realize this one or two model versions ago, see it as a strategic advantage and push for that behavior.
Big tech: "Should we add any functionality at all to filter AI slop in any of our platforms?" "...nah"
Its not 500 moves ahead.
While it certainly didn't enter a mind of any director making decisions (because they can't comprehend not defecting in a prisoner's dilemma, being sociopaths), it was plainly obvious to every person even remotely connected to IT in the past two decades. If one makes a better and faster spam generator and the same unchanged program also works in reverse, by sifting through spam and condensing it to a readable summary, that it will be immediately co-opted in a spam arms race by all sides of the war and become essentially mandatory.
Literally some of the first advertised uses for LLMs were both "You can feed it bullet points and it will compose an entire email" and "You can take long emails and condense them into bullet points!" They've been doing this since day 1.
AI also works better with concise, focused, high information density text. So AI-spam text hurts both humans and AIs, but humans more so than AIs. It is always a negative, except for the "competition" between (human with) AI and human without AI.
This was the original rule in linux kernel as well. No more than 200 loc per patch. We should also introduce this to git commits and pull request descriptions:
1. 400 chars/10 lines per commit
1b. Not more than 3 commits in the initial pull request
2. 20 lines of explanation for pull request
3. not more than 3 pull request open at any one time
Seems like this policy would apply pretty well regardless of who/what generated the code.
YES! "No AI" policies that are purely based on technical grounds make no sense to me. Bad PR's are bad PR's regardless of their source.
Are we really in a situation where good code that solves a problem won't be merged because the person the person checked the "I used AI" box on the PR?
Ban PR's that are too big, don't have a clear purpose, touch too many areas, etc.
It's really a question of how much time you're willing to spend sorting through spam. "No AI" might be a blunt hammer, but the people submitting slop aren't reading guidelines anyway, and it's easier just to reject it early. Frankly, I'm sure if people wanted to sneak in an AI generated code by carefully reviewing it and making sure it's targeted and well tested... I'm sure they could, but those people aren't the problem.
> Frankly, I'm sure if people wanted to sneak in an AI generated code by carefully reviewing it and making sure it's targeted and well tested
But this is exactly the point I'm making. If the code is carefully reviewed, targeted, and tested, then why make people have to lie in order to submit PRs?
Why not just say "Irresponsible use or agentic-based PR's will be auto-rejected"?
And that's not even mentioning that tools like Github Copilot can just act like fancy autocomplete. There are dozens (if not hundreds) of different ways to use these tools.
I guess I'm just really not sure how you can unequivocally forbid these insanely powerful tools when they are almost certainly going to be a large part of developer's workflows going forward?
If a commit is written by AI but reads as authored by a human, the developer has done their job and nothing will be flagged.
If commits written by AI wouldn't be substantially different, there would be no need to reject them.
So I agree with you that it won't discourage AI-based coding. But that's not even the intent.
> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
It's pragmatic. Linus once said, the reason C++ is not allowed in the kernel is to keep the C++ people out.
Joke's on him, many Rust people are current or former C++ people.
just because someone writes or wrote C++ doesn't make them a "C++ person", and "C++ people" are very much against rust
Only true Scots... err C++ people are against Rust, I see :>
> Switching to a more modern topic, the introduction of the Rust language into Linux, Torvalds is disappointed that its adoption isn't going faster. "I was expecting updates to be faster, but part of the problem is that old-time kernel developers are used to C and don't know Rust. They're not exactly excited about having to learn a new language that is, in some respects, very different. So there's been some pushback on Rust."
https://www.zdnet.com/article/linus-torvalds-talks-ai-rust-a...
In my day job, I do a lot of AI coding but almost never have Claude actually create the PR titles or descriptions for me. It produces too much content, and the justification/background sections are often not quite right.
Most PRs to me are not coming out of nowhere anyway, rather they're "here's the linked issue, I started out addressing it by doing X and Y, but then Y got hairy so I switched to Z, hope that makes sense but happy to discuss further as well."
And most feedback is not "let's have you explain the design to me in a diff comment" but rather please explain this design in a code comment so that the next reader of the source will have your context.
I think OSS maintainers are in the middle of intersecting trends:
- tough hiring market, especially for more Jr candidates
- the perception (true or not) that OSS contributions help get attention from recruiters
- LLMs making it very easy to generate “contributions”
i want to figure out some extra practice - get a step of claude sending me a PR, then me accepting it after review, and then rewriting the merge to be a new PR for general review
Yeah I think this is a good approach. I’m pretty AI-optimistic when it comes to making code changes. But reading AI generated descriptions (including pull requests) is absolutely the worst. That content really needs to be human written. Not just for the benefit of the reader, but it also helps the writer exercise their understanding.
How strict is this no AI policy?
Say AI is used to identify and rewrite a single function that improves performance or fixes a bug, then the developer carefully reviews and tests it and submits a nice tight PR with all human communication.
So they don’t want that? They would just reject it?
If I’m understanding correctly, under the policy the higher performance function / bug free submission would be rejected and they could ask for a rewrite.
Should it then be rewritten from scratch, and clean room engineered so it doesn’t resemble the AI too much?
from TFA:
> The Foundation says we can expect Godot's contributing policy to soon include explicit rejections of AI-authored code, noting that contributors should only use AI assistance for "menial things" and must disclose its use. Additionally, the Foundation will reject any AI-generated text in human-to-human communications, saying it's "a basic principle of respect"—though it says machine translations "are still acceptable" if the original text was human-authored.
As long as your bots aren't contributing low-effort garbage in a push to give their operator some of those tasty internet brownie points you should be fine
All these projects should seamlessly run a fork in parallel that accepts AI and has AI for review and approval. Both camps are happy.
Basically a play sandbox for contributors to not get jaded. A honeypot to contain the verbosity vomit, while also serving as positive public relations by keeping young contributor morale from starting in the basement.
Everyone has been that person once early in their life who is told they aren’t welcome and never comes back. Maybe it was SourceForge or IRC, maybe it was Wikipedia.
Do you volunteer to maintain the sloppy fork?
I think most pro-AI people would be happy to let an AI maintain the sloppy fork. What reason would they have to complain, after all?
This is actually a good idea.
I mean, not sure if everyone wants that for their project, and there will surely be plenty of trade-offs.
But it would be a very good compromise: You (the maintainer) get only human-generated PRs in the canonical project, and they (pro-AI contributors) get a lower-threshold sandbox to play with. Best case scenario, you cherrypick the pre-filtered golden nuggets to bring back to the canonical project.
Precisely. Cherrypick nuggets from poo. It's there if you care to wade into it. May occasionally strike gold.
What's the point of this fork if it's just a landing spot for stuff that's not really wanted? Wouldn't that just be more condescending then telling people what's actually required for a contribution? Besides, the nature of forking is you can just do it yourself anyway. If people love their AI changes they can just make their own fork.
Plus I don't know how you could do this "seamlessly" -- someone has to manage merge conflicts, and as the codebases diverge it's just going to get more and more gnarly. (this is the reason most people don't maintain their own forks in the first place)
Yes - if I can tell that you used AI (except maybe because of an unnaturally high work rate, or obviously an AI declaration, which is good!), you fail. Keep up the quality and I don't care too much.
I have some misgivings about AI, but I'm not a fundamentalist - you can't be or the machine will squish you, frankly - but please, don't spam me with text or code that could be much shorter. Relevant quotes:
"I didn't have time to write a short letter, so I wrote a long one instead"
"Brevity is the soul of wit"
I recently wrote a tool to help me read AI generated PRs and it’s pretty sad that it’s got to this point.
The whole point of not-accepting AI authored code is because this line is not respected=>"Submitters actually provide to-the-point, no-bullshit commits and comments". You're putting way too much faith into the human minds ability to resist clout-chasing. AI isn't able to humanize code without human supervision.
My agents operate on their own branch for a feature, they commit code changes after each step or phase with a description of what was changed, why, and what’s left.
This helps with PR reviews as it prevents a giant wall of text but it’s still verbose. However doing it this way cuts down on the wall of text at the expense of increased PR frequency.
Better way is to provide a Claude.md with strong stylistic guidelines and loc requirements. Else it will be a chicken and mouse game of what is from AI.
I made a PR like that, but it was rejected by the community (for some valid reasons and some not so valid). https://github.com/godotengine/godot/pull/118681
Which goes to show that despite all of the rationalization both here and in the comments of that PR, the push to ban AI is religious not reasoned.
> Submitters just add stylistic markers to make their accounts and output seem human-generated
https://xkcd.com/810/
Not quite accomplished, if it's creating text on the pull requests that looks sufficiently human-like, but you're still worried about the quality of the code and that the submitter doesn't understand it.
No different than a human written PR
Right but as they mentioned, at least then they are communicating with a human about it, not going back and forth with a machine which they clearly do not enjoy.
DoS attack is exactly what I describe an AI generated PR!!
> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
Perhaps reconsider "If your feedback on PRs is just being absorbed by a machine and not going towards mentoring a potential future maintainer..."