Bulk LLM-generated module publishing by bigbabert

Note: I am in both the linked Slack threads, I am one of the individual who first hand saw significant flaws in minutes of glancing at the code. I am in full agreement that Advanced File Destination is unsuitable for install on any site in its current condition. I agree that other commenters made points that raise reasonable questions regarding Static Node being functional for the described purpose.

Adding a bit of complication to this discussion:

The individual has published a few other AI written modules in the past, I make no comment at this time in regards to the quality of the code related to the modules in discussion here.

Projects with Stable and Security Covered releases:
https://www.drupal.org/project/auto_translation → - 290 reported installs.
https://www.drupal.org/project/graph_element → - 10 reported installs.
https://www.drupal.org/project/jw_player_media_source → - 6 reported installs

Projects enrolled in security opt in with no stable release:
https://www.drupal.org/project/content_reporting → - 42 reported installs

I think that revoking git access and unpublishing the projects temporarily might be warranted

I certainly understand the desire to stop the publication of significantly buggy code before site owners install it and to avoid new code from being published.

I am however a bit concerned that this could lead to a repeat of community disruption that was discussed in #3094854: Plan for dealing with projects that may be abandoned → .

Is there a plan to minimize the impact? Thankfully these are smaller install count modules compared to the previous incident, however that does not help the unlucky site owner who is using the module and finds the maintainers unable to commit code.

There is no problem with publishing modules that are not functional as starting points for community contribution. I believe that is what sandbox projects are for. We used to require all projects to start in a sandbox and go through review before becoming a regular project. Now that AI has made it easier for people to create projects we should go back to that process.

There is no problem with publishing modules that are not functional as starting points for community contribution.

It's quite possible to publish a full project with only a dev or alpha release, these have stable releases and are opted into security coverage already.

There is nothing that stops a maintainer to directly tag a stable release without first creating a pre-release.

Creating a sandbox project could help to avoid such cases, but sandbox projects are going to be deprecated. That workflow worked for the first project, though; once somebody would have gotten the permission to create full projects, that person could create full projects directly.

I propose we reconsider deprecating sandbox projects in light of the use of AI to abuse the system.

• Non-vetted contributors would only be allowed to create sandbox projects.
• A sandbox project could only be promoted to a full project by a vetted contributor.
• A contributor who released a project without reviewing it would lose vetted status for a year.

@darren oh, that may be worth another issue.

However, it is again not relevant to this particular thread because bigabert is what I would consider vetted, he has the ability to opt projects into security coverage which he did for some of the projects discussed.

Sandbox would not change anything about this case because once you have that permission you can promote and opt in.

Non-vetted contributors would only be allowed to create sandbox projects.

In this case, the projects are from a person using a vetted account. (On https://www.drupal.org/user/3591390 → , I read Can opt projects into security advisory coverage.)
What proposed would not help in this case, which is the reason to propose that in a different issue. (I am not sure the Drupal Association would agree on keeping sandboxes, but I could see a use for sandbox projects, even if that would mean making a step back on what decided about what non-vetted accounts can do.)

In case anyone else was confused, I proposed that vetted contributors could lose their vetted status.

Thanks for taking this so serious, and with a very critical approach.

In hindsight (always 20/20) it would have been great if the discussion had been copied from Slack, and pasted into this issue instead of linking to it, since not everyone is on Slack.

I am linking to the slack discussions [...] note that he attempted to delete this thread [...]

I noticed today that the user is seeking guidance on how to contribute properly on slack.
https://drupal.slack.com/archives/C1BMUQ9U6/p1745611803988939

Alberto Cocchiara
:palm_tree: 10:10 PM
hi all, I'm a Solution Architect working on Global project in the past 12 years, some of this are on Drupal where i moved my first step in Enterprise CMS practice, nowdays i work on Adobe AEM mostly but i want to give back something to Drupal community, that i'm part from almost 7 years, so i'm looking for a mentor to be guided to properly contribute in an effective way, thanks to anyone can point me or become my mentor :good-luck:

Agreed this situation is very troubling. Thanks for opening this issue and considering what to do about it.

Not sure how folks feel about having their Slack transcripts posted publicly. I've got txt versions stashed locally, could upload if desired / needed / agreed.

Personally, I found the "hey, why don't you give me some free mentoring and tell me what's wrong with my code if you don't like it?" attitude very disturbing. People who objected were framed as the villains, squelching an enthusiastic contributor and being unwelcoming as a community. I believe they were rightfully objecting to AI-generated slop being posted, then asking the community to review it in ways the "author" couldn't be bothered (or wasn't able) to do themselves.

Our time is not free. Just because you're posting it as open source code doesn't obligate anyone to "mentor" you. If you want to hire someone to do a security audit on your code, go ahead. But don't post slop garbage as your "contribution", then expect experts in the community to teach you everything the LLM got wrong for free, as if you're entitled to being their apprentice and learning from them solely because you're making the code available under a GPL license.

About "Not sure how folks feel about having their Slack transcripts posted publicly." ...

I do understand this feeling, but Slack should not be thought of as a private forum, since anyone can create an account and read along, or copy-paste the content elsewhere. People posting comments in Slack should consider it the same as posting here, in the Drupal issue queues.

Linking to a Slack discussion, which then gets deleted by the user (as in this case), or Slack itself after a while, has allowed the context to be scrubbed, showing a weakness of relying too much on Slack, and using it as a repository for information.

Not everyone has access to Slack, and the information can vanish.

I am also not sure if the conversation needs to get shared here, but at least having it, in case it's useful for evaluation later on, in a closed group, is useful.

Hi everyone,

I've read through the discussion and wanted to clarify a few things. Like many of you, I contribute to Drupal in my free time, on a completely voluntary basis. The modules I’ve released as contrib projects are based on real-world use cases I’ve encountered in my professional experience, with the goal of giving back to the community by covering gaps or providing helpful tools.

I understand that some of my recent actions—such as adding duplicate screenshots or posting unclear comments—might have caused confusion or been perceived as unhelpful. That was not my intention, and I apologize if it disrupted the issue queue or moderation efforts.

I truly value the Drupal community and respect the work that moderators and contributors do. I will be more mindful moving forward and make sure my input is as constructive and relevant as possible.

Thank you for the feedback and for all your efforts in maintaining a healthy and productive environment on Drupal.org.

It seems there are two things going on here:

- One is there is a very real question that needs to be addressed quickly of how to handle vibe coded modules of poor quality.
- Then there is a question of handling this particular individual.

Re: the first issue:

Obviously I am a strong user and advocate for AI and AI in coding. However, I've seen first hand the kinds of mistakes AI can make that are quite difficult to detect in some automated way. (Necessarily because if it is, AI is quite good at reviewing them quickly and fixing it). Security issues and architectural issues of code not using Drupal correctly are going to become an issue. If we allow this to go unchecked without clear rules the sheer volume may become difficult to handle.

It does seem there should be policies made and this can be a case study on what the policy should be for example:

• We should probably require people who write code or commits or patches to say if they have use AI to create it, how they have used AI and how much. This will be difficult to enforce but then it would justify "going through slack messages"
• We should have some policy for how we go about going from Beta to Release and the level of care we expect from people. That is perhaps stronger than before given how easy it is to vibe code something.
• There has been clear cut evidence of people using AI to write comments on issues to gain credit and this should be addressed. Instead of spam there could be something to do with "Low effort" comments
• There should probably be some way of warning individuals and maybe companies that engage in this behaviour rather than immediately punishing people

Re: the way this thing is handled.

Struggling to know how to reply to this in light of Newtoid's comment which I fully agree with. There is something about the tone of many comments that seem targetted and personal.

I think its because behind the comments that seem "mean", there are real actionable important things that I think are important to turn into rules that people can follows and actions people can take to avoid punishment.

I think I would second Darren Oh's list but have something where we make it explicit a level of human effort expected from people (To address dww's comment). It seems like we need some level of expectation when someone has privalledges that their own human efforts are judged and held accountable.

Disclaimer: The person in question reached out to me and asked for my comment. I don't know them, nor have I looked at the modules specifically. I don't have a strong view of whether or not they should lose their vetting status but just think that the issue is important that the reasons why and how to avoid it (including this person) should be clear.

Hi everyone,

This is bigbabert again. I’d like to respond directly to the recent discussion stopped by moderators on slack then switched here without any notification to me, My goal is clarify my intentions and advocating for a more constructive dialogue.

The two modules I published — Static Node (it not have stable release yet) and Advanced File Destination — were developed based on real-world needs I encountered during client projects. I chose to share them with the community because I believe in giving back to Drupal. About older module like auto_translation or moderation_scheduler they not started their life when AI exist yet and i assume have been already verified by Security team an not only by me. Plus same people opened this issue opened also security issues that at the end are not really security issue but functional issues.

It’s true that I used AI-based tools to help speed up some parts of the development. I don’t hide that. I also carefully reviewed and tested everything before publishing — including writing documentation and ensuring the code adhered to Drupal standards as best as possible. Most importantly, I explicitly asked for mentorship to ensure my contributions could evolve in line with community expectations. Unfortunately, that request was ignored or dismissed with irony.

I’m always open to constructive feedback and improvement — that's what open source is about. But labeling my work as "spammy", suggesting revoking access, or removing projects without any dialogue is neither fair nor inclusive.

We can — and should — do better than that.

Love and peace
Best regards
Alberto

There is something about the tone of many comments that seem targetted and personal.

I opened this issue because the original promotion of these modules in slack led to two long discussions that reached over 200 comments. I was not heavily involved in the discussions because most of comments happened while I was offline, but I did read most of them in the end. This and other things prompted me to open the issue.

The reports to the security team mentioned above were primarily responded two in two ways. I am quoting minimally from the security issues in order to avoid breaking the security team disclosure policy, but I've also asked for permission to post longer excerpts.

1. By replying with vibe coded patches on s.d.o that wouldn't fix the vulnerability and would potentially introduce others. catch: "Was it also 'vibe coded'?" @bigbabert "Yes it was".

2. During the same week, two reports were sent by @bigbabert to the security team for modules authored by the same people who had filed the security issues for the modules discussed here (neither of these people were me). At least one of those reports included the phrase 'Above an analysis done on the module using AI'. Despite repeated questions, no valid steps to reproduce a vulnerability were provided. The fact that both reports were against modules written by people who had filed security reports about his own modules does not seem coincidental.

The common theme here is not specifically that code, security reports and etc. have been written with LLMs, although that is a factor, because it makes it much easier to do this at scale. It's that this is happening without any (apparent) human verification before sending the machine generated content into either stable, security opted-in projects on Drupal.org or reports to the security team. Drupal contributors acting in good faith often report low-severity or hard to reproduce issues to the security team 'just in case', and they often end up as public bug reports in the issue queue, or sometimes people are just mistaken, but this is usually done with that caveat stated fairly clearly from the outset. We do also get lots of other low quality security reports, like tens of pages of PDF generated by automated scanners with no verification, but I don't think I've ever seen two of these from the same person about two different modules in the same week.

Opting a module into security coverage includes the disclaimer "I will cooperate with the Drupal Security Team as needed.". While @bigbabert has co-operated in the sense of replying on issues, the manner in which they're doing so constitutes malicious compliance for me. This has taken hours of security team time that could have been spent on something else.

In the past month, I have only seen one other case of obviously 'vibe-coded' Drupal code - this was in two MRs adding new features to the same module - they contained thousands of lines of code that would never have worked, and were also clearly lacking any human review. However, MRs are not stable projects on d.o opted into security coverage, and neither are they reports to the security team that have to be triaged by a small group of people, so I didn't feel any need to open an site moderators issue in that case. If that makes this issue 'personal and targeted' then I guess we can be thankful that so far not many people are engaging in this behaviour yet.

@Catch, Should we add something explicit then:

"I will cooperate with the Drupal Security Team as needed"

• By Opting into the Security Team you and your maintainers are taking accountability of the code submitted whether it is from third parties in the issues queues or created with AI. All code submitted under security review requires human oversight and you can confirm that you, or another human has been through the code and considered the security implications.
• Where AI has assisted the creation of the code you are explicit on the degree to which AI has written the code and a general process of how you have reviewed or worked with AI

I don't know how easy it will be to enforce. But I do wonder if, its ok to submit a MR, purely vibe coded, but we have something that clearly says it has somewhere. But before its commtted to a branch that will be under securtiy reveal and released, we have a policy that people need to have had human review of the code?

Due to the well known sorry state of documentation and the fact that LLMs can only work from what they ingest it follows that LLMs are typically not capable of good -- or even working -- Drupal code. (Despite this, a few people can wield LLMs successfully to write Drupal code but they are typically exceptionally well versed in writing Drupal code themselves.) If we add the, once again, well known long list of harms LLMs inflict on our society as a whole and the typical inclination of open source developers to do, in a very broad and vague sense, good it is very easy to see why introducing entirely LLM written modules into the Drupal ecosystem was met with significant resistance and disdain. Likely getting emotional over this was a mistake but it's really hard to separate a single act from the widespread wanton destruction wrought by LLMs. The reaction to this was not taking a step back and asking what is our problem but laughing at our concerns which really did not help elevating the tone of the discussion. This retaliatory behavior repeated when, as catch mentions, "two reports were sent by @bigbabert to the security team for modules authored by the same people".

But labeling my work as "spammy", suggesting revoking access, or removing projects without any dialogue is neither fair nor inclusive.

It's exceptionally hard to find inclusive thoughts after all this. Luckily, I am no longer a maintainer in any capacity and the deterioration of the tone of the discussion was partially my fault without a doubt but still, I wanted to give some context why that happened.

Considering punishment has already been announced in 🐛 Clarify policy for revoking security advisory coverage Active is there anything left for this issue to discuss?

All the remaining items I see appear to be "what is the actual policy" and "what changes to the policy do we want to make to retroactively allow the taken action".