Drupal's bot protection is severely broken.

Created on 26 June 2024, 9 months ago

Drupal has a bot protection that triggers in a base firefox with only fingerprinting resistance enabled. What's particularly annoying about this, is that due to the firefox protections against fingerprinting that break the cross-domain scripting that drupal is relying on, not only does the bot detection JS fallback now break (last year it used to at least render) but the "report an issue" feedback is also blocked.

There is also an if-all-else fails prompt to email help@drupal.org - I did try this once on a blocked machine, and got:
connect to smtp4.osuosl.org[140.211.166.137]:25: Connection timed out

Now, enabling fingerprinting in Firefox (or using Chrome) does solve this, but it is not at all obvious from what happens to the website, which is that browsing fails after 2nd page load, with new page loads being restricted to once every half hour or so.

It makes the site completely unusable. Even getting to pages on subjects like sales, security updates, support, basically anything, is impossible.

๐Ÿ› Bug report
Status

Active

Version

3.0

Component

Other

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @Kyber
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States hestenet Portland, OR ๐Ÿ‡บ๐Ÿ‡ธ

    Thank you for the report, @Kyber. This is increasingly important with the change in anti-fingerprinting measures browsers are increasingly taking...

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly apaderno Brescia, ๐Ÿ‡ฎ๐Ÿ‡น
  • While I appreciate the issue being acknowledged, I feel it's worth mentioning to anyone who might drop by here that there has been absolutely no change in the situation.

    The only solution if you need some information on drupal.org these days is to setup a profile with user fingerprinting enabled. There are no other workarounds.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States hestenet Portland, OR ๐Ÿ‡บ๐Ÿ‡ธ

    As an update: I'm actively working with the HumanSecurity/PerimeterX team to try and resolve the specific issue of the captcha being unavailable to manually bypass being caught in the filters. They have been able to reproduce themselves (after some back and forth) so I'm hoping for an update soon.

  • Well, if it's any comfort to you, they break plenty of other websites for me too, I've come to recognise the combo. Zillow.com - same issue, breakage started at same time as drupal.org - only workaround being, again, to use Firefox without resist fingerprinting enabled.

    I do this sort of thing selectively and only if I really need it since the whole "uniquely identify all users" thing online these days really annoys me even if I'm not trying *that* hard to be unidentifiable.

    I normally just take my business elsewhere if a website does this (like zillow), but, I do care quite a lot about drupal since I've been working with it and pushing it as a solution for a couple of decades, and given the website *is* the place to go if there are any security issues I just feel y'all should at least not block access to key pages like your sales and patch pages.

    Anyway, good luck to you, and who knows, maybe this will lead to them not breaking other sites too :)

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States hestenet Portland, OR ๐Ÿ‡บ๐Ÿ‡ธ

    Yup, I totally get it, and I'm a FF diehard as well. There is an irony of fingerprinting be (unfortunately) the best way to identify bot and human spam attackers.

    We had a massive password enumeration attack attempted not long ago - and we used to get 100s of spam accounts created per day, so we're really trying to walk the line between real user friction and general protection of community spaces that we don't have enough staff or volunteers to clean up manually.

    However, that user friction increasing over time because the vendor is not keeping up with changes in the browser/tracking protection landscape is a real problem - so I'm hoping we can make legitimate progress with them. That'd be best for everyone.

  • Well, if people are creating spam accounts and trying to crack passwords that can totally be handled by selectively protecting resources related to account creation and login.

    Static content on your website should probably not have this "protection" unless you are experiencing flooding (something I understand is more and more of a problem nowdays as everyone and their mother is scraping the web for AI - personally I found ByteDance's to be the most annoying since it ignores rate limiting and robots.txt rules).

    But, static content can also be put on CDNs, or reasonable requests per minute per IP put in. The advantage of more nuanced thingies like that is that the website is also still usable in other things people-still-use, like w3m (although, amusingly, and props to them on this at least, the website works just fine in w3m for me โ€” Which is also funny 'cause w3m would look a lot like a bot too (no JS, HTML only fetches) - so honestly, how is putting this in front of static pages helping anything at all).

    But sure, I get it. Y'all are no doubt extremely busy, and tossing the problem over to a company like theirs to handle every resource on the domain instead of on specific resources is easier, so long as they do things in a sensible fashion. โ˜บ

  • ๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand quietone

    I have not been able to login to drupal.org, on Firefox for at least 12 hours. And some point I did use the 'Report a problem' option without error. I also asked in two Slack channels for support but I guess because of the timezone I live I have had only 1 response. That is from cmlara who directed me to this issue. Finally, some idea of what is going on!

    The troubling part of this is that all of drupal.org becomes inaccessible. I agree with @Kyber that some static content should be available and that should include support for people who encounter this.

    I am very tempted to set this to Major.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States hestenet Portland, OR ๐Ÿ‡บ๐Ÿ‡ธ

    I agree with the bump to major.

    There is an option issue in Mozilla's tracker: https://bugzilla.mozilla.org/show_bug.cgi?id=1925236

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States cmlara

    #15 brings up a good points regarding balancing protections vs user experience and adjusting those protections as needed to meet the current situation.

    This fingerprint detection also blocks the Internet Archive ๐Ÿ“Œ Whitelist the Internet Archive so it can archive Drupal documentation Closed: works as designed .

    The first time I saw this service in the request logs it flagged in the incident response plan to ban the host under concern for the amount of data it was pulling.

    I need to run the URLโ€™s through the enterprise lists later to see how they are currently categorized on the enterprise side. I have in the past run in to cases similar to this where a service is dual categorized. In such cases I have seen network security teams decided the keep the block leaving internal teams to either arrange for the protections to be disabled or utilize a different service. May not be the case for this service, however it is at least worth knowing the protections could impact at multiple levels, some the users may not have control over.

    I once had issues with the blocking, making D.O. unusable. I provided D.O. Infra with the request IDโ€™s and they were unable to determine why. This raises questions about D.O. visibility into the service.

    If memory is correct I believe we also had this service block one reasonable sized Drupal event due to the influx of visitors from a shared connection in the event hall.

  • This issue has, if anything, gotten worse.
    Also triggered on login for me now, which used to work most of the time (kind of amusing given the supposed reason for it).

    So, had to open the throwaway browser, and copy and paste the randomly generated password out of the firefox password manager.

    I will note that accounts.drupal.org at least credits who is blocking so horrendously. It is apparently https://www.perimeterx.com (drupal link of https://www.perimeterx.com/whywasiblocked redirects to https://www.humansecurity.com/why-was-i-blocked)

    Which ofc makes no mention anywhere in the long list of reasons that fingerprint protection (NOT javascript blocking, just hiding computer details) might cause blocking as well - they certainly wouldn't want to highlight *that*

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States greggles Denver, Colorado, USA

    Updating title to clarify this affects drupal.org and not the Drupal software more generally.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada andrew.wang

    Not sure if this is related, but I'm starting to see this quite often recently:

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States hestenet Portland, OR ๐Ÿ‡บ๐Ÿ‡ธ

    @andrew.wang - I don't think that is related. Timeouts do happen sometimes, but we're not seeing alerts in our monitors that suggest it should be more frequent. If it's just happening on your dashboard and not on other areas of the site, maybe open a separate issue.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada andrew.wang

    Thanks for the clarification that this issue is regarding the entire site!

    I initially suspected it because itโ€™s weird that it always times out the first time when I visit the dashboard in Firefox (i.e. when Drupal.org is opened in a new tab), but a refresh always fixes it, and it works fine in any other browsers. Anyways, thanks for the information!

Production build 0.71.5 2024