Show a better message (i.e. node content?) for excessive crawling

Created on 23 April 2025, about 21 hours ago

I'm not sure if this defeats the purpose of the module so please feel free to close this if it doesn't make sense.
When loading a page (i.e. shared link) with facets over the threshold, the HTML from the settings is displayed, wondering if it would make sense to redirect to the regular 403 page or use the content of a node that is styled as the site instead?

✨ Feature request
Status

Active

Version

1.0

Component

Code

Created by

πŸ‡ͺπŸ‡ΈSpain pcambra Asturies

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @pcambra
  • πŸ‡ΊπŸ‡ΈUnited States bburg Washington D.C.

    You can modify the responsive message however you like, including creating a version of an access denied page as a basic page, copying that markup, and pasting into the field (be sure to disable aggregation if you do). The default is rather simplistic. The idea was to minimize any processing Drupal needs to do if the site is being overrun by bot traffic. We are taking a similar approach as the Fast404 module. So I'm not sure what a redirect would mean in this context. a generic basic page would still serve as a 200, which would count towards your rate limit on platforms like Acquia.

    I'd be open to ideas, and perhaps this could be configurable. But I do want to keep in line with the intention of serving these sorts of bot requests as cheap as possible for site hosts.

  • πŸ‡ͺπŸ‡ΈSpain pcambra Asturies

    Thanks for the pointer, this makes sense.

Production build 0.71.5 2024