Allow configuration of paths to ignore

Created on 20 April 2024, 2 months ago
Updated 23 April 2024, 2 months ago

Problem/Motivation

I use flysystem S3 which has a custom path for the initial creation of an image style. I thinks S3FS has the same functionality too. There are other modules out there that have paths that are hit that are not static files on the file system as well and may be called as a library or several times on the page, as in the case of the image style urls. We also would run into this for some of our headless applications as well as one call to the page may be several calls to drupal as a backend.

Steps to reproduce

Proposed resolution

Lets make this logic configurable:
https://git.drupalcode.org/project/crawler_rate_limit/-/blob/62a59866ef8...

✨ Feature request
Status

Active

Version

3.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States generalredneck

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @generalredneck
  • πŸ‡·πŸ‡ΈSerbia vaish

    This would be great improvement. Are you planning on creating a MR? Note that issue ✨ [Patch provided] Allow administrators to specify IP addresses and ranges which should never be rate limited Needs review provides a setting for ignoring IP addresses. We should make sure that the structure of the settings array and naming of the array keys remains consistent and intuitive after both settings have been added. E.g. both setting should use the same verb, either "ignore" or "allow", etc.

  • πŸ‡ΊπŸ‡ΈUnited States generalredneck

    Not immediately. I may at some point in the future but I'm unable to secure a client to develop this against right away.

    I was mostly poling around the code to see how stuff worked when I saw the issue...

    At the very least we should have a hook here... that one configuration where you derive the path for styles is a use case for having it as a hook, but all the other paths are simple hard coded instances which lends themselves to configuration like you mentioned.

  • πŸ‡·πŸ‡ΈSerbia vaish

    I would rather try to avoid invoking any hooks in this module. In order for the module to be really useful, it should be able to process each request as early and as fast as possible. Invoking hook adds an overhead which is avoidable. For the same reason I decided not to use Drupal config and create regular configuration page.

    Thanks for the feedback and let me know if you happen to decide to work on this.

  • πŸ‡ΊπŸ‡ΈUnited States generalredneck

    Those are all good feedback. I believe it might be worth adding to a contribution section of the Readme.

    Thanks for framing the ideas... it's a lot like the way fast404 works

Production build 0.69.0 2024