Rate-limit by ASN

Notes from testing this MR against real traffic for about two weeks now:

This is currently performing very well / as designed against real traffic. The addition of the ASN lookup on those requests where it is actually necessary (requester not openly identifying as bot and not already blocked at the visitor-level) is so fast, even without the optional C extension, that it is imperceptible to our end-users. On the other hand, rate-limiting all crawlers, including those that horizontally spread out across multiple IPs and/or user agent strings, is a perceptible performance boost for our end-users.

This looks good, although I'm getting this message now:
Missing dependencies: In order to rate-limit regular traffic at the ASN-level, you need to install the GeoIP2 PHP API.
I did download the a test ASN database, GeoLite2-ASN-Test.mmdb,

Full message is:

CRAWLER RATE LIMIT Enabled

• Configured to use memcached backend.
• Rate limiting bot/crawler requests at the bot/crawler-level. 100 requests allowed per bot/crawler over a 600-second interval.
• Rate limiting regular traffic requests at the visitor-level. 200 requests allowed per visitor over a 1400-second interval.
• Rate limiting regular traffic requests at the ASN-level. 600 requests allowed per ASN over a 600-second interval.
• Issue(s) detected that prevent rate limiter from functioning. In order to prevent fatal errors rate limiting has been disabled. You must fix all the errors or disable the Crawler Rate Limit.
• Missing dependencies: In order to rate-limit regular traffic at the ASN-level, you need to install the GeoIP2 PHP API.

Darell, there are two steps you need to complete before being able to use ASN-level rate limiting. You did download the ASN database already. What's left is to install PHP package geoip2/geoip2. That's what error message you got is about. You can install this package via composer, as usual.

composer require geoip2/geoip2

Please, let me know if you run into any other issues with this feature. I'm about to merge this MR but feel free to open a follow up issue if you find any bugs.

Thanks @chrisolof. Everything works great. I just made few minor tweaks.

That makes sense, thank you. I'll give it another try from the dev branch.

Automatically closed - issue fixed for 2 weeks with no activity.