How do I use this module to block bots but not humans?

Created on 7 May 2025, 3 days ago

Problem/Motivation

I didn't understand how the "Facet parameter limit" worked. Was the module keeping track of how many times I used facets? Or was it keeping track of how many facets I was trying to use in a given request? (Turns out it is the latter.)

Here's a screencast demo of my evaluation of the module.

At the end of my Loom I do wonder if I'm misunderstanding how this module is supposed to protect a website from excessive facet traversal while not blocking users who are legitimately trying to use the website's features. I'd love to get some feedback about my understanding.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

πŸ’¬ Support request
Status

Active

Version

1.0

Component

Documentation

Created by

πŸ‡ΊπŸ‡ΈUnited States capellic Austin, Texas

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @capellic
  • πŸ‡ΊπŸ‡ΈUnited States bburg Washington D.C.

    Thanks for reaching out. The short answer is there isn't a way to distinguish bots from humans. At least the ones this module intends to block. The only real way is to pick a facet "limit" that seems higher than most real humans would likely use. While looking at log data on this topic, I've seen requests using 20+ facet filters. I'm certain that was a bot stuck in a faceted search page, but otherwise, the recent trend we are seeing is these bots do not identify themselves as bots in their user agent string in the http header. They are also using browsers capable of running javascript, so we can't just load these links asynchronously.

    Without a lot of options, the approach in this module is what I came up with. Companies are coming out with tools to deal with the issue, Like Cloudflare Labrynth. But, this module caters to websites who don't have a WAF like tool available to them. It is a simple approach, and it's been a huge help to the sites I've installed it on, who were at risk of exceeding their rate limits.

  • πŸ‡ΊπŸ‡ΈUnited States capellic Austin, Texas

    Thanks for the speedy response @bburg.

    This module makes a lot of sense for websites 20 facets (and therefore 20x the headache!). I don't see it making sense for our websites where he user journey naturally maxes out at 3 facets.

    We've been upgrading our websites to Facets 3, so I think this is our next stop to sort this out for websites that don't have a WAF and for when the bots circumvent the WAF config to then mitigate their impact. I appreciate you posting about Facets 3 on the project page.

Production build 0.71.5 2024