Make rate limiting of bot/crawler requests optional

Created on 5 June 2025, about 2 months ago

Problem/Motivation

In the past, when crawler/bot rate limiting was the only module feature, it made sense to disable rate limiter if bot limiting is not configured properly (e.g. due to a typo). Now that module provides several additional features, this behavior is not optimal because error in bot limiting configuration ends up disabling rate limiting of regular traffic and all other features.

Steps to reproduce

  1. Configure rate limiting for bot_traffic. Set either "interval" or "requests" to an invalid value (e.g. negative integer)
  2. Configure rate limiting for regular_traffic. Set both "interval" and "requests" to valid values.

Current behavior: Both bot rate limiting and regular traffic rate limiting are disabled.
Expected behavior: Bot rate limiting is disabled while rate limiting of regular traffic works fine.

Proposed resolution

Handle each type of rate limiting independently.

User interface changes

N/A

API changes

N/A

Data model changes

N/A

📌 Task
Status

Active

Version

3.0

Component

Code

Created by

🇷🇸Serbia vaish

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024