Does User-Agent string mean different things?

Created on 14 October 2025, 5 days ago
Updated 16 October 2025, 3 days ago

Problem/Motivation

These are the regular_traffic and bot_traffic options:

/**
* Limit for crawler / bot traffic (visitors that openly identify as
* crawlers / bots). Optional. Omit to disable.
*
* Note: If this section is omitted (undefined), bot traffic will be treated
* in the same way as regular traffic.
*/
$settings['crawler_rate_limit.settings']['bot_traffic'] = [
[...]
// Number of requests allowed in the given time interval per crawler or
// bot (identified by User-Agent string). Must be a whole number greater
// than zero.
[...]

/**
* Limits for regular website traffic (visitors that don't openly identify
* as crawlers / bots). Optional. Omit to disable.
*
* Visitor-level (IP address + User-Agent string) regular traffic rate
* limit.
*/
$settings['crawler_rate_limit.settings']['regular_traffic'] = [
[...]
// Number of requests allowed in the given time interval per regular
// visitor (identified by combination of IP address + User-Agent string).
[...]

Setting it up, I wasn't sure what is meant by "User-Agent string" ... is it correctly understood, that they mean different things, under each option?:

  • bot_traffic: A bot openly self-identifying as such, with ClaudeBot in its User-Agent string.
  • regular_traffic: The unique User-Agent string string, like Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0

Steps to reproduce

Proposed resolution

If it is correctly understood, it could be emphasized in the README, updating them to something like this (adding "ClaudeBot" and "uniqueness")?

bot_traffic:

// Number of requests allowed in the given time interval per crawler or
// bot (identified by bot name in User-Agent string, i.e. "ClaudeBot").

regular_traffic:

// Number of requests allowed in the given time interval per regular
// visitor (identified by combination of IP address + User-Agent string uniqueness).

Remaining tasks

User interface changes

API changes

Data model changes

💬 Support request
Status

Active

Version

3.0

Component

Documentation

Created by

🇩🇰Denmark ressa Copenhagen

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024