Contrib.social

Get rate limit information normalized

Open on Drupal.org →
Created on 24 September 2025, 4 months ago

Problem/Motivation

Some of the providers support returning limit information in the response of for example chat requests.

See for example:
https://platform.openai.com/docs/guides/rate-limits#rate-limits-in-headers
https://docs.claude.com/en/api/rate-limits#response-headers

For the OpenAI based clients, ince we inject a Guzzle client by default, we should be able to use a Middleware to collect the header data and use it.

We should also add this to our OpenAI default provider with a default open ai solution, and then add a method that providers can use to set this, depending on headers used.

We will start with none-streamed Chat.

Remaining tasks

  • Create a DTO called ChatProviderLimitsDto.
  • Use the DtoBaseMethodsTrait
  • Add a constructor to the ChatProviderLimitsDto with RateLimitMaxRequests (int), RateLimitMaxTokens (int), RateLimitRemaingRequests (int), RateLimitRemainingTokens (int), RateLimitResetRequests (int seconds), RateLimitResetTokens (int seconds). All optional and nullable.
  • In the ChatOutput add a getter and setter for RateLimits, like getRateLimits and setRateLimits that gets and sets the ProviderLimitsDto. Instantiate with a ChatProviderLimitsDto, with all values nulled.
  • In the OpenAiBasedProviderClientBase add a Middleware for Guzzle that takes the response and writes the headers to a variable.
  • In the OpenAiBasedProviderClientBase create a method called setChatRateLimits, that maps the variables to a constructed ChatProviderLimitsDto object. The method returns the object or null. Assume OpenAI, but do isset around each key, so it works with none OpenAI providers.
  • In the OpenAiBasedProviderClientBase after the none-streamed response invoke this method and look if an object was given back and set this on the ChatOutput.
✨ Feature request
Status

Active

Version

1.2

Component

AI Core module

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024