Graceful handling of rate limits

Created on 25 November 2024, 27 days ago

Problem/Motivation

Different ai provider accounts have different rate limits depending on the tier you are at. As testers we have got the highest tier as we use a lot of AI and want to try out later features but it means things we build sometimes doesn't work well with a newly created account with lower rate limits.

If you have a complex set of agents you send messages to the api as fast as possible. Which isn't THAT fast as you are regularly waiting for the ai to respond.

But still if you hit a rate limit there is nothing you can do about it right now apart from spend more money on a higher tier.

Proposed resolution

- have better error messages informing the end user that they can likely upgrade their rate limit with their provider by paying for a higher tier.

- have a setting in advanced settings that can add a delay in milliseconds to send the prompts out slower.

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

1.0

Component

Code

Created by

🇬🇧United Kingdom yautja_cetanu

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024