Rate and Usage Limits for API Interactions

Created on 5 December 2024, 13 days ago

This module is great, and I think that a useful feature addition would be to apply rate limits to API interactions.

AI costs can add up quickly. I believe if site builders could limit AI interactions for some users to prevent runaway costs, it may improve adoption of these features. I don't have the coding skills to make such limits, but I hope someone who has the ability likes this suggestion, and considers implementing it into the module! I would be happy to assist testing or any other way I can. I would suggest the following rate limit options to be adjustable for API Interactions:
-----------------------------------------------

Reason:

- Ensure individual users do not use the API too much

How To Apply The Limit:

- Role based - Each role could have restrictions for usage.
- User based - Individual users could also be limited if needed.

Way to Limit:

- Number of interactions / time frame - Ex: 100 interactions per day, Or maybe token based like 200k tokens per day.
- Types of Request / time frame - Ex: Limit to 5 text-to-image requests per day, Or maybe 50 tts requests per month.
- Total usage limit- A maximum usage limit per role or user.
- Combination of the above

Where / How to implement:

- A pair of fields (maybe like a paragraph). One to set the limit, another to set the time frame. Add the ability to add more pairs of fields, just like paragraphs. There could be an admin page for this feature that lists the roles in the system, with an ability to apply the limits from there.
- Add unlimited rate limit options to each entity - Ex: a monthly rate limit, and a daily rate limit could be applied to roles or individual users. Or multiple rate work together to limit the same role or user. This structure would work for many different use cases

Resulting ability for the site admin is to be able to apply multiple limits to different people, as needed. I think this would cover alot of use cases for alot of people:

- Ex for anonymous: Limit to 100k tokens per day, and 2 images per day.
- Ex for authenticated users: Apply a 200k tokens per hour limit, and 100 images per month total limit.
- Ex for different roles as needed: A different user role of "Blog Writer" could have a limit of 500k tokens per hour, 5 mil tokens per month, and 10 images per day.

Feature request
Status

Active

Version

1.0

Component

AI Core module

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024