- Issue created by @valthebald
- 🇬🇧United Kingdom yautja_cetanu
We probably want a separation of errors shown to end-users verses admin logs. We want the end-user errors to be something generic such as
"An error has occurred, contact the site admin for more information"
Then in the Drupal logs, I think say very specifically that it is due to the quota being used up.
- 🇱🇹Lithuania mindaugasd
Related with issue on openai module 🐛 Misleading error: You exceeded your current quota, please check your plan and billing details Active
In that case I think openai service was returning the error wrong - Assigned to Marcus_Johansson
- 🇩🇪Germany Marcus_Johansson
@valthebald - So I think there will be some separation here - the more specific exception the better.
I have implemented this in Anthropic provider first, since I didn't have any money on my private account:
The underlying clients have no context if this is the error, so in this I have done error message pattern recognition. This is of course not a stable solution, but as written above it would fail back gracefully to the generic ai response error.
I will try to add these for the providers where you can discern it from the client/error message.
But for the disabling of providers if they are over their quota, I would suggest that we create a follow up ticket that would create a submodule for this. Not everyone would want that logic and the event system makes it very easy for another module to handle such logic.
@yautja_cetanu - This is controlled via $config['system.logging']['error_level'] configuration, if you set it to very low level (on a production) system, you logic entails. On a local development machine its great to see full stack traces, exception types and messages.
-
Marcus_Johansson →
committed 99919f36 on ratelimit-quota-exceptions
Issue #3459836 by Marcus_Johansson: Introduce AiQuotaException
-
Marcus_Johansson →
committed 99919f36 on ratelimit-quota-exceptions
- 🇩🇪Germany Marcus_Johansson
@valthebald - If its ok to move out the logic of any provider that is seeing a lot of exceptions to some external module, this would be done and testable in the branch: ratelimit-quota-exceptions (https://git.drupalcode.org/project/ai/-/tree/ratelimit-quota-exceptions?...)
This also fixes another exception: https://www.drupal.org/project/ai/issues/3460274 📌 Introduce AIRateLimitException Active
The quota one works for Anthropic and OpenAI, while the rate limit currently works for Groq, OpenAi, HuggingFace.
For Mistral I added this ticket that needs to be done first: https://www.drupal.org/project/ai/issues/3460283 ✨ Build custom client for Mistral or switch to HelgeSverre lib Active
For Ollama and LM Studio these exceptions shouldn't occur. Both have queuing systems on multiple requests.
- Status changed to Needs review
2 months ago 5:52pm 9 July 2024 - 🇧🇬Bulgaria valthebald Sofia
@Marcus_Johansson - what do you mean external module? As in having it's own composer package, or submodule of drupal/ai?
- 🇩🇪Germany Marcus_Johansson
@valthebald - I think that would be up for discussion, having that available to enable when you download the AI module probably makes sense I would think? Assuming someone has the time to work on it.
I just want to keep the core abstraction module as lean as possible so it basically does input to output normalization and any logic that needs to happen before, during normalization or after a request is up to the providers or external modules to handle.
- 🇧🇬Bulgaria valthebald Sofia
I'd put the logic of when to throw the new exceptions (quota/rate) into providers, the core handling them in the unified way
- 🇩🇪Germany Marcus_Johansson
Ah, missed the response here. Right - the core would handle those in a unified with the change I made and according to how you write in the last comment.
The thing that is "missing" from your initial description is that it doesn't take care of flagging/disabling an provider. This would have to go into some 3rd party contrib module or a submodule to the AI module.
Pull request here: https://git.drupalcode.org/project/ai/-/merge_requests/27?resolved_confl...
- 🇧🇬Bulgaria valthebald Sofia
@Marcus_Johansson: I've added a few comments, none of them are show stoppers
-
Marcus_Johansson →
committed 8ddde811 on ratelimit-quota-exceptions
Issue #3459836 by Marcus_Johansson: Introduce AiQuotaException
-
Marcus_Johansson →
committed 8ddde811 on ratelimit-quota-exceptions
- Status changed to Fixed
about 1 month ago 7:13am 25 July 2024 - 🇩🇪Germany Marcus_Johansson
Thanks @valthebald - they are fixed together with some other phpcs complaints. I will merge it into 1.0.x-dev now.
Automatically closed - issue fixed for 2 weeks with no activity.