Failover provider module

Created on 11 July 2025, about 1 month ago

Problem/Motivation

OpenAI goes down at least for 1 hour per month - Anthropic is even worse. We have the possibility to change the provider in the pre request event. We have specific exceptions for failed API calls, rate limit errors etc. If you want stability you need a failover.

When you run a MySQL server and need good failover, you have a primiary and a secondary (and even more in other srever halls, on real Enterprise setups). We might need the same for AI.

Proposed resolution

Create a module that recognizes when a provider has gone down or is having problems and that starts pinging this on cron jobs.
While its down it should forward all requests to a secondary provider.

🌱 Plan
Status

Active

Component

Ideas

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024