Support external Hosts

Regarding the code in the validation, that should be fixed for sure, since we offer it.

The host conifg is mostly there for testing against spoof apis and for the initial users that were using it for Azure OpenAI. However for the later there is a real provider now.

I don't think we want to expose the host in the GUI, because there is so much magic happening when the models are loaded that it would be unusable outside of OpenAI/Azure/OpenAI proxies. So opening up the setting in GUI might just lead to confusion and a lot of issues being created here why a specific provider X can't be used even if its OpenAI compatible. See here: https://git.drupalcode.org/project/ai_provider_openai/-/blob/1.0.x/src/P...

I think the better solution is to create a generic provider for openai compatible providers, without all the magic we add in the OpenAI module to make it easy to use. WDYT?

marcus_johansson → sounds good! I would like to start on this new provider module. What name would you suggest ai_provider_openai_compatible? Do you have some tips for what parts need to be altered to make it general?

Don't ask me about naming, I write function names that are so bad that I can't understand them myself one week later :D No, maybe ai_provider_generic.

The biggest issue is that https://github.com/openai-php/client that is the most mature client for OpenAI doesn't handle any exceptions from OpenAI structure - so for instance in Azure you can use older versions of how things are outputted or in Mistral the error messages are handled different. All this makes for a poor experience with error handling with this client. Both of them gave back error messages related to code instead of what the API endpoint answered.

For Azure provider we had to make our own client for certain models (https://git.drupalcode.org/project/ai_provider_azure/-/tree/1.0.x/src/Cl...), just to handle this better. So I think that will be the biggest issue. A generic client that doesn't give back good error messages, is even worse for a generic provider, since you might need to try configurations back and forth until you get it right.

Also there are OpenAI compatible providers for the chat endpoint, but that has their own list models or embeddings endpoint. Which makes it even harder.

We long thought about taking this general client and extend it more (streaming doesn't work for instance), but currently its just not there.

So I think that would be your largest issue by far. But maybe just start in the simplest way and work from there - just implement an almost copy of OpenAI provider where you just add a lot of configurations for the things you can set in OpenAI::factory method.

If you need help, let me know. It has been on our backlog as well, but its still very far back - we need the "normal" providers to be stable first :)

marcus_johansson → Thank you for the feedback. After going back and forth on the problem I created https://www.drupal.org/project/ai_provider_vllm → instead. Because thats what we actually needed. Support for vLLM.

I ran into the same problem but I think this is a bug because the host variable is already implemented and generally working fine. The problem for us is that the API key check bypasses the host variable and therefore it fails for us (We use our own API keys on our proxy).

This module seems like exactly what we need so I hope we won't have to make a separate `ai_provider_openai_compatible` module that would duplicate all of the work in this project.

I'm attaching a patch that solves the reported problem as well as a near-identical issue with the rate limit checker.

I see that OpenAiProvider has public "::getClient()", which at first uses host setting, at second it uses Drupal http client (for cases when proxy is configured in Drupal.

The provider manager is already injected in the form, so would the example below work?

    $client = $this->aiProviderManager
      ->createInstance(static::PROVIDER_ID)
      ->getClient($api_key);

This issue should be obsolete with #3531134.

Yes - agreed with Jan Kellermann. We (or the community) might setup a module that can create deriatives from the base class, to fit and OpenAI based provider.

At the same time, this will not break anything and could be used for any OpenAI compatible solutions, I have tested it for regression - so this will go into 1.1.x and 1.2.x. Thank you all!