Contrib.social

Error and missing handling on LMStudio Provider

Open on Drupal.org →
Created on 21 October 2024, 3 months ago

Problem/Motivation

I've setup AI with with LMStudio and a local LMM-Service. The connections seems to be fine in the first place, as the different models can be selected via the backend, and I also managed to use it create embeddings into a Vector DB.

But as soon as it comes to chat generation, I receive this strange error message (either by using the admin/config/ai/explorers/chat-generation or by actually adding a chatbot block ) :

TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 51 of /var/www/html/vendor/openai-php/client/src/Responses/Chat/CreateResponse.php).
OpenAI\Responses\Chat\CreateResponse::from(Array, Object) (Line: 35)
OpenAI\Resources\Chat->create(Array) (Line: 225)
Drupal\provider_lmstudio\Plugin\AiProvider\LmStudioProvider->chat(Object, 'multilingual-e5-large', Array)
ReflectionMethod->invokeArgs(Object, Array) (Line: 153)
Drupal\ai\Plugin\ProviderProxy->wrapperCall(Object, Array) (Line: 81)
Drupal\ai\Plugin\ProviderProxy->__call('chat', Array) (Line: 319)
Drupal\ai_api_explorer\Form\ChatGenerationForm->submitForm(Array, Object)
call_user_func_array(Array, Array) (Line: 129)
Drupal\Core\Form\FormSubmitter->executeSubmitHandlers(Array, Object) (Line: 67)
Drupal\Core\Form\FormSubmitter->doSubmitForm(Array, Object) (Line: 597)
Drupal\Core\Form\FormBuilder->processForm('ai_api_chat_generation', Array, Object) (Line: 146)
Drupal\autosave_form\Form\AutosaveFormBuilder->processForm('ai_api_chat_generation', Array, Object) (Line: 326)
Drupal\Core\Form\FormBuilder->buildForm(Object, Object) (Line: 97)
Drupal\autosave_form\Form\AutosaveFormBuilder->buildForm(Object, Object) (Line: 73)
Drupal\Core\Controller\FormController->getContentResult(Object, Object)
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 638)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 181)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 43)
Drupal\nrw_general\Middleware\Redirect->handle(Object, 1, 1) (Line: 53)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 28)
Drupal\Core\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 32)
Drupal\big_pipe\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 36)
Drupal\Core\StackMiddleware\AjaxPageState->handle(Object, 1, 1) (Line: 49)
Drupal\remove_http_headers\StackMiddleware\RemoveHttpHeadersMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 741)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

Steps to reproduce

I'm using the latest Alpha 8 release right now.

Proposed resolution

I could imagine the problem beeing timeout related, as the the local network service takes quite a while to answer.
I've tried to raise the timeouts in LmStudioControlApi.php, which did not work yet.

   $options['connect_timeout'] = 120;
    $options['read_timeout'] = 120;
    $options['timeout'] = 120;

However, having a backend config form to change these would be a good idea anyway, maybe.
Any ideas according to the issue and the debugging/ catching the exception?

🐛 Bug report
Status

Active

Version

1.0

Component

Providers

Created by

🇩🇪Germany netv

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @netv
  • Comment 3 months ago →
    🇩🇪Germany netv
  • Comment 3 months ago →
    🇩🇪Germany marcus_johansson

    So this has the similar problem as this: https://www.drupal.org/project/ai/issues/3460283 ✨ Build custom client for Mistral or switch to HelgeSverre lib Active

    The error codes from LMStudio are not OpenAI compliant, and because of this the client fails without giving back a human readable error. The one above could mean that anything went wrong with the connection, the payload or with LMStudio.

    The good news is that we got sponsorship to do the Azure client that in certain circumstances - this is already done here https://www.drupal.org/project/ai_provider_azure → . What is missing is streaming capabilities, but as soon as this is done we could move it to LMStudio and Mistral.

  • Comment about 1 month ago →
    🇫🇷France nicolasgoudard

    Same problem with MISTRAL AI : mistral-large-latest model
    When I go to translate a content in the translate tab, then click on Translate with mistral-large-latest

    #0 /home/ism2/public_html/drupal/vendor/openai-php/client/src/Responses/Chat/CreateResponse.php(50): array_map()
    #1 /home/ism2/public_html/drupal/vendor/openai-php/client/src/Resources/Chat.php(35): OpenAI\Responses\Chat\CreateResponse::from()
    #2 /home/ism2/public_html/drupal/web/modules/ai_provider_mistral/src/Plugin/AiProvider/MistralProvider.php(211): OpenAI\Resources\Chat->create()
    #3 [internal function]: Drupal\ai_provider_mistral\Plugin\AiProvider\MistralProvider->chat()
    #4 /home/ism2/public_html/drupal/web/modules/contrib/ai/src/Plugin/ProviderProxy.php(155): ReflectionMethod->invokeArgs()
    #5 /home/ism2/public_html/drupal/web/modules/contrib/ai/src/Plugin/ProviderProxy.php(81): Drupal\ai\Plugin\ProviderProxy->wrapperCall()
    #6 /home/ism2/public_html/drupal/web/modules/contrib/ai/modules/ai_translate/src/Controller/AiTranslateController.php(183): Drupal\ai\Plugin\ProviderProxy->__call()
    #7 /home/ism2/public_html/drupal/web/modules/contrib/ai/modules/ai_translate/src/Controller/AiTranslateController.php(261): Drupal\ai_translate\Controller\AiTranslateController->translateContent()
    #8 /home/ism2/public_html/drupal/web/core/includes/batch.inc(296): Drupal\ai_translate\Controller\AiTranslateController->translateSingleText()
    #9 /home/ism2/public_html/drupal/web/core/includes/batch.inc(138): _batch_process()
    #10 /home/ism2/public_html/drupal/web/core/includes/batch.inc(94): _batch_do()
    #11 /home/ism2/public_html/drupal/web/core/modules/system/src/Controller/BatchController.php(52): _batch_page()
    #12 [internal function]: Drupal\system\Controller\BatchController->batchPage()
    #13 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array()
    #14 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/Render/Renderer.php(638): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
    #15 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext()
    #16 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext()
    #17 /home/ism2/public_html/drupal/vendor/symfony/http-kernel/HttpKernel.php(181): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
    #18 /home/ism2/public_html/drupal/vendor/symfony/http-kernel/HttpKernel.php(76): Symfony\Component\HttpKernel\HttpKernel->handleRaw()
    #19 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/Session.php(53): Symfony\Component\HttpKernel\HttpKernel->handle()
    #20 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle()
    #21 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/ContentLength.php(28): Drupal\Core\StackMiddleware\KernelPreHandle->handle()
    #22 /home/ism2/public_html/drupal/web/core/modules/big_pipe/src/StackMiddleware/ContentLength.php(32): Drupal\Core\StackMiddleware\ContentLength->handle()
    #23 /home/ism2/public_html/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\big_pipe\StackMiddleware\ContentLength->handle()
    #24 /home/ism2/public_html/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass()
    #25 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle()
    #26 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle()
    #27 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/AjaxPageState.php(36): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle()
    #28 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\AjaxPageState->handle()
    #29 /home/ism2/public_html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(741): Drupal\Core\StackMiddleware\StackedHttpKernel->handle()
    #30 /home/ism2/public_html/drupal/web/index.php(19): Drupal\Core\DrupalKernel->handle()
    #31 {main}

  • Comment 4 days ago →
    🇩🇪Germany marcus_johansson
contrib.social Blog FAQ Discussions
Production build 0.71.5 2024