should use the AI module under the hood

Created on 26 July 2024, 4 months ago

Problem/Motivation

This module makes the ai calls directly to the Ai providers,
But more than only these providers exist. (google vision / AWS / ...)

There is a module that does these calls for you:
https://www.drupal.org/project/ai

ideally you would connect your module to the ai module and they would allow you to switch to other AI providers seamlessly.
(it is to AI what search_api does for search).

Proposed resolution

Other modules have already connected to the AI module too.

Example LLM call via AI module

    // Find the default selected LLM
    $sets = \Drupal::service('ai.provider')->getDefaultProviderForOperationType('chat');

    $service = \Drupal::service('ai.provider');
    $provider = $service->createInstance($sets['provider_id']);
    $messages = new ChatInput([
      new chatMessage('system', 'You are helpful assistant.'),
      new chatMessage('user', $input),
    ]);
    $message = $provider->chat($messages, $sets['model_id'])->getNormalized();
    return $message->getText();

Example image generation call via AI module

   $config = [
      "n" => 1,
      "response_format" => "url",
      "size" => "1024x1024",
      "quality" => "standard",
      "style" => "vivid",
    ];
    $tags = ["tag_1", "tag_2"];
    try {
      $ai_provider->setConfiguration($config);
      $input = new TextToImageInput($prompt);
      $response = $ai_provider->textToImage($input_prompt, $default_model, $tags);
      $url = $this->saveAndGetImageUrl($response);

I dont know how to give an image to the AI module to receive text, but I know it's possible since the Chat explorer does this in the ai module itself.
So it's certainly possible.

Remaining tasks

1 Check if providers for Microsoft Azure Cognitive Services API or Alttext.ai exist in Ai module (or create them)
2 Remove the calls to the providers directly from this module
3 Add a way to choose the model from the AI module (or go with the default model from the ai defaults)

User interface changes

A settings form somewhere to allow you to choose a model for the alt tags for a field.

API changes

Use the AI module & it's provider architecture under the hood.

Feature request
Status

Active

Version

2.0

Component

Code

Created by

🇧🇪Belgium wouters_f Leuven

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024