Should use the AI module under the hood.

Created on 10 July 2024, 4 months ago
Updated 12 September 2024, 2 months ago

Problem/Motivation

We should not do the curl calls.
We should use the AI module. It allows you to plug in whatever ai provider to generate images.
Tested with openai.

This way we dont need to support Dalle and stable diffusion, but we support everything the AI module supports

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

✨ Feature request
Status

RTBC

Version

1.0

Component

Code

Created by

πŸ‡§πŸ‡ͺBelgium wouters_f Leuven

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @wouters_f
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven

    I've made a version that uses
    The AI core module (this allows you to switch AI generating models on a global level).
    it's not yet finished.
    the ckeditor plugin is still present, in the config you can choose the
    - default text_to_image provider
    - or choose one (explicitely)

    then when you enter a prompt we'll pass this data on to the AI module
    but I'm not yet sure how to render the image.
    this is what I have now:

    This code does not work.
    Since the use Drupal\ai\Enum\Bundles; can not be found. Work in progress.

     if($provider_name == '000-AI-IMAGE-DEFAULT') {
          $ai_config = \Drupal::service('config.factory')->get('ai.settings');
          $default_providers = $ai_config->get('default_providers') ?? [];
          $ai_provider = $service->createInstance($default_providers['text_to_image']['provider_id']);
          $default_model = $default_providers['text_to_image']['model_id'];
        }else{
          $ai_provider = $service->createInstance($provider_name);
          // TODO if no $default_model how to define this? via the ckeditor admin?
        }
    
        $config = [
          "n" => 1,
          "response_format" => "b64_json",
          "size" => "1792x1024",
          "quality" => "standard",
          "style" => "vivid",
        ];
        $tags = ["tag_1", "tag_2"];
    
        $ai_provider->setConfiguration($config);
        $response = $ai_provider->invokeModelResponse(Bundles::TextToImage, $default_model, $prompt, $tags, TRUE);
        dd($response);
    
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven

    The description should also be adapted.

    STEPS:

    • Install ai module (dependency)
    • Choose default text_to_image model (in ai settings) for example openai or aonther image rendeinr provider.
    • enable ai_image (this) module
    • Add ckeditor plugin to toolbar
    • use the toolbar, and its ckeditor button to generate images with prompt.
  • Status changed to Needs review 4 months ago
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven

    Would be great if someone could test the module and approve it. Thanks!

  • First commit to issue fork.
  • Status changed to RTBC 4 months ago
  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven

    lets do it!

  • πŸ‡ΊπŸ‡¦Ukraine quadrexdev Lutsk

    +1 to RTBC, I just tested it and it worked pretty well!

  • πŸ‡§πŸ‡ͺBelgium wouters_f Leuven

    @elias can you release this?

Production build 0.71.5 2024