function ai(), like t() that calls the default LLM

Object oriented code can be quite extensive and flexible:

$ai = $provider->textToImage('A cow with earrings');
$entity->set('field_ref_media', $ai->getAsMediaReference());
$entity->set('field_ref_file', $ai->getAsFileReference());

$ai = $provider->chat($messages);
$entity->set('body', $ai->getResponseAsString()); // ?

// etc. etc.

While ai() function could only have few limited uses.
When reading the code, it is not exactly clear what values can it accept or what response can it provide, unless you memorized it.
While object oriented code I demonstrated above can be flexible and self understood.

However, how does ai() look with objects today?

Like this? (I "copied" from video)

  $service = \Drupal::service('ai.provider');
  $provider = $service->getInstance('openai');
  $messages = new ChatInput([
    new chatMessage('system', 'You are helpful assistant.');
    new chatMessage('user', $question);
  ]);
  $message = $ai->chat($messages, 'gpt-4o')->getNormalized();
  $entity->set('field_text', $text->getMessage());  

📌 Add Readme Active I think is priority, video was difficult to find and "copy".

Also ->getNormalized() is better to be ->getMessage()
and ->getMessage() is better ->getText()
:-)

Also @wouters_f, it would be useful to have proposal how you would like ai() to work. For example, how one would choose the provider (openai, mistral) or model? What exact parameters and what response? What use-case would it be best be limited to?

I love the analogy with for example cache_get . I don't want to know if we're using the memcache or db cache system (or redis).
It would be great if things would keep working if I switch to another AI provider.

Calling AI() ideas, good and bad

You can go multiple directions with this.

Point directly to the provider and model.

ai('openai', 'gpt-4o', 'Summarize ' . $value . ' in 2 sentences');

This would not allow switching to another LLM and keeping things working. But it's more verbose and clear.

No provider
I don't think going to the model without provider would be a good idea (different vendors with same name models).

ai($model, 'Summarize ' . $value . ' in 2 sentences');

We should not do this.

Just take the first Available
If you take abstraction of Providers and models you could think of this

ai('text_completion', 'Summarize ' . $value . ' in 2 sentences');

It's the simplest, with the most assumptions.
This takes the defaults you've configured (I've configured Openai and GPT4-0 for example).
Coming back to the cache analogy, swapping out LLM's would not break this integration.
I absolutely agree Multiple outputs are possible here. (Generate text / generate an asset).
In short: It takes the first available (or configured) text model (LLM) and gives it the prompt.
I like this approach best.
I also think that most users of ai() will just use one LLM.

Expected behavior (ideas)

Then whatever it returns, will be returned (maybe even "dumbed down" ) so that only the output or some basic wrapping comes back.
I'm not knowledgable on the "normalisation" process (yet) so it could be naive.

If you want to choose providers or have more options. I absolutely agree that you should not be doing things with ai().
In other words ai() can be your training wheels, we'll hold your hands, Anything else should be done differently.

MEDIA

If you start working with media I think you're not in the target group for the ai() function. and you should be doing things differently.
But if you would really want to make that work however:
You could test if the input is not a string (prompt) but an array (prompt,asset) and "just" work with that.
ai('image_to_image', [$asset, $prompt] );
But I don't know enough about what is expected in the back to give sound examples here.
This does feel off.

I could be oversimplifying things here, and I'm sorry for that.

Also: If it's not the direction we want to go: I also understand. (I can just make a submodule that provides the function call to try it).

I'll try to make a submodule that does this on the dev days, so we can play around with it.

I'll try to make a submodule that does this on the dev days

👍

// Example implementation of the first one
function ai_based_on_settings($input) {
  $instance = \Drupal::config('ai_function')->get('ai_function_provider');
  $model = \Drupal::config('ai_function')->get('ai_function_model');
  $service = \Drupal::service('ai.provider');
  $provider = $service->getInstance($instance);
  $messages = new ChatInput([
    new chatMessage('system', 'You are helpful assistant.'),
    new chatMessage('user', $input),
  ]);
  $message = $provider->chat($messages, $model)->getNormalized();
  return $message->getMessage();
}

practical for people that really just hobby coders doing some changes

Hobby coders in the future will code with AI help, where AI should access AI module documentation without any hassle and produce correct and high quality object oriented code.

This is what I will continue to experiment during dev days :-)

And that is why I am focused on #3455852, because this way AI can have the documentation easily accessible.

Looking forward to meet and brainstorm!
Maybe I'm still in an old paradigm/mindset :D

I've cooked up a simple version that allows one to do:

$response = ai('Who built you');

Check the branch / merge request for the code, it works!

Tested to work with

• Mistral
• Openai

So the abstraction layer works very fine :D

Altered it a tiny bit that allows the following too (type as a input param).
Have only tried out the chat version yet however.

$embedding = ai('Who built you'); //chat
$embedding = ai('Who built you', 'chat');
$embedding = ai('Who built you', 'embedding');
$embedding = ai('Who built you', 'moderation');
$embedding = ai('Who built you', 'text_to_image');
$embedding = ai('Who built you', 'text_to_speech');

I think you will have to eiher input a normalized object as input or the providers input, doing something in between will be really confusing since you now have two normalizing layers, and one that can't do everything that in the spec. In you example with Chat, attaching images or having chat between two humans and one AI or having chat history is not possible.

I would also do the assumption that if someone uses a function like this, they do not want to meddle around with objects, thats what gets them to do procedural code in the first place, so taking raw data and receiving raw data, with the option of normalizing is the way to go I think. People still trying to use Drupal 7 syntax are most likely used to associative arrays, rather then objects.

so you can do something like

$messages = [
  ['role' => 'system', 'content' => 'You are an image reader'],
  ['role' => 'user', 
    'content' => [ 
       'type' => 'text,
       'text' => 'Could you describe the image for me?'
    ],
    [
      'type' => 'image_url',
      'image_url' = base64_encode_helper($someimage),
    ],
  ],
];

ai($messages);

// Returns something like in assoc array
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "gpt-3.5-turbo-0125",
  "system_fingerprint": "fp_44709d6fcb",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "\n\nThat is an image of a dog",
    },
    "logprobs": null,
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21
  }
}

This would of course not be possible to abstract into other providers since its OpenAI specific, but I'm guessing people using this are only looking for a quick fix to call the api.

The other way would be to have ai and ai_normalized, where ai_normalize takes and returns normalization objects, but since its one function for all different calls, it will not be able to suggest whats in that object in your IDE.

Or you could take the decision that its only for chat, but even then things like images (or other files) or conversations for chatbots needs to be solved.

Or its a pure text producing endpoints that takes text in and spits text out?

Also ->getMessage() is better ->getText()
and ->getNormalized() is better to be ->getMessage()
:-)

@mindaugasd - I agree it should be getText(), its gets even more obvious with this ticket: https://www.drupal.org/project/ai/issues/3456629 🌱 Add Image into ChatMessage object (refactor or no breaking change) Active

For getNormalized(), I think its good to have a known way of receiveing the data that is known to be normalized over all the operations. So you use getNormalized() for the output from text-to-image etc as well, they just happen to be different objects returned with a common interface.

Like this?
https://git.drupalcode.org/project/ai/-/merge_requests/8

On line 37 the _ai_check_default_provider_and_model(); should be _ai_function_check_default_provider_and_model
Also if you didn't configure a default warnings are thrown:

Warning: Undefined array key "chat" in ai() (line 41 of modules/contrib/ai/modules/ai_function/ai_function.module).
And
Warning: Trying to access array offset on value of type null in ai() (line 41 of modules/contrib/ai/modules/ai_function/ai_function.module).

If no default is configured it would be best that any execution is halted in function ai().
Also wouldn't it be better to have function ai() work like the t() function using a object like AiMarkup like TranslatableMarkup?

On line 37 the _ai_check_default_provider_and_model(); should be _ai_function_check_default_provider_and_model
Also if you didn't configure a default warnings are thrown:

Solved this one.

If no default is configured it would be best that any execution is halted in function ai().
Also wouldn't it be better to have function ai() work like the t() function using a object like AiMarkup like TranslatableMarkup?

I disagree. For this simple function. In T, you also just pass in a string.
I want to make this as easy as possible.

Reviewed functionality and everything worked except the requirements in /admin/report/status
The install file was missing the use Drupal\core\Url statement.
I fixed that in latest commit and also provided the install file with a @file block.

If someone can do a quick check this could also move to reviewed.

I like it!
Thanks!

So are wo doing this or what?