Support tools calling

Going to work on this during DDD25

@jibla: it turns out that function calling is not yet supported by google-gemini-php. There is a PR https://github.com/google-gemini-php/client/pull/67 against beta branch, but not merged yet.

Should we park the issue until PR is merged, or switch to vanilla Guzzle requests as suggested in related Google Vertex issue https://www.drupal.org/project/ai_provider_google_vertex/issues/3506263 🐛 Vertex provider uses PredictionServiceClient and slows down response time. Active ?

Heads up: https://github.com/google-gemini-php/client/pull/92 is merged into main

@valthebald exactly! :) I will take care of it, this week

Any updates on this? This module still doesn't work with AI 1.1.0.

@balazswmann technically, it does, but you can't use gemini as a chat with tools.
@jibla do you need some help with that?

@valthebald

technically, it does, but you can't use gemini as a chat with tools.

Yes, I meant the chat and tools compatibility. That's exactly I'd need.

@valthebald
Actually if somebody from the community would suggest a MR, it would help. Otherwise I will try to push this in my team.

Hi,

I've created an MR to support the chat tools.

@yce Thank you for the contribution!

Before the code review of the MR, we need a test case to check with. As we don't have any automated tests for the providers yet, I would love if somebody tested this manually.

I'll mark the issue as Needs Review.

Attaching a patch file for the current MR to test with.

Trying to test this with /admin/config/ai/explorers/chat_generator and the existing HtmlToMarkdown function. I ran into an error with the generator not supporting the 'Top K' being a float. Here is the error

TypeError: Gemini\Data\GenerationConfig::__construct(): Argument #6 ($topK) must be of type ?int, float given, called in /var/www/html/web/modules/custom/gemini_provider/src/Plugin/AiProvider/GeminiProvider.php on line 260 in Gemini\Data\GenerationConfig->__construct() (line 39 of /var/www/html/vendor/google-gemini-php/client/src/Data/GenerationConfig.php)

Switching that argument type, I was able to generate code but not execute the function. I will continue to try to test but if someone has a better approach please do share.

An update, after enabling AI agents module and seeing that I can reference other tools like 'Calculator', I tried /admin/config/ai/explorers/chat_generator once again with xdebug and I can see that the code from the MR is calling the Calculator code it is just not outputting it to the screen as tool output. Maybe that is missing?

Going to check this today and report the results

I get this error: TypeError: Gemini\Data\GenerationConfig::__construct(): Argument #6 ($topK) must be of type ?int, float given, called in /var/www/html/web/modules/custom/gemini_provider/src/Plugin/AiProvider/GeminiProvider.php on line 260 in Gemini\Data\GenerationConfig->__construct() (line 39 of /var/www/html/vendor/google-gemini-php/client/src/Data/GenerationConfig.php).

(probably related to the signature change in the library between the versions).

Steps to reproduce:

1. Enable AI Explorer module
2. Go to AI Explorer > Chat generation
3. Select any tool

Here's my configuration:

Actually the signature has not changed between version 1 and 2, here is version 1 line https://github.com/google-gemini-php/client/blob/1.0.15/src/Data/Generat... (still an ?int)

I have this script that I am testing with drush and it calls the function but doesn't generate an output. I left a comment on the MR

<?php

use Drupal\ai\OperationType\Chat\ChatInput;
use Drupal\ai\OperationType\Chat\ChatMessage;
use Drupal\ai\OperationType\Chat\Tools\ToolsInput;

$messages = new ChatInput([
  new ChatMessage('user', 'What is 20 * (4 + 3)?'),
]);
$provider =  \Drupal::service('ai.provider')->createInstance('gemini');

// Set a function call.
$functions = [];
$function_instances = [];
$function_call = \Drupal::service("plugin.manager.ai.function_calls")->createInstance('ai:calculator');
$function_instances[$function_call->getFunctionName()] = $function_call;
$functions[] = $function_call->normalize();

// Set chat tool.
$messages->setChatTools(new ToolsInput($functions));

/** @var \Drupal\ai\OperationType\Chat\ChatOutput $response */
$response = $provider->chat($messages, 'gemini-2.5-flash', ['my-custom-call']);
/** @var \Drupal\ai\OperationType\Chat\ChatMessage $return_message */
$return_message = $response->getNormalized();
echo $return_message->getText() . "\n";
// Returns something like "140"

I submitted a commit that captures the output of the function call if it is an instance of ExecutableFunctionCallInterface. Attaching new patch and inter diff.

This solution https://www.drupal.org/project/ai/issues/3528549 ✨ Allow tool calling in streamed chat Active might make this redundant so something to watch as well.

@dmundra: thanks for the update! Checking soon