Exception: Failed to insert into collection:

What happens if you set the dimensions to 1024?

I still get the same error

I just changed the Vector Dimension to 1024 and now the error is "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)"

the path takes me to the function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I just changed the Vector Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php).

The path takes me to the function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I changed the vector Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)."

the path takes me to this function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I changed the Vertor Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)."

the path takes me to this function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I changed the Vertor Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)."

the path takes me to this function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I changed the Vector Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)."

The path takes me to this function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I changed the Vector Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)."

the path takes me to the function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

I changed the Vector Dimension to 1024 and now i get the error "TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)."

the path takes me to the function
/**
* Acts as static factory, and returns a new Response instance.
*
* @param array{object: string, data: array, index: int}>, usage: array{prompt_tokens: int, total_tokens: int}} $attributes
*/
public static function from(array $attributes, MetaInformation $meta): self
{
$embeddings = array_map(fn (array $result): CreateResponseEmbedding => CreateResponseEmbedding::from(
$result
), $attributes['data']);

return new self(
$attributes['object'],
$embeddings,
CreateResponseUsage::from($attributes['usage']),
$meta,
);
}

Whatever you're doing to constantly send notifications please stop. It definitely won't make us want to help you.

I am sorry about i dont not see my comment for some weird reason and did not want to leave you with no response

Excessive notifications were sent due to a quirk in the site's spam filtering. User fessouma is not to blame for this.

Okay no problem, I guess some issue with drupal.org then with new users, I see your comments now which oddly weren't showing yesterday. Glad they came through!

So that's good progress then, we have a new issue. It seems the error is the dependency outside the module but probably we are calling it missing some info. Could you enable the stack trace so you can see where in the AI module we are triggering the error please? https://www.drupal.org/docs/develop/development-tools/enable-verbose-err... →

Hi fessouma - judging from your code it seems like the Mistral provider might not be answering correctly. Unfortunatly we are using a not 100% compliant web client for the Mistral client, we have a ticket for it here: https://www.drupal.org/project/ai/issues/3460283 ✨ Build custom client for Mistral or switch to HelgeSverre lib Active .

This means that we can't even catch the error here at all times, so we will not be able to output a nice message or log message what went wrong.

What this means is one of two things could go wrong here:

1. You can not change vector dimensions after they are set. You would need to setup another search database/colleciton then and make sure its 1024.
2. The Mistral API is not responding correctly.

The second point you can easily try by enabling the module "AI API Explorer" and then go to /admin/config/ai/explorers/ai-embeddings

There you can choose your Mistral Embed model and just put anything into the big textarea. If it works you should get 1024 numbers back.

i just changed the error log the first error
TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php)
#0 /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php(45): array_map(Object(Closure), NULL)
#1 /code/vendor/openai-php/client/src/Resources/Embeddings.php(30): OpenAI\Responses\Embeddings\CreateResponse::from(Array, Object(OpenAI\Responses\Meta\MetaInformation))
#2 /code/web/modules/contrib/ai/modules/providers/provider_mistral/src/Plugin/AiProvider/MistralProvider.php(220): OpenAI\Resources\Embeddings->create(Array)
#3 [internal function]: Drupal\provider_mistral\Plugin\AiProvider\MistralProvider->embeddings('# COPYCAT FRIED...', 'mistral-embed', Array)
#4 /code/web/modules/contrib/ai/src/Plugin/ProviderProxy.php(150): ReflectionMethod->invokeArgs(Object(Drupal\provider_mistral\Plugin\AiProvider\MistralProvider), Array)
#5 /code/web/modules/contrib/ai/src/Plugin/ProviderProxy.php(81): Drupal\ai\Plugin\ProviderProxy->wrapperCall(Object(ReflectionMethod), Array)
#6 /code/web/modules/contrib/ai/modules/ai_search/src/Plugin/EmbeddingStrategy/MetadataEmbeddingBase.php(72): Drupal\ai\Plugin\ProviderProxy->__call('embeddings', Array)
#7 /code/web/modules/contrib/ai/modules/ai_search/src/Plugin/EmbeddingStrategy/MetadataEmbeddingBase.php(42): Drupal\ai_search\Plugin\EmbeddingStrategy\MetadataEmbeddingBase->getRawEmbeddings(Array)
#8 /code/web/modules/contrib/ai/modules/ai_search/src/Plugin/search_api/backend/SearchApiAiSearchBackend.php(339): Drupal\ai_search\Plugin\EmbeddingStrategy\MetadataEmbeddingBase->getEmbedding('mistral__mistra...', 'mistral__mistra...', Array, Array, Object(Drupal\search_api\Item\Item))
#9 /code/web/modules/contrib/search_api/src/Entity/Server.php(350): Drupal\ai_search\Plugin\search_api\backend\SearchApiAiSearchBackend->indexItems(Object(Drupal\search_api\Entity\Index), Array)
#10 /code/web/modules/contrib/search_api/src/Entity/Index.php(1006): Drupal\search_api\Entity\Server->indexItems(Object(Drupal\search_api\Entity\Index), Array)
#11 /code/web/modules/contrib/search_api/src/Entity/Index.php(935): Drupal\search_api\Entity\Index->indexSpecificItems(Array)
#12 /code/web/modules/contrib/search_api/search_api.module(117): Drupal\search_api\Entity\Index->indexItems(10)
#13 /code/web/core/lib/Drupal/Core/Cron.php(335): search_api_cron()
#14 /code/web/core/lib/Drupal/Core/Extension/ModuleHandler.php(395): Drupal\Core\Cron->Drupal\Core\{closure}(Object(Closure), 'search_api')
#15 /code/web/core/lib/Drupal/Core/Cron.php(318): Drupal\Core\Extension\ModuleHandler->invokeAllWith('cron', Object(Closure))
#16 /code/web/core/lib/Drupal/Core/Cron.php(159): Drupal\Core\Cron->invokeCronHandlers()
#17 /code/web/core/lib/Drupal/Core/ProxyClass/Cron.php(75): Drupal\Core\Cron->run()
#18 /code/web/core/modules/automated_cron/src/EventSubscriber/AutomatedCron.php(65): Drupal\Core\ProxyClass\Cron->run()
#19 [internal function]: Drupal\automated_cron\EventSubscriber\AutomatedCron->onTerminate(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#20 /code/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func(Array, Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#21 /code/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...')
#22 /code/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(66): Symfony\Component\HttpKernel\HttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\LocalRedirectResponse))
#23 /code/web/core/lib/Drupal/Core/DrupalKernel.php(715): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\LocalRedirectResponse))
#24 /code/web/index.php(22): Drupal\Core\DrupalKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\LocalRedirectResponse))
#25 {main}

I also have an error for each content in the content type i want to embed
"Error while trying to render item entity:node/9:en with view mode default for search index AI testing collection: Call to a member function getDimension() on null in Drupal\search_api_ai\Plugin\search_api\data_type\Embeddings->getValue() (line 61 of /code/web/modules/contrib/search_api_ai/src/Plugin/search_api/data_type/Embeddings.php). "

#0 /code/web/modules/contrib/search_api/src/Item/Field.php(568): Drupal\search_api_ai\Plugin\search_api\data_type\Embeddings->getValue('\n\n #1 /code/web/modules/contrib/search_api/src/Plugin/search_api/processor/RenderedItem.php(226): Drupal\search_api\Item\Field->addValue('\n\n #2 /code/web/modules/contrib/search_api/src/Item/Item.php(281): Drupal\search_api\Plugin\search_api\processor\RenderedItem->addFieldValues(Object(Drupal\search_api\Item\Item))
#3 /code/web/modules/contrib/search_api/src/Entity/Index.php(987): Drupal\search_api\Item\Item->getFields()
#4 /code/web/modules/contrib/search_api/src/Entity/Index.php(935): Drupal\search_api\Entity\Index->indexSpecificItems(Array)
#5 /code/web/modules/contrib/search_api/search_api.module(117): Drupal\search_api\Entity\Index->indexItems(10)
#6 /code/web/core/lib/Drupal/Core/Cron.php(335): search_api_cron()
#7 /code/web/core/lib/Drupal/Core/Extension/ModuleHandler.php(395): Drupal\Core\Cron->Drupal\Core\{closure}(Object(Closure), 'search_api')
#8 /code/web/core/lib/Drupal/Core/Cron.php(318): Drupal\Core\Extension\ModuleHandler->invokeAllWith('cron', Object(Closure))
#9 /code/web/core/lib/Drupal/Core/Cron.php(159): Drupal\Core\Cron->invokeCronHandlers()
#10 /code/web/core/lib/Drupal/Core/ProxyClass/Cron.php(75): Drupal\Core\Cron->run()
#11 /code/web/core/modules/automated_cron/src/EventSubscriber/AutomatedCron.php(65): Drupal\Core\ProxyClass\Cron->run()
#12 [internal function]: Drupal\automated_cron\EventSubscriber\AutomatedCron->onTerminate(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#13 /code/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func(Array, Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#14 /code/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...')
#15 /code/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(66): Symfony\Component\HttpKernel\HttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\LocalRedirectResponse))
#16 /code/web/core/lib/Drupal/Core/DrupalKernel.php(715): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\LocalRedirectResponse))
#17 /code/web/index.php(22): Drupal\Core\DrupalKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\LocalRedirectResponse))
#18 {main}

I created a whole new Database/collection. in the AI API Explore embedding works but when i try to index the content it still get the error

An error has occurred.
Continue to the error page

An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: /batch?id=81&op=do_nojs&op=do
StatusText: error
ResponseText: The website encountered an unexpected error. Try again later.TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 45 of /code/vendor/openai-php/client/src/Responses/Embeddings/CreateResponse.php). OpenAI\Responses\Embeddings\CreateResponse::from(Array, Object) (Line: 30)
OpenAI\Resources\Embeddings->create(Array) (Line: 220)
Drupal\provider_mistral\Plugin\AiProvider\MistralProvider->embeddings('# DOUBLE DECKER TACOS', 'mistral-embed', Array)
ReflectionMethod->invokeArgs(Object, Array) (Line: 150)
Drupal\ai\Plugin\ProviderProxy->wrapperCall(Object, Array) (Line: 81)
Drupal\ai\Plugin\ProviderProxy->__call('embeddings', Array) (Line: 72)
Drupal\ai_search\Plugin\EmbeddingStrategy\MetadataEmbeddingBase->getRawEmbeddings(Array) (Line: 42)
Drupal\ai_search\Plugin\EmbeddingStrategy\MetadataEmbeddingBase->getEmbedding('mistral__mistral-embed', 'mistral__codestral-mamba-latest', Array, Array, Object) (Line: 339)
Drupal\ai_search\Plugin\search_api\backend\SearchApiAiSearchBackend->indexItems(Object, Array) (Line: 350)
Drupal\search_api\Entity\Server->indexItems(Object, Array) (Line: 1006)
Drupal\search_api\Entity\Index->indexSpecificItems(Array) (Line: 935)
Drupal\search_api\Entity\Index->indexItems(5) (Line: 160)
Drupal\search_api\IndexBatchHelper::process(Object, 5, -1, Array) (Line: 296)
_batch_process() (Line: 138)
_batch_do() (Line: 94)
_batch_page(Object) (Line: 52)
Drupal\system\Controller\BatchController->batchPage(Object)
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 638)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 121)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 181)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 53)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 28)
Drupal\Core\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 32)
Drupal\big_pipe\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 36)
Drupal\Core\StackMiddleware\AjaxPageState->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 741)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

So if you have it working in API Explorer, I just tried it and it works for me, though the default embeddings value is wrong, I will change this - could you follow this to hopefully get it working:

1. When you add the database, click the Set Dimensions Manually
2. Add 1024
3. When you add an index, you have to add at least one field of type embeddings.

Then it should work.

I will push a change for the defualt embeddings value.

the Database
So the backend is AI Search, The vector Databse is Milvus DB,
The tokenizer Chat model is Mistral AI - open-mistral-7b,
the embedding Strategy is Metadata enriched chunk
I left the maximum chunk size blank
the minimum chunk overlap is 100
metadata maximum percentage is 30%
The embedding Engine is Mistral AI | mistal embed and the dimension is 1024

The Index
Datasource Content
i selected one content type for the datasource
the cron batch size is 5

I have 2 field to embed; General: Rendered HTML output with type embedding, Content: Title set as a string type