When using the AI Search module, which leverages the Search API for indexing content, we encounter a significant issue with large items that require extensive processing. The module needs to chunk content and generate embeddings for each chunk. However, this process can be time-consuming, especially for large items that result in hundreds of chunks. As a result, the indexing process can exceed the max_execution_time limit set in PHP, causing timeouts during Cron runs or user-initiated batch processes.
Needs work
1.0
AI Search
This is great, nice to be able to handle giant single nodes, e.g. guides or books perhaps. Added some comments to the MR, but the only general thing is I think we probably need to make this an opt-in given there are some less configurable hosts well used in the drupal community like Pantheon that have very infrequent cron