[ai_search] small caching improvement

Created on 2 April 2025, about 20 hours ago

It would be practical if we could in the vector store (eg. milvus/pinecone) also cache a rendered version of the chunk (as is done with elasticsarch and solr).
This would mean that on retrieval we could immediately use the rendered markdown to pass into the prompt without further processing and data loading.
This might mean a significant performance improvement on rag searches (where we render the content to pass into the prompt).

Feature request
Status

Needs work

Version

1.1

Component

AI Search

Created by

🇧🇪Belgium wouters_f Leuven

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @wouters_f
  • 🇬🇧United Kingdom scott_euser

    I suppose that it's possible, though if we would do this, it should be optional in case the vector storage is paid by storage and the user has no purpose for it. E.g. It's also very likely the results would display the full entity in a view mode (eg card or teaser) rather than that of the chunk.

    It may also be more difficult if the chunk does not have valid html due to chunking.

    (setting back to active since no actual code here that needs work)

Production build 0.71.5 2024