Contrib.social

Handle chat history vs reduced context length with sensible defaults.

Open on Drupal.org →
Created on 2 July 2024, 6 months ago

Problem/Motivation

Search API AI out of the box has no chat history even though it looks like it does because its using a chatbot. This was particularly an issue because when it was built the context limits were so small that putting a history would prevent there being much possible space for the context.

But with larger context windows it might be possible to have some form of history. But we need to decide what goes into the history (Do we provide every Question + Context in the past? Does the LLM perform a RAG search with every new question? Or does it decide for itself if it needs to do another RAG search or provide its answer based on previous user and assistent messages? When it has a user message does it seperate the user query from the "User Prompt + Context" + system message".

I've set to major because I think a chatbot expects this and so it would be cool to solve but its not strictly necessary for a Search API AI port and so can be bumped to minor. We can also pull it out of the META issue.

(Perhaps though we shouldn't release this as a chatbot until this is resolved and instead create some other UI, something like the prompt explorer)

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

✨ Feature request
Status

Active

Version

1.0

Component

AI Search

Created by

🇬🇧United Kingdom yautja_cetanu

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024