Basics of Threading and Improvements to Assistants

Created on 23 August 2024, 5 months ago

Problem/Motivation

In alpha6 the Chatbot with assistants doesn't support threading and history even though it looks like it should. We should improve this for Alpha7.

Proposed resolution

  • Handle Threading with the assistants api.
  • Can we make it so you can choose threading if its stored in the session vs stored on the server? (Even if we don't implement Server, can call it "Todo"), also choose no threading.
  • Context system - How do we know how much context we send to subsequent prompts?
  • Choose different options for how much history there is (Just cut off after a certain tokens, summarise, RAG (use the simplest for Alpha 7)
  • Can we make it so that it chooses when to use RAG, rather than always doing a RAG search. This could be achieved via a simplechat model to save money (Such as ChatGpt 4o mini)
  • Implement "SimpleChat" - default -
  • Chatbot should be able to receive files. (Where are those files stored if we don't store history?)
  • Add - Flag - Handles JSON well - Allow providers to mention if a specific model handles JSON well. Modules can provide warnings as a model too small and ineffective to handle JSON is asked to do a task.
  • If we decide a simple model decides if its RAG or not check the flag for JSON
  • Add the ability for a "Pre Search" prompt and a "Post search Prompt". If its empty don't use it.

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

1.0

Component

AI Search

Created by

🇬🇧United Kingdom yautja_cetanu

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024