Contrib.social

Create ShortTermMemoryPlugin

Open on Drupal.org →
Created on 6 June 2025, 3 months ago

Problem/Motivation

Currently we do not cut off any long running agent reasoning at all, if it gets to long it would run out of tokens and die. This means that for real research issues our agents are working quite poorly at the moment.

For assistants we currently just cut of after X amount of sessions if you would like that enabled (or no memory).

There are a lot of different ways to handle this:
1. No memory - it will not even remember the last thing you said.
2. Just keep X amount of the newest messages, the problem is that the agent might repeat using a tool it already used or repeating answers it already answered.
3. Tokenize and just keep X amount of tokens - can be process heavy.
4. Just keep the latest X amount and summarize anything before that into X amount of tokens.
5. Embeddings - you can store the memory in temporary embeddings and get the information based on that.
6. Structured memory template - this is like summary, but it structures more into clear paths like goals, observations, actions, reasoning etc.)
7. And probably 50 other things

We need to make this into something that you can extend any chatbot, agent or other thing that actually works with threads.

Note that ShortTermMemoryPlugin, could be used for storing threads over longer time. The memory definition is if its a memory that has been built over longer time or not: https://superagi.com/introduction-to-agent-summary-improving-agent-outpu...

We will also introduce LongTermMemoryPlugin, but this is a little bit more complex, since its a learning algorithm and from privacy point of view, since you are profiling users - where if you for instance have bought blue dresses before, and ask for a t-shirt it will have a profile of you wanting blue things and give back blue t-shirts.

Proposed resolution

  • Create a ShortTermMemoryPlugin with a plugin manager, attribute and interface.
  • The attribute only needs label and description.
  • The interface should extend configinterface
  • Make an abstract class.
  • Interface should have a method called setThreadId and getThreadId that is also in the abstract that does have an empy identifier.
  • Interface should have a method called, loadHistory where a plugin can be used to load potential history kept somewhere. Returns ChatInput object.
  • Interface should have a method called, setHistory where a consumer can add the history. Takes ChatInput object.
  • Interface should have a method called transform where a plugin can transform the history.
  • Interface should have a method called summarize, and added to abstract that can summarize the thread in few words.
  • Interface should have a method called transformHistory that takes a ChatInput object and runs setHistory, transform, getHistory and returns the potentially transformed ChatInput object. Here we also can add logging, events etc.. This should be in the abstract and have a notice for children to not touch, unless necessary.
  • Abstract should keep threadId and chatOutput as variables
  • Create a first plugin for remember N amount of threads

How it works for a consumer

If the AI Assistants API for instance wants to use this, it will

  1. Add the different plugins as a configuration item on the assistant form.
  2. Expose any subform that has detail settings for the plugin.
  3. Save those in its own configuration
  4. When someone opens a thread whether its via a new thread or an older thread, it will set a made up or remembered threadid via setThreadId
  5. It will then use loadHistory method to get potential history and show it to the end user
  6. When the end user writes a new message it will use transformHistory and give the result to the LLM, with the result back from the LLM it will once again run transformHistory (or setHistory if it wants it to be less processing).
✨ Feature request
Status

Active

Version

1.2

Component

AI Core module

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024