Contrib.social

Create LLM abstraction layer

Open on Drupal.org →
Created on 8 June 2024, about 1 year ago

Problem/Motivation

Drupal being a truly open project has to ensure users can choose which AI service they want to use instead of depending (hard-coding) a single one, therefore Drupal needs an LLM abstraction layer separating and decoupling Drupal features from AI services, enabling to quickly switch service provider when needed without any hassle.

LLM abstraction has to cover needs of different Drupal projects (wysiwyg, chat, translations, interpolations etc...) providing a single API to define, configure and use LLMs.

Proposed resolution

There are great examples in Python world where libraries such as https://github.com/BerriAI/litellm managed to create a single API to access them all. Its tons of work and maintenance, so Drupal don't have to cover all those LLMs, but Drupal has to provide baseline API for everyone build upon.

Proposal is to create these building blocks:

  1. form element (to configure LLMs)
  2. plugin system (to define LLMs)

Example would be AI chat user interface → , where it is best to embed LLM configuration form inline in multiple existing places (entity type form, chat form), also have it overriddable configuration within context where user is actually using the app instead opening separate windows and getting lost. Separate windows would also be there depending on application requirements.

Having LLM as form element would provide flexibility. and where to actually store that configuration, I am not sure. Each module can store configuration where they want. And AI module can provide or recommend a default way to store config. For example, we could even imagine that Augmentor → module could plug into this LLM abstraction layer and store configuration the same way like before but get access to great deal of new LLMs and have configuration form sorted.

Actual form element should be configurable: enable, disable and change various features.

Actual LLMs I think have to be provided by user contributed modules using a plugin system, examples:

Examples of similar projects:

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

1.0

Component

Code

Created by

🇱🇹Lithuania mindaugasd

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024