Make Embeddings Engine Pluggable

Created on 24 May 2024, 6 months ago
Updated 13 June 2024, 5 months ago

Problem/Motivation

Currently the embeddings are hard coded to be generated using OpenAI. This should be possible to use any engine like open-source stuff like Nomic Embedding, GTE Large or stuff like Claude/Anthropic.

Proposed resolution

Add a pluginmanager that exposes a plugin system. Require Drupal 10.3 so annotation support is not needed.
Add title, description, vector dimension size as required values.
Add a method for generating embeddings.

User interface changes

Add a configuration where you can choose which embeddings engine you want for a specific index.

Feature request
Status

Fixed

Version

1.0

Component

Code

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024