Finish VectorClient plugin code.

Created on 9 November 2023, 8 months ago
Updated 18 November 2023, 7 months ago

Problem/Motivation

The Embeddings settings screen can allow 1 or more vector storage plugins to be used and configured. Currently it is allowing a plugin selection, but not a plugin configuration.

Proposed resolution

Implement configuration forms and schema for vector client plugins.

✨ Feature request
Status

Fixed

Version

1.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States kevinquillen

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Pipeline finished with Skipped
    about 1 year ago
    #12723
  • Pipeline finished with Skipped
    12 months ago
    #13362
  • Pipeline finished with Skipped
    11 months ago
    #14738
  • Pipeline finished with Success
    8 months ago
    #43901
  • Issue created by @kevinquillen
  • πŸ‡ΊπŸ‡ΈUnited States kevinquillen
  • First commit to issue fork.
  • Assigned to scott_euser
  • Status changed to Needs work 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom scott_euser

    Actually seems like there is quite a lot to do here - I have refactored the config storage to follow Core's book/image_style mapping and handling saving in the VectorClientPluginBase so heavy lifting there does not need to be duplicated per Plugin (though Plugins can of course override the Base methods).

    There is a fair bit of duplication though in PineconeClient that now does not work (eg, the SearchForm) as I have not updated these to use the Plugin's configuration. I believe that PineconeClient should be removed in favour of the Pinecone plugin, and so actually next steps would be for me to check each place PineconeClient is used and move it to using the Plugin instead. Do you agree? If so I'll action that + I can update the scope of the summary of this card. I think we cannot move to a separate task as this is not mergeable as is given all it breaks, so unfortunately its going to end up being a slightly bigger merge request.

  • πŸ‡ΊπŸ‡ΈUnited States kevinquillen

    Yeah - that was the idea (removing PineconeClient). I had hit a blocker and don't recall what it was at the time. But the calls and logic in PineconeClient would be copied into Pinecone.php (its a little outdated) and everything routes through the plugin(s).

    It seems like they should at a minimum, support/implement:

    • - Configuration storage
    • - "Test" form definition, the barebones search UI to quickly check your integration/data (like whats there now)
    • - "Status"/index stats tab, where it lists collections data if supported
    • - General methods (CRUD?) required for working with embedding related data
  • πŸ‡¬πŸ‡§United Kingdom scott_euser

    Sounds good, thanks! I will try to progress on it the upcoming days a bit.

  • Issue was unassigned.
  • Status changed to Needs review 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom scott_euser

    Probably we'll need a bunch of follow-ups, particularly once we add other vector clients, I think then the differences will become clear and the interface can updated then to match?

    I did a full test run from index, settings configuration, search, and stats, the only bit that did not work was the delete but that is a problem from the Pinecone starter as you noted in the description in the delete form (possibly can be a follow-up as maybe Delete All will work, which is anyways what the Delete form is doing)

  • Pipeline finished with Skipped
    7 months ago
    #51999
  • Status changed to Fixed 7 months ago
  • πŸ‡¬πŸ‡§United Kingdom scott_euser
  • Automatically closed - issue fixed for 2 weeks with no activity.

  • Pipeline finished with Success
    6 months ago
    #72661
  • Pipeline finished with Success
    6 months ago
    #76413
  • Pipeline finished with Success
    5 months ago
    Total: 153s
    #78905
  • Pipeline finished with Failed
    5 months ago
    #83368
  • Pipeline finished with Skipped
    5 months ago
    #83381
  • Pipeline finished with Failed
    5 months ago
    Total: 238s
    #84947
  • Pipeline finished with Failed
    5 months ago
    Total: 240s
    #84956
  • Pipeline finished with Success
    5 months ago
    #84972
  • Pipeline finished with Success
    5 months ago
    Total: 262s
    #84977
  • Pipeline finished with Skipped
    5 months ago
    #89989
  • Pipeline finished with Success
    4 months ago
    Total: 176s
    #96836
  • Pipeline finished with Success
    4 months ago
    Total: 257s
    #100613
  • Pipeline finished with Failed
    4 months ago
    Total: 149s
    #116799
  • Pipeline finished with Success
    4 months ago
    Total: 149s
    #116804
Production build 0.69.0 2024