Clean up API client and Milvus Backend, getting them closer to working state

Sorry I know its a lot of changes, but it just seemed so far off from working code and did not seem to make sense to make many issues as they would all be merge conflicts. Essentially I hit a ton of errors attempting to connect and index anything.

Okay now I was able to use Milvus free cloud trial and install the module, create a key via Key module (making that a
can be a follow-up perhaps?), and connect successfully to the collections.

Big thanks for your efforts, @scott_euser!

I've just pushed a work-in-progress version of the module. Over the weekend, I'll be building upon the solid foundation you've established in the code. Looking forward to advancing it further!

Not a problem, I wanted to give it a proper whirl as I see great benefit in leveraging Search API. Please note however, I believe we should consider moving this into Open AI module in favour of the lighter weight Search API AI module which leverages that.

Details here https://www.drupal.org/project/openai/issues/3402579#comment-15330851 📌 Create a Milvus.io plugin for openai_embeddings module Active
Also added https://www.drupal.org/project/openai/issues/3403562 ✨ Leverage Key module for storage of Vector Storage API details Active as its a great idea to use Key module as well

Will leave to you to decide of course whether you want to move ahead with this module, but as per my message above (and Search API AI is actually older than this module too, functionality aside), at this time I do not plan to use this myself, sorry!

Thank you for the update.

I believe we should consider moving this into the Open AI module.

While I value your suggestion, my approach is geared towards a design that emphasizes extensibility and flexibility, free from stringent dependencies. The existing structure of that module doesn't quite resonate with this philosophy. Hence, my advocacy for separate modules dedicated to each vector database.

Incorporating vector database management directly into AI modules might risk undermining their adaptability and the crucial principle of separation of concerns. It's important to maintain a clear distinction in functionalities to ensure each module operates optimally within its intended scope

Search API AI is actually older than this module too, functionality aside.

I have indeed explored the Search API AI module. However, there are key differences that constrain my ability to effectively contribute to it:

• I firmly believe in the approach of having individual modules for each vector database.
• My approach is to avoid hard dependencies on any modules, except for the Search API.
• The generation of embeddings should be an independent feature, allowing for broader utilization across modules, and should be handled via processors.
• My goal is to support a wide range of users, including those looking to build semantic search capabilities on their sites, not just those focused on AI-centric applications.

For a more detailed discussion on this topic, you can refer to the conversation on Drupal Slack.

Search API indeed goes independent route, Commerce and Salesforce Suite conversely opted to keep things together. I do not feel strong enough myself in either direction. So I understand your reasoning, personally I am going path of least resistance/easiest to implement to achieve own goals, and for me that means using existing modules. I am fine with having dependencies on other modules in order to achieve that. In any case, feel free to grab/borrow any code I write within Search API AI / Open AI modules if it helps achieve your goals here of course.

Its not the first overlap between modules within Open AI related work and I would have preferred if all was together but I am coming late to the party and just trying to achieve own goals, so I hope no hard feelings here. Its very possible I contribute more here eventually in the future.

Thanks for your understanding and insights. It's great to hear about your approach and how you're working towards your goals. I appreciate your openness to sharing code and ideas – collaboration like this is what makes the community thrive.

No hard feelings at all; I'm looking forward to seeing how our paths might converge in the future. Let's keep in touch and continue to support each other's work!