Add support in Augmentor AI for alternative AI-service(s)

Maybe this belongs in https://www.drupal.org/project/augmentor → ?

open source assistant maker to rival OpenAI

@ressa assistant maker is a configuration tool of AI. There was plan to configure GPTs with Drupal itself (like https://www.drupal.org/project/aipersona → ) without an external tool, but later those configuration tools were developed by OpenAI and now Hugging Face (but Drupal itself could be configurator 📌 [META] Drupal could be great for building AI tools (like ChatGPT) Active ).

Drupal also does not have integration with OpenAI GPTs maker either. This is the issue ✨ Integrate custom gpt's Active

Even if Drupal would have integration with OpenAI or Hugging face GPTs maker, Drupal does not have good enough chat interface to interact with it https://www.drupal.org/project/aichat → , but possibly those external apps will provide drop-in JS script to integrate chat UI within any kind website.

To create some sort of integration for Hugging Face, one has to create a new contrib module (I modified the description), because Augmentor AI is an ecosystem of contrib modules.

I also modified title to match the description, because Hugging face is quite a big website with all sort of features.

Thanks @mindaugasd, but I just like to have basic prompt support for Hugging Face, like OpenAI / ChatGPT Integration → and this module offer.

I mainly added the link for a recent article about Hugging Face, not to get "maker assistant" support. I'll remove the link, sorry for the confusion.

Add link.

Hi ressa, as mindaugasd says, the Augmentor module provides a plugin system for integrating external AI services. If you would like to integrate in Hugging Face a new Contrib module would be needed which implemented an Augmentor plugin. An augmentor provides the ability to customise the prompt and to send it through to an external service which returns some sort of payload. The payload is then put into a response (associative array) on an endpoint for consumption by Drupal (node edit, or some other code). The big question is which API (method) you would like to use. Can you provide a link to documentation in Hugging Face to the API you would like to call. We could then make some recommendations as to feasibility. At the end of the day, a new contrib module would be needed to implement it.

First off, I owe you an apology @mindaugasd. My understanding of AI is too basic I now see, and you provided a great and thorough explanation. My request was too vague and broad, due to my limited understanding, and I have updated the Issue Summary, with a feature request. Perhaps I have made it to broad ...

And thank you @murrayw for laying out the situation so well. Perhaps this issue would be better placed in the https://www.drupal.org/project/artificial_intelligence_initiative → , or another project?

@ressa thank you for your interest in Drupal AI.

"To avoid monopolies, there should be a wide selection of products to choose between,"

AI open source champion so far was META corporation. They developed Lamma series of models which other open source models were based on (others built on top), and also they started open source AI revolution and are biggest advocates for it (during EU regulation negotiations for example). Recently, Mark Zuckenberg announced that they will continue to lead open source AI.

This is little video overview about that
Zuck: "Meta AGI will be OPEN-SOURCE!" (I was wrong about Meta)

There was also open source alliance formed with many large corporations cooperating together:
IBM, Meta form “AI Alliance” with 50 organizations to promote open source AI

Funny that OpenAI is not open, while other corporations make it open (in order to compete against OpenAI monopoly).

Personally, I don't know what is better, open source or closed source, because there are many arguments for and against, but your interest in is very much welcome. I think that anti-monopoly work is important, and I try to build company agnostic modules, so people could switch the provider (to not be depended on OpenAI only). We had discussions on slack, that openai module could do OpenAI integration only, and migrate other features to other modules, but were was no willingness and it seems like it will not happen, and that is also happening for practical reasons (example #3402579-3: Create a Milvus.io plugin for openai_embeddings module → ) and openai module continue to accumulate features. Also there is an issue about it: 📌 Compare/collaborate with Augmentor Active

But augmentor/interpolator modules are not dependent on OpenAI, so if you want to continue work on having many service providers and reducing monopoly, creating modules for Augmentor or Interpolator ecosystems is the way to go. Augmentor have user interface widgets, so it is most similar to OpenAI module.

I just like to have basic prompt support

For for basic prompt support of any modules, I think the best path is https://www.drupal.org/project/aichat → . It has "aichat_backend_example" submodule with openAI integration. One can quickly edit that module to integrate any service or other module. As a bonus, it is a chat and will be saved for having multi-day AI conversations too.

"Initially we could list 4-5 candidates, and then select one or two based on parameter"

Yes, this would be very welcome work. Probably somebody already did many of such reviews on the internet, and one needs to find it.

@ressa I seen you tested llama in other issue ✨ AI chat user interface module support Active , so sending one very useful link for testing many options very quick https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard

Its AI model leaderboard as voted by people.

And this is the link where you can try all those different models yourself and also vote which one is best,
https://chat.lmsys.org/

At the top you can select different ways to test the models:

• Arena (battle)
• Arena (side-by-side)
• Direct Chat
• Leaderboard

Or this is another link https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard with automated evaluation of huge list of models.

Hi all,

since there was a similar ticket ✨ Add support in AI Interpolator for Hugging Face Fixed on the AI Interpolator and since I had interest to use custom finetuned models I did start on a module for this called Huggingface → .

Based on @mindausgasd suggestion ✨ Add support in AI Interpolator for Hugging Face Fixed I created a generic module for this, rather then a pure submodule/plugin for the AI Interpolator since its such a vast repo of models.

So its split up in a core module, that is just a pure crude API service that can do all the base inferences to any model supporting in on Huggingface + a form to save the API token.

And then there is a submodule specifically for AI Interpolator, that requires that installed.

Anyone interested in helping out with an Augmentor (or any other) submodule, based on this, please let me know and I'll add you as co-maintainers. Or you can create a new module and use this as base if it helps you.

I will look into how the Plugin API for the Augmentor looks, and I can probably add something myself soon for summarization.

I forgot to thank you for sharing links to the different LLM ressources @mindaugasd. It's a pretty complex and overwhelming subject to try to maintain just a basic understanding of, with the constant development of new products and combinations of them ...

@ressa - I added one Augmentor plugin to the 1.0.x-dev version of the Huggingface → module.

It can be used by:

1. Installing the Huggingface Core module and the Huggingface Augmentor module.
2. Adding the API key at /admin/config/huggingface/settings
3. Add an Augmentor at /admin/config/augmentors based on Huggingface Image Classification.
4. In the settings add a model - dima806/facial_emotions_image_detection is a good model for testing emotions on images of faces for instance.
5. Add an image field.
6. Add a taxonomy field that can generate terms, Form formatter Autocomplete Tags Style.
7. Add an augmentor field, Form config, use image as source field and the taxonomy field as target.
8. Try it out - if you don't use dedicated machines it will give you error messages sometimes since machines are not available.

@murrayw - it would be super awesome if someone that worked on the module or have experience working with plugins could have a look at the code or the commit to check that I'm not doing a faulty implementation.

If it looks good, I will try to free up some time too look into other places the Huggingface API works together with the Augmentor module.

Impressive how many different things you achieve, https://podcast.workflows-of-ai.com and now even Augmentor implementation :-)

I created and referencing related issue ✨ Collaboration with existing projects Active from a new module LLM Provider Service → and

LM Studio API →

which also has a goal to integrate many different LLMs.

This new module is needed in a way, because Drupal does not cover a lot of AI models and service providers, but its a large task to integrate those services, so its good thing to collaborate, and also avoid duplication of work and modules, so the Drupal user could have an easier choice, while modules themselves have more users and be able to grow to good quality.

Adding another related issue

The implementation looks good @Marcus_Johansson thank you for that!

@ressa, this augmentor module doesn't include any built-in integration with a particular LLM, this module only provides the ecosystem, tools and rules to write submodules than handle the integration with other services, like https://www.drupal.org/project/augmentor_nlpcloud → , https://www.drupal.org/project/augmentor_aws → or https://www.drupal.org/project/augmentor_chatgpt →

Here you will find a presentation about how to create augmentor submodules: https://drupal-camp2023.den-japan.org/session/c05

Thanks for the plug-in @Marcus_Johansson! And thanks for the links to the submodules @eleonel, and the video which I look forward to watch.