Add support in AI Interpolator for Hugging Face

@ressa linking comment from Augmentor which applies here as well #3420590-5: Add Hugging Face assistant maker support in Augmentor AI? → .

Thanks! But as in the other issue, this is just about basic Hugging Face support.

Add link.

Update Issue Summary.

@ressa Is the idea to add some specific models inference endpoint via serverless or dedicated huggingface instances? Like https://huggingface.co/docs/api-inference/en/index?

I have been thinking about adding the mistral LLM just to try it out.

But in general the problem is that there are over 100k different models and each have their specific use case. I could surely add modules if there is a featureset that is not covered by the current models, but for LLM's for instance a generic local/self-hosted solution for certain models from HF like https://ollama.com/ makes more sense, something that is already being worked on here:
https://www.drupal.org/project/ollama →

Could you please specify exactly which model or tool from HF you are interested in?

First, let me just thank you for all the great work you are doing @Marcus_Johansson! I have watched a quite a few of your AI Interpolator videos on https://www.youtube.com/@drupal_ai/ and it's an incredible ecosystem you have created, so thank you very much for that.

I did see that @Orkut Murat Yılmaz created an Ollama AI module, which is really interesting (great logo!). Using a self-hosted LLM would be nice, when it's released.

Initially, I am simply looking for a free ChatGPT-like solution, which is not from OpenAI, offering an API for text prompts. I am not sure if HuggingFace Hosted Inference API or the new HuggingFace Assistants on HuggingChat could fit that requirement? (Though HF Assistant don't yet offer an API ...)

Or perhaps a free service already exists, like Mistral LLM which you mention? It looks like they do offer some free AI services.

@ressa Thanks for the kind words.

So, having it for free without using self-hosted models (which comes with its own costs) is problematic, because no one wants to cover that cost freely, even not Huggingface.

Huggingface does have the none-dedicated inference API for models that works on CPU. This means that the smaller models like Mistral could be run for prototyping for instance. But it comes with a huge caveat, that it would only work when there are machines available, which is almost never true when America is awake and at the office. So, it would fail a lot.

Also these smaller models might be able to maybe do a summary or answer a question, but they will never be able to do complex output that is needed for stuff like categorization, charting data, pinpoining on maps, custom fields etc.

Huggingface has a dedicated inference API that would work better, but this costs money. Making a module that can work against this could make sense for most common models and since you can host and use your custom models, it will even work for this. But in that case, as mentioned, it would still cost money, see https://huggingface.co/pricing.

I will try and see what I can make out of it - but my best guess is that it would be a model that covers the dedicated inference API for certain use cases. So if your goal is to save money, not to use Huggingface, it will not help you. Ollama is you best shot at that.

Thanks for the clarification about the improbability of free AI-services. As you write, that leaves us with the option of self-hosting, if you want to avoid creating an account, and registering your credit card with an AI-service.

About Hugging Face, during the discussion in the other issue ✨ Add support in Augmentor AI for alternative AI-service(s) Active I realized that Hugging Face has a lot of different AI-features, and that integration with HF is not straightforward, and also that my request was too vague ... So, I fleshed out a more general feature request. I'll add it in the Issue summary here as well. It does feel like double posting, but since there is no monopoly on these AI-solutions, and multiple modules may support the same AI-services, it may be ok under these conditions?

Ollama is you best shot at that.

I agree, I just tried the Ollama Docker image (see #3399485-6: AI chat user interface module support → ) and it was up and running in minutes on my machine. So Ollama could be the primary candidate, for a self-hosted, feature-rich LLM integrated in AI Interpolator?

@ressa - So I thought about it a little bit and since I have worked with Huggingface on Python based AI stuff, I know it fairly well and it will be easy enough to implement.

The idea is that you can choose the free inference API when its available and the error handling when it's not will be as any other thing in AI Interpolator where you get an error message. Then you can add the dedicated API endpoints if you want to have paid stability and the cool thing we can do then is that you can manipulate that endpoint and use your own fine-tuned models. This is actually something really useful.

Since there are 100k+ models there and they do different things and so-on, it makes little sense to create on module per model, instead I will make this into one model, where the difference is that you need to enable rules on a settings page.

I would start with the following:

• Mixtral-8x7B (just dedicated, with similar rules as OpenAI)
• Mistral-7B-v0.1 (free and dedicated, can do summary and text generation)
• detr-resnet-50 (free and dedicated, object detection)
• wav2vec2-lg-xlsr-en-speech-emotion-recognition (free and dedicated, voice emotion recognition)

The later two are edge-cases, but finetuned object recognition is something my company needs and emotion recognition of voices is just to try something completely different from the Interpolators that exists.

Thanks @Marcus_Johansson, that sounds fantastic! I see now that I forgot to update the title, which I'll do now, to make it less HuggingFace-centric.

Perhaps this issue could be a meta-issue, for supporting different AI-services?

And, of course, feel free to edit any part of the Issue Summary, to make it more precise.

Since hugging face is quite a big website with a lot of features, and Augmentor also have such issue, maybe it makes sense to call the new module "huggingface" instead of "ai_interpolator_huggingface".

Therefore, in theory, these could be sub-modules:
huggingface
huggingface_interpolator
huggingface_augmentor
huggingface_...etc

Similar like we started with https://www.drupal.org/project/ollama →

But fine whatever makes more sense.

@mindausgasd - that sounds smart. I'll do the initial module and setup for AI Interpolator and then set that the module seeks co-maintainers, so anyone wanting to do something specific with the Huggingface services could do this.

I started on this and understood that they have generalized input and output for models, so it will not only support those 4 use cases, but it will support every model that have opted in for a free or dedicated inference API :)

So the rule for instance will be "Audio Classification" to classify audio according to some model and it will always use one or many audio files as input and then use text or text list or taxonomy as output.

It makes setup for the Interpolator a little bit harder, but it makes it soooo much more flexible.

@ressa - the first version is ready to test. 1.0.0-alpha1, see here: https://www.drupal.org/project/huggingface →

If you for instance want to have free summaries (until you get rate limits), you can try the following steps.

1. Download the AI Interpolator module
2. Download the Huggingface module
3. Install the AI Interpolator Huggingface module, it will install the dependencies
4. On some node type, create a field Text (formatted, long) called Body, that holds the article that you want to summarize
5. On the same node type, create another field Text (formatted, long) called Summary.
6. In the settings, click "Enable AI Interpolator"
7. Choose the rule Huggingface Summarization
8. Choose Article in the Interpolator Base Field
9. Just write {{ context }} in the Interpolator Prompt, the summarization tools doesnt take commands
10. Under advanced settings, paste sshleifer/distilbart-cnn-12-6 or some other summarization model into the Huggingface Model field
11. Save the settings
12. Create a new article, just fill in the article field with some long article
13. Save, and if you are not rate limited (you get warning message), you will get a summary

Its still very crude, I need to do validation and also add a lot more rules, since there are around 20 different generation types and I only did 3 so far (Audio Classification, Text Generation and Summarization). But its a start :)

Thank you for the idea @ressa btw - I didn't know that they had dedicated machines and this is something our company will use now!

I'll set this to needs review, since the first version is done. If you find it usable, please set to fixed. The improvements or minor bugs can be added as tickets in the module

You're welcome @Marcus_Johansson, I am also very thankful for your great work here :)

Your steps were perfectly understandable (only minor detail was step #10 "Article" > "Body") and I just generated a summary of the article https://evolvingweb.com/blog/what-is-drupal:

At its core, Drupal is a content management system (CMS) A CMS gives non-developers a way to create, structure, update, and manage content. A DXP is an integrated set of technologies that deliver user experiences across various digital touchpoints. Evolving Web partners with Acquia—an open-source DXP designed solely for Drupal.

The returned text resume itself is not fantastic, but as a proof of concept -- that Hugging Face works with AI Interpolator, it's perfect!

And it worked for free, just by creating an account and using my Hugging Face token. I can't see there's any active metering on my account, with any limit ... Is https://www.drupal.org/project/huggingface → maybe using this model? https://huggingface.co/docs/api-inference/faq

We can turn this into a Hugging Face issue again, or we could also keep this issue as is, and use it as a meta issue, for several more LLM integrations? It's up to you what you prefer.

I have created an issue for Ollama support, and added it in the Summary.

Its good that it works!

And yeah, most open source models are bad for general tasks in comparison to OpenAI/Gemini, but they are finetunable.

Regarding being free or not - it is free and there is no exact number how and when it works. They have a rate limit on your account and they have a rate limit on the total servers. So the free way of doing it, should not be used for production websites. It also penalizes models that use more resources. I saw that Mistral was not usable freely, even if it is available in the free version (the waiting time just became longer and longer).

You can keep it as a HuggingFace ticket and set it to fixed if the first version works.

Regarding the Ollama ticket here, I will close it since @Orkut Murat Yılmaz is already working on a solution at: https://www.drupal.org/project/ollama → .

Thanks for clarifying the situation with being free, and the limitations. I'll change the issue titel to being Hugging Face specific.

Great work and quick.

FYI: New stabler version out + many new rules. Documented here: https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →

Only text to image rule left to do.

Fantastic work with the documentation of HF, and AI Interpolator overall, very impressive! I'll give it a thorough read-through later.

Perhaps add a link at the top of the https://www.drupal.org/project/ai_interpolator/ → project page to make the users aware that it exists?

It will be there soon, I just want to finish developers documentation first :)

Sounds great, though with the current documentation, it's already in the top 1% of best documented modules :)

Thanks :)

The last piece of documentation is added and now all the modules have documentations linked!

Beautiful, thanks Marcus_Johansson!

Automatically closed - issue fixed for 2 weeks with no activity.

I created a feature request for Claude, in ✨ Add support in AI Interpolator for Claude Active .