🇬🇧United Kingdom @yautja_cetanu

yautja_cetanu → created an issue.

One thing we should do with this in documentation. If we use this for moderating potentially harmful content (eg suicidal ideation) we could write some documentation after doing some research on how to handle it with the various models as some won't do it, and others will as long as you get the prompt right.

We could explore whether or not we are OK recommending jailbroken open source models for things like this too.

This looks so cool!

ah good point. Think you could post a high res version of the middle colourful one?

Is it worth trying to aim for some consistent naming for LLM providers to make it easier to see them all.

AI_Provider_Fireworksai

For example?

Or just
AI_Fireworksai

So alphabetically all the modules in the AI ecosystem will go together.

Ok! I'll go with that quote on the module page and Dev Days Talk for now!

I'm going to put this into Automators for now. This is very easy to do with Automators manually for each kind of Content.

But there is a design pattern that may be more similar to what Augmenters does where its a reusable design pattern you apply to all content that might be worthwhile.

Also, correct me if I'm wrong Marcus, but it seems like validation with Automators could prevent something being published. But it could prevent it being uploaded (Maybe if we expose it as a widget or something).

100% this should be supported functionality somewhere as it will be used SO much! It possibly overlaps with the proposed AI_Security module.

yautja_cetanu → created an issue.

I think the things that need be said about the AI module (But maybe this is overkill)

• Provides tools to build complex AI applications
• Provides simple functionality out of the box
• Pluggable, abstraction layer so any model can be used

Do we need this? Is abstraction layer implied? Its cool now because all the other modules don't have one but the more this gets used it will be taken for granted it has one. Search API doesn't need to go on and on about abstraction layer?

This is how Search API describes itself:

"This module provides a framework for easily creating searches on any entity known to Drupal, using any kind of search engine. For site administrators, it is a great alternative to other search solutions, since it already incorporates facetting support and the ability to use the Views module for displaying search results, filters, etc. Also, with the Apache Solr integration, a high-performance search engine is available for this module.

Developers, on the other hand, will be impressed by the large flexibility and numerous ways of extension the module provides. Hence, the growing number of additional contrib modules, providing additional functionality or helping users customize some aspects of the search process.
"

yautja_cetanu → created an issue.

I like the banner idea. Eventually we can put links to an introductory video and documentation there too.

Marcus_Johansson → credited yautja_cetanu → .

Marcus_Johansson → credited yautja_cetanu → .

Yeah I think something like AI_Advanced_Image or AI_Image_Manipulation

Yeah don't worry about the Text, I'm just suggesting the colour scheme

yautja_cetanu → created an issue.

Feel like the Prompt Explorer should have some default configured. Stream should be on by default.

(Also even if I scroll up, when the page has finished loading it jumps me back down to the buttom. So I can't see if Streaming is on or not)

One issue. The menu on the left is so long that I have to scroll down quite a lot to get to the actual typing of chat past a lot of config I might not want to do. Then click the chat button.

However the error messages (if I select no LLM provider) and the actual chat responses appear near the top of the screen which I have to scroll up to.

I think LLM provider, Chat and the "Ask AI button" should be nearer the top. and everything else in collapsed divs (I think)

That is worth doing but a couple of things:

• ollama can run a lot of opensource models. Some are very tiny and can be run on any server without a GPU
• This provider here will be a better use case for what you bought up. For people installing it locally with a decent GPU, LMStudio's UI is just so much easier than Olama. https://www.drupal.org/project/ai/issues/3453592 📌 LM Studio LLM Provider Active
• I think Olama will eventually be for organisations that really want this for privacy reasons. Whilst documenting pricing in documentation, especially in your AI Initiative page will be very helpful. Knowing what people have now is probably not helpful as many clients are talking about wanting self-hosted AI anyway because the privacy is worth whatever cost.
• I think an Ollama implementation is much more likely to be included in Starshot. Gabor was inspired by: https://hacks.mozilla.org/2024/05/experimenting-with-local-alt-text-gene...

Worth putting in AI_Extra_Providers

Assigning to seogow to take a look at when he's explored a working search api ai module.

Think this should go in a seperate AI_Extra_Providers module as LMStudio isn't quite opensource and is more going to be used as a developer tool (probably). Happy to be disagreed with here but I think when Olama works that will be the idea tool for this usecase where it can be controlled by Drupal.

Would need to include

• Functionality to choose with models you want to include into your site. This could be a "Find as you type" or you could go to hugging face and copy and paste the exact model name.
• When a model is selected this will appear in the normal drop down
• It's possible we just always allow all the models but this is complicated
• Need to explore supporting all the models that arn't LLMs

We probably want to make it so that AI_Automator can create widgets both on the page and within CKEditor. Then we want some way of easily editing that widget using AI_Automator so the prompt can be changed, a workflow implemented or even RAG with AI search.

OOB you get simple but you can do more with it easily.

Features Olama might need:

• Some ability to do content moderation or use another LLM's https://www.drupal.org/project/ai/issues/3454452 ✨ [META] Create an AI Security module for custom moderation calls Active
• Ability to find models that work with Olama in either some browser or by copying and pasting IDs from an external site
• Ability to just download and install a model directly on the server
• Ability to connect to another model hosted on another server using ollama
• Perhaps the ability for the Drupal site to have remote control over the other ollama server so can download and setup models directly on that

Yes totally agree I think we'd want three things:

• Ability to find and install providers, modesl etc
• Ability to find all the "AI Interpolator plugin" modules
• Perhaps the ability to see all the Recipies that use the AI module

My view was to wait until the project browser comes out before starting this but will have to see.

We've been exploring a logo for the AI Web Connect group, thoughts on this colour scheme or purple and teal?
It feels a little futuristic.

Your colours feel more "Drupal". I definitely like the colourful version. Might change it.

Probably should log:

- Response time
- Tokens
- moderation response
(Maybe Tokens / minute)

As you'll want to do tests and then model how long/ expensive something will be when it scales up

Right you could imagine some kind of debugg or prompt module that looks at a chain within AI Interpolator and then takes a single snapshot of the specific prompt that was sent to OpenAI. This snapshot is opened on a new page where all tokens, context etc are saved in stone but the specific prompt can be interated over to see how the LLM will respond. Once you have found the prompt you like you can bring it back into AI interpolator.

As its a developer tools if they developer sees the response as JSON (so it doesn't actually end up doing anything), that's fine.

I definitely think this needs to be resolved as interating over prompts is so important. Similarly if we do Agents or anything we need to be able to "undo" an LLM easily.

Current idea is:

AI_Automations - For the module name
AI_Automation - For the name of the entity that will include the Automation config

Alternative inspired by https://en.wikipedia.org/wiki/Automator_(macOS)

AI_Automators - module
AI_Automator - To describe a specific entity

Marcus_Johansson → credited yautja_cetanu → .

Marcus_Johansson → credited yautja_cetanu → .

Just a heads up, we're waiting until Wednesday for an official announcement and plan at this event: https://www.drupal.org/community/events/drupal-ai-meetup-2024-06-19 →

But the plan is to port this whole module to the AI module here: https://www.drupal.org/project/ai →
We have a plan to make this module significantly more production ready in the process.

However it will likely not be until the w/c 1st of July or 8th of July that we will make a start on this and it might take 1-2 weeks to finish. (Need to first port AI Interpolator), Whats your timeline on this as we might be able to help.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

Yeah agree looks great! It feels automation ey

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

I've been told that bad chunking of cutting things off is fine. But the solution is to start the next chunk slightly before the previous one ended.

yautja_cetanu → created an issue.

We've worked on things a great deal and particularly put a lot of effort into the ChatGPT block for Search API AI.

- You have to select which index you want to use
- You can select a "View" that the block is linked to. This doesn't change the index but instead limits which entities can be passed into chatgpt when you search.
- You can also select a specific entity to look at. If the block sees an entity id in the route it will only speak to that specific entity not the whole knowledge base.

When the reply uses all of ChatGPT instead of a specific context that is a sign that something is wrong with the index and contexts arn't being passed to chatGPT. If its working perfectly it should be restricted to pinecone data only.

Created an issue here: https://www.drupal.org/project/search_api_ai/issues/3375565 📌 Implement Weviate as a vector storage backend Active

yautja_cetanu → created an issue.

@yelban - We have looked into Weaviate and ran into a problem and submitted a patch to Weviate.

https://github.com/milvus-io/milvus/issues/25206 - We need this patch to be in a release

https://www.youtube.com/watch?v=wZACtTMHlJ0 - Created a video summarising where I am with the research currently

yautja_cetanu → created an issue.

Summary

In this thread ✨ Unify anonymous and registered users Active Sun proposed a potentially simpler method of only achieving an anonymous user API that helps with the comment module situation described in this issue #355513: Create an anonymous user API → . We have invested how both ideas could be achieved. Sun’s is in theory and our idea can be found both in the Party module and fubhy’s sandbox: http://drupal.org/sandbox/sumsi/1806794

After further analysis we think that Sun’s proposal could work but is a little more complicated than it appears at first. Our idea appears complicated but think it is actually simpler then initially appears and has more scope at assisting many contrib cases. Both proposals would be helpful and so we have gone into detail here.

Sun/ Damien Tournoud’s Idea

We keep users as they are but put a flag on the user of whether it is anonymous or authenticated. We then move all systems that deal with anonymous users to deal with users + anonymous flag. For example when a comment is input with some information such as name, website and e-mail it is saved on a user that is flagged as anonymous. This user will be treated in almost the same way bar some exceptions.

Dealing with Names and Passwords requires some extra code.

• You will have to force comment names to be unique. OR have to enable non-unique usernames at the database level for all users and enforce unique usernames at a higher level like with e-mail.
• You will have to set a random hidden password for anonymous users OR enable NULL passwords at the database level for all users and enforce that it is non-null at a higher level like with e-mails.
• In order for this to happen there are a number of situations where the anonymous flag on a user needs to be checked before something can happen.
• Every form that changes that edits or creates a user has to first check the flag before it can change the username or password to implement the restrictions that were previously at the database level.
• This code is shown below but it feels unpleasant

Change the Database scheme or not?

We think that changing the database schema seems to be the reasonable conclusion as making it completely impossible even in contrib to allow anonymous users to have non-unique names does not make sense. However due to #845774: Regression: Anonymous users can post comments in the name of registered users → this is arguably a good thing.

However if we want to allow anonymous users to be non-unique (And we definitely want them to not have a password) we need something like below

Example of how we would make an anonymous user API that deals with saving and updating usernames and passwords.

Wordy Code

Comment Module says I want to save a name on the user 
Thing - Takes this information and looks at the user, it it anyonmous or authenticated
Thing - Finds its anonymous
Things - Passes it to anonymous save
Anonymous Save - I will allow non-unique usersname
Saves the username

Code Snippet

// Comment module gets information about a comment from a form. In CommentStorageController

$user = user_create();
$user->name = “Cheese”
$user->save();

// $user->save() eventuall calls UserStorageController->save()

if ($user->isAnonymous()) {
  hook_presave	
  save normally, calling parent::save();
  hook_insert/update
}
else {
  db_select() // find if another user has the same name
  if (other user exists) {
    // reject username change
  } 
  if (user with same email) {
 	 //reject email
  }
  if (password is empty) {
  	 // reject password save.
  }
  // save entity after bad changes have been rejected.
}

Our Proposal
Our idea completely separates the user information from the information required to authenticate that user. Our idea puts the username as a unique property on the account entity and deals with the name of a user with our “nice name concept”.

Potential Performance Worries and our solution

Loading a user and an account from the from the database will be slower and more memory intensive than loading one record from one table.

So instead we have found a way of not needing to load both tables in the vast majority of cases. For example, using the concept of a name on the user entity (nice_name), our patch in the issue only loads the account on 2 occasions.

We add a ‘name’ field onto the user table that is essentially free text and non-unique so that it can take any value. When comments are saved they can create a user and write directly to the name field. When accounts are created they will write to both the account name (which is unique) and the user name which is not unique.

This allows us to maintain unique account names, but allow for non-unique comment-poster names. We think this is no worse than sun’s proposal, especially regarding performance but allows for contrib to much more easily take control of the display name of a user entity.

This has a bunch of further benefits

• The nice name is free to have duplicates, simplifying the problem of allowing multiple anonymous comments with the same name.
• The nice name can be used like a cache. This means name alteration using hook_user_format_name only happens on write => faster rendering of a user name => faster load times for pages with lots of user represented
• Accounts can set the name as the username to start off with.
• A Full name module can easily use fields from the profile module
• In Party we have built a tool that uses ctools plugins to set the entity label on our “Party” (which is essentially like the user). On every save it looks for suitable name in a variety of places and chooses the most appropriate.
• This allows fields that are not required (such as full name on a profile) to be used for the label
• This allows for labels to be constructed from multiple fields such as Name + Town.
• Saving the name allows name calculation to be fairly intensive without effecting the performance of a load operation.

It also opens up the potential for contrib to some really interesting things in the future

• Acquisitions - Much more easy to allow people to take control of their content upon registration #1816822: User Acquisitions - Accounts taking control of a previously created User → .
• Proper User Types #1816836: Different User Types - (Party Hats) →
• Pluggable authentication - Which we have said is separate.

Updated issue summary.