πŸ‡±πŸ‡ΉLithuania @mindaugasd

Account created on 4 July 2008, about 16 years ago
#

Merge Requests

Recent comments

πŸ‡±πŸ‡ΉLithuania mindaugasd

That is one of the reasons AI abstraction layer was created.

Drupal being a truly open project has to ensure users can choose which AI service they want to use instead of depending (hard-coding) a single one, therefore Drupal needs an LLM abstraction layer separating and decoupling Drupal features from AI services, enabling to quickly switch service provider when needed without any hassle.

More at πŸ“Œ Create LLM abstraction layer Active

I think AI module is well place to be part of Drupal CMS within starshot initiative, so module has to be not too much bloated and bundled with features people most commonly use, and also solid features prepared to be in Drupal CMS.

More at ✨ Create AI ecosystem "add-ons" page Active

In general, people show demand to experiment with local AI. So if there is demand, it could be included.
[...]
Another question is how to make it easy enough and accessible to regular Drupal CMS users. How does one install it on the server actually. How much knowledge, investment and experience does it require to do it in the proper way.

More at πŸ“Œ Ollama LLM Provider Active with actual experiments/process to run it,

As per πŸ“Œ [meta] Discussion: what LLM providers to include Active AI module currently have support for these options to run local or open AI:
Ollama πŸ“Œ Ollama LLM Provider Active , LM Studio πŸ“Œ LM Studio LLM Provider Active , Hugging face πŸ“Œ Huggingface LLM Provider Active , discussion about jan.ai πŸ’¬ Can a module be provided for use with Jan.ai as a provider? Closed: won't fix .

πŸ‡±πŸ‡ΉLithuania mindaugasd

New API integration is https://www.drupal.org/project/ai β†’ , which enforces implementations to use key module by default :-)

πŸ‡±πŸ‡ΉLithuania mindaugasd

"what modules should I use to accomplish X?" - and I think it would be a great thing to bolt on to PB someday, maybe

For this particular feature, FreelyGive team have built a working demonstration here: http://project-browser.freelygive.io/

πŸ‡±πŸ‡ΉLithuania mindaugasd

Changed the title, because module can already be used with inline entity form, but not edit prompt in detail.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Exact use from https://git.drupalcode.org/project/ai/-/blob/chat-stream/modules/ai_api_...

  if (is_object($response) && $response instanceof StreamedChatMessageIteratorInterface) {
      $http_response = new StreamedResponse();
      $http_response->setCallback(function () use ($response, $code) {
        foreach ($response as $key => $chat_message) {
          if ($chat_message->getRole() && !$key) {
            echo '<h4>Role: ' . $chat_message->getRole() . "</h4><p>";
          }
          echo $chat_message->getText();
          ob_flush();
          flush();
        }
        echo $code;
        ob_flush();
        flush();
      });
      $form_state->setResponse($http_response);
πŸ‡±πŸ‡ΉLithuania mindaugasd

If using default toolbar or using new drupal experimental navigation, additional menu items are not accessible, because they don't have a common route yet.
And those menu items may never be exposed with new navigation, because it max. goes 3 level deep.

πŸ‡±πŸ‡ΉLithuania mindaugasd

end up with menu iitems to 20 explorers

DX is more important than UX

20 menu items are both bad DX and UX.

knowing that you're missing providers is also super valuable for developers

Yes, "Explorers" is to explore, so discovery of new things should be programmed in.

Currently routes are inconsistent (alpha3):

  • /admin/config/ai/development/chat-generation
  • /admin/config/development/ai-image-generation
  • etc.

They could be changed to:

  • /admin/config/ai/explore/chat
  • /admin/config/ai/explore/image
  • /admin/config/ai/explore/narration
  • /admin/config/ai/explore/transcription
  • /admin/config/ai/explore/embeddings
  • /admin/config/ai/explore/moderation
  • etc.

Menu item "Overview" does not work yet. New path could be introduced for it:
admin/config/ai/explore

User could be greeted there with all operations grouped into categories, full list.
Maybe also choice of 'providers' could be listed under each category.

Menu could be changed:

  • Chat
  • Image
  • Overview / All explorers / Discover more

When other explorers will be used frequently, all of these options can exist at the same time:

  • Drupal already has "Shortcuts" feature to star frequently visited pages.
  • User can always create menu item manually.
  • Implement Option 2. " add a settings where you can enable/disable them", maybe even under overview page there can be a "star" to click to expose particular explorer as a separate menu item, or in general configuration when user finds it at some point.

Plus I grouped similar explorers together. For example, all these operations listed here 🌱 Discuss: Interface suggestion for image to image operator type. Active are good to have under the same page to explore.

Since 3 menu items are too little, there can be more - up to 7 can be added (by default in settings) depending what we see are of most frequent use.

πŸ‡±πŸ‡ΉLithuania mindaugasd

It is incorrect code change on multiple levels...

πŸ‡±πŸ‡ΉLithuania mindaugasd

mindaugasd β†’ created an issue.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Fixed the issue about prompt entity context.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Automator Instruction Config Entities

or just "automator".

and then maybe in code they are "AI_Automator.Types". or something

Code and reality preferably has to match.

In reality, people will just call automations and automators, like I demonstrated in #7 comment. So are additional words like "instruction" needed?

πŸ‡±πŸ‡ΉLithuania mindaugasd

Red - automation
Green - automator
Purple - automator type
Yellow - automator setting

looks complete from my side and I don't have further opinion, because my understanding of interpolator architecture and improvements is patchy.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Related with issue on openai module πŸ› Misleading error: You exceeded your current quota, please check your plan and billing details Active
In that case I think openai service was returning the error wrong

πŸ‡±πŸ‡ΉLithuania mindaugasd

If all is setup, maybe just try again later?
OpenAI service can sometimes return false errors.
We have a new issue here about this: ✨ Introduce AiQuotaException Active

πŸ‡±πŸ‡ΉLithuania mindaugasd

The question is where it would be used as you write

My first wonder was how extensive functionality it will be.

Actually developing this takes 30 minutes. And arguing for it being an AI functionality is very easy

My second wonder if it will be used is indeed dull one. I am convinced, it should it done. On the other hand, like in recent years, in the future we can expect AI to continue get better, and this separation line between translation and AI will continue to blur even more, but for now, it still make sense.

πŸ“Œ | AI | Add Readme
πŸ‡±πŸ‡ΉLithuania mindaugasd

@wouters_f

done is better than perfect

Having documentation looking as https://ecaguide.org/ or https://docs.drupalcommerce.org/v2/developer-guide/ is very attractive, when documentation size grows.

Readme files alone would become difficult to navigate, but with https://www.mkdocs.org/ md will still be available.

MkDocs is a fast, simple and downright gorgeous static site generator that's geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file.

When linking between pages in the documentation you can simply use the regular Markdown linking syntax, including the relative path to the Markdown document you wish to link to.

Interpolator have very good documentation https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... β†’ which can be ported to md files git version controlled. It will take clicking "Edit" and merge on drupalcode.org to edit something for someone in docs.

have to do 3 repos

I would prefer one.

The MD files are a minimal security concern, where you at best can figure out older unsecured versions

If that is all, then it is fine.

πŸ“Œ | AI | Add Readme
πŸ‡±πŸ‡ΉLithuania mindaugasd

I guess it mean nothing. Modules don't have it, but js libraries do have it, which is the same.

πŸ“Œ | AI | Add Readme
πŸ‡±πŸ‡ΉLithuania mindaugasd

Of course this will then lead to 100s of MD files inside the AI module and its submodule, but I think people can live with 1-2 MB of md files - any proper production setup should use build jobs to remove them (and test files) before being pushed to production anyway.

Modules usually don't have 100s of MD files. This is new thing. What does it mean of having it now? And those files will be there, not removed. Future Drupal is to install and use easily without production setup.

πŸ‡±πŸ‡ΉLithuania mindaugasd

More practical examples were people would use it?

πŸ‡±πŸ‡ΉLithuania mindaugasd

I used to do translations chaining quite a lot, because AI models were not good in Lithuanian and unusable, but today, as we see, AI models do chaining by themselves behind the scenes.

πŸ‡±πŸ‡ΉLithuania mindaugasd

A common case can be a chain: translate to English before doing TextToImage, because textToImage models are not multi-lingual.

I just tested dall-e-3 with languages.
It converted a prompt to another prompt behind the scenes.

  • If I write "KΔ—dΔ— ant stalo" in Lithuanian it translates by itself behind the scenes to English "Chair on the table"
  • But if I write "Chain on the table" it converts behind the scenes to "A wooden chair with intricately carved details on the backrest is precariously balanced on..."

Providing it in English is still better to begin with. All more important for Stable Diffusion.

But still regular "chat" type can translate this fine as already coded in ai_translations module.

πŸ‡±πŸ‡ΉLithuania mindaugasd

I am reopening the issue, because according to this page https://www.drupal.org/docs/drupal-apis/plugin-api/attribute-based-plugins β†’ Drupal 10.2 also supports attributes.
I no longer need this myself, but there was another question on slack.

πŸ‡±πŸ‡ΉLithuania mindaugasd

The scope is probably:

  • automator string in, string out
  • ✨ ai_translate submodule Needs work for translating entities. But entities can have many fields to translate.
πŸ‡±πŸ‡ΉLithuania mindaugasd

Unclear what is the goal.
Can this be proper abstraction without detailed research about translation services?
What and why is within scope of AI module?
For example, I used translations in array a lot, translating 1000 sentences in a batch for example.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Latest code

protected function getAIPromptEntity()

is incorrect, because it tries to determine aiprompt entity from route, but prompt can be loaded on any route, not just admin pages.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Rewrote description. Added new information about:

  • Plugin system for arguments
  • Separate routes and forms for add / edit / remove operations

Will follow same/similar pattern like prompt segments.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Added modal approach to the list, which may be the best one.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Added information about adding user interface for pre-defining arguments.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Adding related issue.
Both issues are about inline prompt editing.
The difference is storage mechanism. This issue remains entity storage, while another issue introduces new field storage.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Alternative is after implementing ✨ Implement conditions for prompt segments Active , create a condition which would disable related plugin if some other segment has no output (empty).

πŸ‡±πŸ‡ΉLithuania mindaugasd

Added info about API changes introducing new methods.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Updated issue description with more details.

πŸ‡±πŸ‡ΉLithuania mindaugasd

One more alternative is to use entity fields for this.

Workflow:

  1. User creates a new entity field defining a variable and its defaults
  2. Developer programatically set value of entity field to pass the variable

Benefits:

  • It already works, nothing to be done.

Drawbacks:

  • A little more complex for the user, because not fine-tuned for prompt engineering
  • Would not work with configuration prompts, because they don't have fields
πŸ‡±πŸ‡ΉLithuania mindaugasd

Another more complex alternative is to have a separate table where user would pre-define what variables this prompt accepts.

There would be a new table below segments table, with columns:

  • variable_name
  • type (string/integer/boolean/etc)
  • default_value
  • description
  • [Button "Remove"]

Below a table, a button to "Add new row".

Benefits of this approach:

  • Having variables definition would make clear what variables can be provided and what are available. Well defined tokens could be generated accordingly.
  • Having default values means arguments are always useful (and can be used and referenced) and prompt is always usable with these defaults
πŸ‡±πŸ‡ΉLithuania mindaugasd

How it would look like rewording this issue πŸ“Œ First Version of AI Automator Active

Move over AI Interpolator to AI Automator

* Make it possible to create multiple rules per field.
* Make the rules work with the AI abstraction layer for all generative AI stuff.
* Build those rules into the core module.
* Make it possible to fill meta values (summary on text with summary, alt text on images etc.)

Renamed:

Move over AI Interpolator to AI Automator

* Make it possible to create multiple automators per field.
* Make automators work with the AI abstraction layer for all generative AI stuff.
* Build those automators into the core module.
* Make it possible to fill meta values (summary on text with summary, alt text on images etc.)

So rule would be automator.
And specific rule type would be automator type.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Do you think we should do away with "Rules" as a word entirely then?

As from someone who is not too familiar with interpolator, word "rules" does not say or mean much.
While "Automator type" is immediately clear that is a type of automator, while "rule" is meaningless.

Except that "entity type" exist within Drupal, so a little confusion exists still.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Red - automation
Green - automator
Purple - automator type
Yellow - automator setting

πŸ‡±πŸ‡ΉLithuania mindaugasd

Red - automator entity
Green - automator
Purple - automator type
Yellow - automator setting

But red and purple mix, because there is "entity type".

πŸ‡±πŸ‡ΉLithuania mindaugasd

Related alternative idea.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Hi,

I just tested with simplytest.me and added steps to issue description.

πŸ‡±πŸ‡ΉLithuania mindaugasd

Also throw an error that deprecated function is still used somewhere.

  public function deprecatedMethod() {
    trigger_error('Method deprecatedMethod() is deprecated. Use newMethod() instead.', E_USER_WARNING);
  }
πŸ‡±πŸ‡ΉLithuania mindaugasd

That function was replaced with a different one.

When deprecating/replacing a function, both methods can exist at the same time for a while and old method being marked "deprecated", instead of removing it completely.

Production build 0.69.0 2024