Add processor and field widget assist plugin system to AI Content Suggestions

Created on 5 June 2025, 28 days ago

Problem/Motivation

Currently we have modules like AI Content Suggestions and AI Image Alt Text, that has only one purpose - on a widget, take some input, process via AI and output something. While this does show case what is possible with AI, it does not suite actual business needs.

Alt text generation needs context from the article it is on, a Title suggesion might be based on an image in the article or some external source. In the end this is just field generation

We need something that abstracts the processor and abstracts the field widget assist plugin system, so that any module can:

  • Create a processor that can take custom input, processes it and output in a structure way that a field form widget requires.
  • Create a extra assist widgets that can be attached to specific field widgets in a specific state.

This will then open up for processors like ECA, AI Automators and AI Agents to be able to have custom workflows or agents to process the entity and fill out form fields on entities. And field modules like Address module can expose their own Extra assist widgets.

Since everything will be configuration, this means that we can ship any business specific widget solution as recipes!

A initial version was created here for AI Automators, but we should make it more generic: https://git.drupalcode.org/project/ai/-/tree/ai-automators-widget?ref_ty...

Proposed resolution

These are just suggestions on how to solve it.

In AI Content Suggestions create a processor plugin that:

  • has an attribute that takes Title, Description and something like "Can give Suggestions"
  • has an interface that is using ConfigurableInterface and ExecutableInterface
  • has a method that can flag if the processor can handle a specific output/field widget assist plugin
  • has a method that can pre-validate and send form validation errors
  • has a method that can take the input - input can be a ContentEntityInterface or prompt for now? (TBD)
  • has a method that can take the entity type and bundle before loading the config form on new processors or be set via form_state
  • has a method that can get either the fieldconfigurationinterface and optional delta of the output field before the execute or some array/data structure representing it. Delta is being used when a button is attached to a multifield, for instance alt text generator where you only want to generate text for the delta.

In AI Content Suggestions create a field widget assist plugin that:

  • has an attribute that takes Title, Description, Field Type and Field Widget Types (array)
  • has a method for getting field type and field widget types
  • has a abstract base class with a helper method to call the processor of choice that takes form state and changes the values.
  • this can be overriden for more complex returns that requires an AjaxResponse return type
  • has a ajaxCallback method that runs when the button is triggered that triggers the helper method to process that returns the value
  • Maybe the base class has a formAlter method to actually setup the button, since it might need some specific solutions on certain fields, so it can be overriden and even take form state, for things like knowing that an alt text is uploaded

To setup:

  • Alters the form display form of the details of the field widget so the processors that has flagged that they handle the field type becomes a choice and after you picked a processor it calls via ajax to get the subform of the processor.
  • Stores these settings in third party settings (TBD) - we have to add the plugin deriver modules as dependencies then
  • Multiple field widget assists items can be added to a specific field - think generate image and then write alt text

To run:

  • Alters the entity form for any content element and adds the form button.
  • On click triggers the ajax callback that triggers the processor.
  • Returns via form_state change or AjaxResponse
  • (TBD) How to handle return type for suggestions
✨ Feature request
Status

Active

Version

1.2

Component

Code

Created by

πŸ‡©πŸ‡ͺGermany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @marcus_johansson
  • πŸ‡©πŸ‡ͺGermany marcus_johansson

    This is very high level for now to discuss and have a common understanding of what and how to solve this.

  • πŸ‡©πŸ‡ͺGermany a.dmitriiev

    a.dmitriiev β†’ made their first commit to this issue’s fork.

  • Merge request !665Resolve #3528684 "Add processor and" β†’ (Merged) created by a.dmitriiev
  • πŸ‡©πŸ‡ͺGermany marcus_johansson

    I have tested adding an ugly version of a text automator - I will fix better code when we are getting close to finishing this issue and will only write down architecture problems for now, code, features, UX/UI improvements we can focus on later. In general its very impressive on such short time.

    1. I'm not sure about the correct architecture on how to solve this, but right now the system only allows one field widget per plugin. In the automators case, you might have 3 different automators doing some minor differences and want to set them all up. I would suggest that there is an "Add action" button and you can add as many as you want that are derivatives - even from the same plugin. With content suggestion this would also work - since you might have two different base prompts you want to run depending on the content you are writing.

    2. In the automators case we might have the plugin, but no setup automator. It would be good to have an isEnabled/isAvailable method that returns true in the abstract, but where a processor can hook in and say that it should not show up in the form display settings.

    3. Does it work currently to have one button to generate multiple values on a multi field? Right now the logic for the button is running inside hook_field_widget_single_element_form_alter, so you need to create another field and then click the button to ask for another value. For a text field that means that you can have doublets, since it might generate the same answer, if its not aware of the other values.

  • πŸ‡©πŸ‡ͺGermany a.dmitriiev

    I have changed the UI/UX in the way that it is possible to add multiple instances of the same plugin. It is also now possible to remove and sort field widget actions in 3rd party settings form.

    I have added `isAvalailable` method to base class, so that each plugin can perform any checks needed, for example user permissions, or whether some module enabled or not, etc.

    For the 3 point, I have added the new multiple property to plugin definition. In case it is TRUE, the button will be added to each single element of the field widget, if FALSE, only one button for widget will be added. TRUE is by default.

  • πŸ‡©πŸ‡ͺGermany marcus_johansson

    I have checked the code now, without having to tried it out with the Automator. It looks nice! What I would try to do next week is to make a more complex field like an image generation based on multiple fields and see how weel that work.

  • πŸ‡©πŸ‡ͺGermany marcus_johansson

    So, I have merged with 1.2.x and also fixed some minor cspell, phphcs etc. issues. I did remove a hook from content suggestions, that I assume have moved to Field Widget Action.

    There are a lot of things in terms of UX/UI implementation we can do better with helper functions, ready made css/js etc, and I could find nitpicks, like the plugin should not show up if all widget isAvailable returns false etc. - but the most important part - the architecture is solid. Really great work!

    This means that its better that we merge now and write follow up issues. I also never liked the thing in Drupal where one huge issue like this get 1 credit, and fixing a misspelling gets 1 credit, but that is a dicussion for another day :)

    I will merge it, its not a finished product yet - if nothing else we should add a lot of automator widgets to it, but it is working and its good enough to start testing and adding follow up issues for.

  • πŸ‡©πŸ‡ͺGermany a.dmitriiev

    Thank you, @marcus_johansson! Now it would be nice if others join and create more plugin implementations in their modules. This will broaden the usage and of course testing. It is worth mentioning that this is not an AI specific feature, any task that can be done with a click of a button fits here :)

    By the way, there is already an example plugin in this issue ✨ Add field widget action plugin based on agents Active that does the similar thing like https://www.drupal.org/project/ai_image_alt_text β†’ module with Field Widget Actions and Agent.

  • πŸ‡ΊπŸ‡ΈUnited States Kristen Pol Santa Cruz, CA, USA

    We are doing some issue management housekeeping and adding/removing components.

    We are removing the "Code" component and want people to categorize issues with the best module/submodule component.

    Moving this issue to "AI Content Suggestions".

    See πŸ“Œ Update AI module project components Active for more details.

Production build 0.71.5 2024