Exploring AI Prompt management within AI core

Getting there; now there is a form element you can just change #textarea to #ai_prompt_tableselect, was thinking two steps needed as a result:

1. Update hook to convert whatever user has created as textarea value already; this can be a pretty simple helper class to let devs easily update to this element. Documentation can be very straightforward for this.
2. Allow setting a unique prompt usage ID as an attribute so it auto-creates a default/recommended prompt (and maybe event prompt type) that gets deletion protected/edit protected (ie, controlled in code by the module)

E.g.

$form['my_prompt'] = [
  '#title' => $this->t('Choose a prompt to run'),
  '#type' => 'ai_prompt_tableselect', // This is the custom form element to do the heavy lifting.
  '#prompt_types' => ['test_type'], // These are the allowed types.
  
  // Optionally control a default prompt in code.
  '#default_prompt' => [
    'prompt' => 'My prompt here: {INPUT}',
    'id' => 'default_prompt_for_my_prompt',
    'type' => 'default_prompt_for_my_prompt_type',
  ],
  
  // Optionally manage new type in code.
  '#default_type' => [
    'id' => 'default_prompt_for_my_prompt_type',
    'variables' => ...,
    'tokens' => ...,
  ],
];

If a developer wants to change default id or type they'd be responsible for cleaning up old auto-created ones.
If they just change the default prompt, as long as the ID remains the same, everything else can be auto-updated.

Screenshot of implementation example can be found in AI Content Suggestions sub-module (one prompt changed, one not):

Lots of todo's still e.g. around update hooks, plus I am sure many follow-ups like:

1. Use @mention style editor?
2. Inline form creation of prompt rather than new window and return on save (its fine, but ajax create would be more slick)
3. Helpers to insert suggested variables/tokens into the prompt edit form
4. Token browser? Not sure... it could be misleading showing plenty of things not available... maybe opt-in for a prompt type
5. Views for the list of prompt types + prompts rather than list builder so exposed filters can be added?)

This is awesome @scott_euser!

I don't have that much to add, the things I can think off, but they are also part of possible future todos (and not that thought through yet):

1. Some places will have a lot of prompts, so we might need an ajax pager on the table select.
2. Allowing twig (with wrapper around tokens). There are places where conditionals or traversing of the variables in prompt is really nice to have.
3. Minor and I might even be wrong that its needed - but if its not working with ECA, have a simpler fallback form element that works in BPMN.io, so it works at least with selecting inside ECA.
4. Mention style editor would be awesome as you mention - when working with technical writers/editors etc, they always found tokens to hard to work with.

If you need help with the editor or something else, let me know.

There is another discussion to be had here I think, and that is that we are now adding configurable agents in AI Agents, which is basically a system prompt, some metadata and (optional) tools. You can see the entity here: https://git.drupalcode.org/project/ai_agents/-/blob/configurable-agents/....

The question is if it makes sense to merge efforts here, or just have the agents extend this entity type.

Thanks for the feedback! In general status update: I'm working on the inline adding (similar to inline entity form), hopefully will get that in here later this week.

Some places will have a lot of prompts, so we might need an ajax pager on the table select.

Good shout thanks

Allowing twig (with wrapper around tokens). There are places where conditionals or traversing of the variables in prompt is really nice to have.
Sure, do you have an example where you have used twig approach?

Minor and I might even be wrong that its needed - but if its not working with ECA, have a simpler fallback form element that works in BPMN.io, so it works at least with selecting inside ECA.

How might I test if it works with ECA? I imagine it should work anywhere given its ultimately just a form element like any other.

Mention style editor would be awesome as you mention - when working with technical writers/editors etc, they always found tokens to hard to work with.

If you need help with the editor or something else, let me know.

Thanks! Maybe the mentions bit could be really helpful once this draft is ready.

we are now adding configurable agents in AI Agents

I think it makes sense to just swap out the textarea in AI Agents with the prompt once this one lands?

Sure, do you have an example where you have used twig approach?

I have from earlier companies I worked with the Automators, I can't expose there exact prompts, but something like:

Could you write an explanation for each of these rating categories and description of this tweet, how well it aligns with the company statement for {{ company_name }} given below.

{% if image %}
An image was also given in the tweet, make sure to look at before you take your decision.
{% endif %}

The following rating categories are important for this content:
{% for category in categories %}
* {{ category.name }} - {{ category.description }}
{% endfor %}

The company statement is:
{{ company_statements[company_id] }}

How might I test if it works with ECA? I imagine it should work anywhere given its ultimately just a form element like any other.

BPMN.io has limited forms that works currently, specifically around ajax functionality. Autocomplete won't work for instance when I was testing integration with the Automators. And since BPMN.io is the main tool being used for controlling ECA, we should test or set it up so it works. You can see an discussion here for instance: https://drupal.slack.com/archives/C0287U62CSG/p1740848222472139?thread_t...

I suppose on the twig side of things, given its just a textarea, the provider of that AI Prompt can decide to render it as twig. Optionally we could add helpers to help them render as twig, like part of prompt manager service.

Its a bit dangerous as evaluating as twig needs to be properly sandboxed I think... unless its maybe an option in the tableselect like 'Use the twig template for this type' (if the AI Prompt Type has opted into that), and if so, the twig template provided by the given module is used + the user can copy that twig template into their theme or own module to override.

This will be a game changer I guess, as now a lot of operations will be done with AI and having a place where the prompt can be found easily is very important if you don't know from where to start.

I have a small concern about the storage though. Maybe until it is not too late, it is better to use the content entity? There will be more flexibility, and also the machine name as id will be harder to make unique when the number of prompts will increase with time. And of course with content entity it will be easier to track who and when created/updated the prompt, maybe even have some revisions (as while you are in a search of a good prompt, it could be that you have a couple of tries and maybe it is good idea that you can come back to some nice prompt that was 2-3 attempts before) and of course it would be easier to have some access restrictions. I understand, that this will not allow to easily create prompts on module installation like it can be done with config objects, but there are other ways to provide default content, especially with recipes.

What do you think?

Config entities can before more like content with Config Ignore but the other way around it's much harder to make content deployable. Same for machine names vs IDs, much harder to make deployable target to UUID from content. All prompts are currently already config (at least in AI suite of modules), so making the config target content also problematic (and we will have a lot of unhappy developers if config export from production leads to changes).

Contrib/Core have largely not had such machine name issues with vocab names or entity type clashes, and we could easily advise people to prefix with module name or abbreviation for example.

This matches for example Webform: commonly people want to have forms deployable and therefore it's config entity by default but it's clear (and even in suggested ignore) how to use Config Ignore to let site builders give content management control over all or some webforms (e.g. wildcard ignore webform.* but unignore eg !webform.specific_form to make just it deployable.

Just a note for a follo up issue on this - I think as a second step we should look into creating MCP Plugins for this, to expose the prompts you want to expose via MCP. See the specification here: https://modelcontextprotocol.io/docs/concepts/prompts

I've yet to test out the actual prompt part of MCP, but my understanding is that you can give example prompts for instance for how to invoke multiple tools in the right order, when using something like Roo Code or Cursor or give suggested prompts in general to the end user.

Re #9 I have updated it so each prompt gets stored like ai.ai_prompt.my_prompt_type.my_prompt.yml so its easy to use Config Ignore like `ai.ai_prompt.my_prompt_type.*` + updated docs page to also note how Config Ignore allows you to make a specific property within a config (e.g. a Settings form in your module) be content manageable yet everything else remain as config.

Re #11 I think I don't quite know enough about that yet to know if that causes any fundamental changes to this to be considered? Any concerns Marcus, or is that more a subsequent bridge between this and MCP and not really changing the AI Prompts themselves?

Okay I think this is ready for review now. Follow-ups then needed for

1. autocomplete
2. @marcus_johansson's comment about supporting Twig
3. @marcus_johansson's comment about making available for use in MCP

I don't know why functional JS test not running in 10.4 - runs perfectly fine in my 10.4 local, but its getting stuck at the simplest thing in gitlab CI (ie, open a form and save it, without any ajax or anything yet). Maybe someone can spot the issue with that...

To review, best to read the merge request's update to docs

The branch was out of date, and appears to have originally been split from 1.0 but is being tested against 1.1. I can see you've updated it, but I wonder if that's what's causing the seemingly random test fail? I can't test because the patch version of the MR doesn't apply to the current 1.1 dev branch, but I don't want to break anything by attempting the rebase.

The code itself doesn't look to have any major issues, and seems like a good way of handling a very complex idea.

scott_euser → changed the visibility of the branch 1.0.x to hidden.

Thanks for checking! Updated with latest from 1.1.x (which introduced 🐛 Field #parents of all AI Content Suggestions are now incorrect Active ).

Otherwise this is back to ready for review. The test failure is on the simplest thing; saving of initial checkbox in form without even yet starting to test the Element, so I think its more a test runner issue, perhaps due to our custom implementation of gitlab-ci.yml to handle the AI Search with FFI extension... I think better I mark the test as skipped.

Okay marked as skipped for now, let's see others opinions on it, but I'm thinking not worth the energy - its the only Functional JavaScript test we have (thus far)