🇬🇧United Kingdom @yautja_cetanu

First thing:
Anmol
- Create a simple AI Agent flow with Tools API and ANYTHING as a tool.
- Create a Tool with the Tool API for editing / viewing the alt text on a media entity (Might be able to make use of existing Tools).
- Then try and add on layers of complexity (Like one agent for viewing and evaluating out of 10, and the second for editing if its too low, and now we have flow with an orchestrator that asks the evaluator first and decides to move on)
- Then see if we can have a PoC version with Guardrails working at all.
- Then make sure Guardrails is using a proper abstraction layer for multiple providers.

Guardrails:

• Make sure Guardrails stuff works from the AI Framework point of view.
• We want some level of abstraction so RAILs, OR amazon or another provider of guardrails can be used.
• We need a nice UX for how an end-user creating the flow pics where the guardrails is done and how its configured.
• There can be advanced configuration for the site admin behind the scenes, but the flow creator shouldn't have to think about things they wouldn't intuitively understand.
• We may have settings on how the flow responds to guardrails. Could be simply stop it, could be retries and then stops it, could be provides the option for user feedback to tweak the response (or approve it anyway), might want to have option to merely report in the moderation page and a human evaluates.
• The goal for this talk is to mention RAILS a bit, and show off the safety that guardrails allows, mention specifically that RAILS allow for sovriegn guardrails and the AI framework allows for multiple sources.

https://www.drupal.org/project/twig_to_sdc → - This might be relevant

TODO - Andrew - Create the actual site on Github for the demo.
- We might make a Sandbox module.

Using Flowdrop UI instead of pure FlowDrop

• Figure out if we use Modeller API
• FlowDrop when you click save, it actually saves things to Config itself (So could then be edited like a normal agent).
• Its a bit wierd but there are multiple content forms on one flow.
• We want to use Tool API not function calling (and how easy is it to just add a new tool)
• Flowdrop UI is ONLY for agents now so we need to remove workflow related AI elements (like no deterministic steps).
• Need to put effort into the "Tool" drawer so there aren't loads of unusable ones.
• Need to figure out how to handle the "ChatInput" so we can set up the "Assistant" that is used by the chatbot (Or chatconsumer)
• Some way of saving a flow and coming back to it (Where that is not in config right now?)
• Or can we take an existing Agent flow (like Drupal CMS ones) and the whole thing appears in flowdrop and just loads.
• In this situation flowdrop ui is just a UI, so it doesn't handle any triggers or flows.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

I've finding myself able to resuse these slides a lot: https://docs.google.com/presentation/d/1bidV5b7xYbReCPUX8wVXLGu25A6DI6jk...

This is what I gave over the weekend. Some are from Paul initially some are new and couple be probably made into nicer slides such as "What is MCP" as that comes up a lot. Perhaps my slides arn't quite what this issue is talking about. It's more about what the AI Initiative is doing.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

yautja_cetanu → created an issue.

It looks like there is a descision to make use of the Official Php MCP SDK which is lead by the Symfony AI team. We have had an initial conversation and have started contributing directly to the SDK.

We can close this issue when a proper roadmap is released.

Things we likely want to do:

Immediate term:

• Integrate the php MCP SDK with the MCP module.
• Get a tagged version of the MCP SDK.
• Integrate MCP with the Tools API module.
• Create the UI so that Drupal Tools can be exposed in MCP.
• Create a UI so that Drupal resources can be exposed via MCP (These might just be tools but the UI might be different as its more about selecting content_
• Create a simple recipe for potential inclusion into Drupal CMS
• MCP Sampling (Still don't fully understand this!)

Longer Term:

• Get MCP working with Oauth.
• Get an MCP Client working so Agents can make use of MCP, into the symfony AI framework (We have one with a different framework).

Note: Plan is to have a call soon showing the current PoC state

A dream PoC for me brings together:

• We are able to create the Agent flow using FlowDrop to do something truly Agentic (For example the Drupal CMS taxonomy agent but it also does support)
• The FlowDrop UI creates Agents using the existing AI Agent config agents so can be made to make all the other AI Agents.
• The tools are made using the Tool API not the existing Function Calling.
• We show how anything else made for the Tool API can seamlessly work with FlowDrop UI. (For example Publish Content Agent).
• The Agent Flow can be added to a Drupal CMS Chatbot (Using the Chat user?)
• The runner behind it is using Symfony Messenger so we are able to use true streaming and async.

Then next steps:

Some more complex examples and things that show off the power of async.

• We show that we can take a complex set of tools (Like Create Blog Post) and it appear as a single simple tool (So hiding the complexity of Entity Load, Field set, field set, Entity save).
• The Agent flow can be more than one agent (Like the Drupal CMS agents).
• We show the Agent's having their own permission.
• We have a Drupal CMS or Canvas AI flow that allows more than one agent to work in parrel.
• We show background/ Autonamous agents working using this.
• We should how the Agent workers can be pushed out to external machines.
• Anything cool with Logging.
• Eventually AI Agent runner that mixes deterministic and non-deterministic so normal flowdrop or ECA could be used

Changed the Title as trying to bring about a plan of everything the AI Initiative needs to do. In this meta issue we can seperate what is Tool specific and therefore available to everything else that uses tool such as MCP or ECA and what is AI Agent specific (which is moving to core AI module). However there is too much overlap that for now I think it would be better to have 1 meta issue for eveything to do with Tool.

If however, we finish everything in Tool and release a beta, but there is AI Agent work left to be done we could move that stuff out of this meta issue to close this one.

Moving this back to active:

• We have two PoC's for this here: ✨ Create PoC demo of the Background agents and SM approach. Active where we will be having a demo and video to show them off and link to them.
• We will require this modue: https://www.drupal.org/project/sm →
• We will likely present this to the Symfony AI team and see if some of this work should go into Symfony itself.

https://github.com/lussoluca/ai_react_agent

yautja_cetanu → created an issue.

Comments on prioritise (and what issues are related to what). (From Jamie's perspective)

- A nicer UI for create and adding tools to Agents (but this might not be in the tool module this might be better as an issue in the AI Agents or AI module). That is "Secure by design". This means the UX for tools is likely to be constrained to specific entities the user wants, and if the tool is supposed to work for everything thats an explicit descision (or maybe not supported).

- Composable Tools - I can have a nice simple looking tool which is "Create Blog Post" (The complexity of Entity Load, Field Set, Field Set and Entity Save is handled behind the scenes). (Though it would be possible to go into more detail and have a specific custom tool that has only a limited set of those field sets instead all by default).

- Derivative Tools.
From a generate tool such as "Publish Entity" or "Create Entity" I can make a new tool for a specific contrained version "Publish Blog Posts" "Create Articles" that only has permissions/ capabilities for those things.

- Tools are inherently tied to permissions and therefore Roles, and when Agents have roles, the tool will check that the Agent can use it AND the end user can use it (If one exists).

- Initial conversation with harivansh is that it might be easier to explore the Symfony Messenger work in the AI Agents contrib rather than do the move right away.
- We probably want to check with Luca and his work https://github.com/lussoluca/ai_react_agent to see what he thinks too.
- We should show this to Christopher
- Can we get a tagged release of ONLY the symfony AI Agents runner? Would Drupal CMS be ok with a release that has a stable module based on an "Experimental" library? (Could we even fork the componenet for now? Given the alternative is to write and maintain one from scratch anyway?

yautja_cetanu → created an issue.

Seconding https://www.drupal.org/project/ai_initiative →
It benefits from being quite good to experiment with. It would help as we could do our planning for AI issues accross the ecosystem even if about other modules.

https://www.drupal.org/project/ai →
https://www.drupal.org/project/ai_agents →

Would also like to opt into the Gitlab migration

It would be good to have a hook as well so that third party modules could override the advanced field

Looks good! My thoughts:

• Change the order, put the MVP Scope and Not in scope above the terminology discussion.
• I think we might want "Context Provider" plugins (which is alluded to in the section on extensibility). The Context Control Center would provide context but also YAML or JSON files provided by some modules might provide Context that is only editable by code (such as javascript libraries Canvas Code Component agents can use).
• I think we should seperate the roadmap into Context Control Center End user Features and another for Context AI Developer features.
• We don't have much time so maybe its ok, but I'm a little nervous our focus is on too many developer features and structure and not enough on the UX of end-users getting context into it. Could we at least have an issue for prototyping improved UX/UI? So for example we could have something where you upload a pdf and it automatically loads the context in, but its a demo that doesn't actually work (maybe slides or a video) as I'm nervous that as it stands there would be worry that no one would use it. We could pick University of Edinburgh's brand guideline website as an example?
• We had Knowledge Base in the Dries Note. Where does that fit (Just uploading a bunch of files to be indexed and used as context)

From a Dev Roadmap:

• - We want a Configurable with the Tools API Context Tool
• - Ability to use RAG to search for Context (both simple and advanced info mode) and degrade gracefully if not there.
• - Context provider sources as mentioned. Some can be Context Control Center, Some could be the knowledge base (though that could be part of CCC) some could be YAML or JSON files that are not overridable, some could be files that can be overridden and added to, and then external sources as mentioned.
• - Likely want this exposed via MCP as it will be invaluble for vibe coding with Drupal. Also might be good for Drupal Agents (like all Drupal Documentation for a Support Chatbot)

These were my notes from the meeting on Dev Roadmap stuff:

Context Meeting:

• Might be nice to have this as a seperate module to the Context Control Center.
• AI Context API - Aimed at developers and Site managers
• Context Control Center - Aimed at end-users and marketeers and totally non-technical non AI people.

End-user features:

Prioritisation decided by UX people

• Make it easier to bring in Context into the CCC. (Take a PDF or existing brand guidelines and it creates all the contexts)
• AI Agents that can help you automatically split Contexts into lots of granular context. For example a Brand Guideline might be seperated into:
• Global Voice
• Design
• Acronyms
• (Maybe we don't want to have settings of (Always use)
• Create an Issue for building a UX profile:
• Personas
• Use Case
• Workflows.
• It needs to be easy
• Works well with Canvas and easy to get to the context in Canvas (Probably need Markdown behind the scenes)

Context API related stuff
Essential Prior to CMS 2.0

• We need "Categories of Context" Some of it will be dev only and not in the CCC end-user UI
• We need to move away from Taxonomy (maybe a config entity type).
• We should create a Context "Tool" (Function) tool. This should allow the agent to pick the context when it wants. (Configuration on the tool could choose which kinds of context it wants to use)
• We need "Context for the Context" to use the same design pattern as the Components (So each Context has. Title, Description, that will appear in long lists, and then the AI can get all the context if it needs it.
• We want to explore "Global Context" that Agents just have it there.

Nice to have

• The context tool could be configured to force the context for something
• (Bring Permissions into Context, so specific roles can see certain types)

Less Important

• We want to explore the idea of Context Control Center being used for things other than Agents (Jamie thinks that should be low priority).
• (Being able to override all plugin descriptions) - This is kind of linked to tools API and what we want to do. (Could put it in Tool? Could be its own thing)
• Maybe Developer Context UI for managing ALL contexts

yautja_cetanu → created an issue.

Context Meeting

Might be nice to have this as a separate module to the Context Control Center.

• AI Context API – Aimed at developers and Site managers
• Context Control Center – Aimed at end-users and marketers, totally non-technical and non-AI people

End-user features

• Prioritisation decided by UX people
• Make it easier to bring in Context into the CCC (e.g., take a PDF or existing brand guidelines and it creates all the contexts)
• AI Agents that can help automatically split Contexts into granular units, for example:
  • Global Voice
  • Design
  • Acronyms
• (Maybe we don't want to have settings of "Always use")
• Create an issue for building a UX profile:
  • Personas
  • Use Case
  • Workflows
• It needs to be easy
• Works well with Canvas and easy to access context in Canvas
• (Probably need Markdown behind the scenes)

Context API related stuff

• Essential prior to CMS 2.0
• We need "Categories of Context"
  • Some of it will be dev-only and not in the CCC end-user UI
  • We need to move away from Taxonomy (maybe a config entity type)
• Create a Context "Tool" (Function tool)
  • This should allow the agent to pick the context when it wants
  • Configuration on the tool could define which kinds of context it wants to use
• We need "Context for the Context" – should follow the same design pattern as Components:
  • Each Context has: Title, Description
  • Appears in long lists
  • AI can retrieve the full context when needed
• We want to explore "Global Context" that Agents just always have

Nice to have

• The context tool could be configured to force the context for something
• Bring Permissions into Context, so specific roles can see certain types

Less Important

• Explore using Context Control Center for things other than Agents (Jamie thinks this should be low priority)
• Ability to override all plugin descriptions
  • Linked to Tools API
  • Could be in the Tool or its own thing
• Maybe a Developer Context UI for managing ALL contexts

Another thing to think about is the SM module to use symfony messenger to handle async communication with MCP but still in Drupal.

yautja_cetanu → created an issue.

https://www.drupal.org/project/canvas/issues/3541783 🌱 [Meta] Create an AI Powered Design System PoC Active - Might be a duplicate of this.

First step is the classification: https://www.drupal.org/project/ai_recipe_image_classification →

We then need:

• AI Search tool that uses this to allow AI to find the images.
• It will need graceful fallback. So check RAG first and then if not use traditional search.
• It isn't a simple fall back as AI will have to query the second in a different way knowing that it isn't semantic search

- This means Canvas AI might require 2 additional Recipies (Although tbh I think image classification and AI Search for images should be one recipe.)
- It will require Search API at the very least.

Given that this will be using AI Agents to do things. It could be in Drupal instead of Drush and use AI Drush agents to do things instead.

Duplicate of 📌 Simple approach to bringing advanced metadata into Canvas AI Active