🇮🇳India @Akhil Babu

akhil babu → created an issue.

Feedbacks addressed. Please review

Thanks for the patch @aalin. I have included some visibility settings as well so that taxonomy term related form elements wont be there when user select's 'Language' option

akhil babu → made their first commit to this issue’s fork.

Currently, only the last two messages are sent as history to the orchestrator, which affects the overall user experience.

See the attached videos for reference. When asked to create a CTA for a tech journal website, the orchestrator asks whether a code component should be created or an existing component should be added (meaning the page builder should be used). When asked to create a code component, the component agent fails. So I asked the agent to "add an existing component then." The agent then responds with:
"Please let me know what type of existing component or section you would like to add to your page, and any specific details or placements you have in mind."
This happens because it doesn’t have access to the complete chat history.

Another issue is that intermediate status messages from sub-agents are also included in the history. For example:

When you ask the agent to "Create a button code component with a red background," the component agent is invoked. Once the component is created, it outputs:
"I have created a new 'Red Button' component with customizable button text and a red background. Let me know if you need any further adjustments!"

When the orchestrator receives this message, it also outputs something like:
"A new 'Red Button' component has been created with customizable button text and a red background. You can now use and adjust the button text as needed."

Now, when the user asks to change the color to blue, the message history sent to the orchestrator looks like this:
This includes the outputs of both the component agent and the orchestrator, but not the initial user message.

[
  {
    "role": "ai",
    "html": "<div style=\"margin-top: 10px;\"><div style=\"display: flex; align-items: center; padding: 8px; background-color: white;\">\n        <span style=\"margin-right: 8px;\"><span class=\"aiCompletedIcon\"></span></span>\n        <span style=\"font-weight: 400;\">Thinking</span>\n      </div><div style=\"padding: 8px; background-color: white; font-size: 14px; line-height: 1.26;\">\n          I have created a new \"Red Button\" component with a customizable button text and a red background. Let me know if you need any further adjustments!\n        </div><div style=\"display: flex; align-items: center; padding: 8px; background-color: white;\">\n        <span style=\"margin-right: 8px;\"><span class=\"aiCompletedIcon\"></span></span>\n        <span style=\"font-weight: 400;\">Generate a component</span>\n      </div><div style=\"padding: 8px; background-color: white; font-size: 14px; line-height: 1.26;\">\n          A new \"Red Button\" component has been created with a customizable button text and a red background. You can now use and adjust the button text as needed.\n        </div></div>"
  },
  {
    "role": "ai",
    "text": "A new \"Red Button\" component has been created with a customizable button text and a red background. You can now use and adjust the button text as needed."
  },
  {
    "role": "user",
    "text": "change its color to blue"
  }
]

I think the sub-agent output can be ignored in the history since the orchestrator already returns a similar message. Instead, the history should look like this:

[
  {
    "role": "user",
    "text": "Add a button with red color"
  },
  {
    "role": "ai",
    "text": "A new 'Red Button' component has been created with a customizable button text and a red background. You can now use and adjust the button text as needed."
  },
  {
    "role": "user",
    "text": "change its color to blue"
  }
]

Here is a Demo of how it is working now

HI All, We are planning a webinar on 29 October 2025, to introduce Drupal AI to a wider audience and encourage contributions. This issue already has an existing patch, but its been in active state for 3 months. I am assigning it to myself so I can test the patch, make any necessary corrections, and raise a Merge Request during the session.

Updated the orchestrator prompt to fix #57. While testing it, I observed another issue

I am using the OpenAI provider to test the prompt:
“Create a header for a restaurant site with a logo, search bar, and main navigation menu.”

The GPT-4.1 model often doesn’t delegate the task to the corresponding sub-agent. It simply says “Your request will be processed.”

I tried the same prompt with GPT-4.1-mini, but this time it refused to create the header, saying that the entity type is not a Canvas page, even though it actually was.

I created 📌 Canvas AI: Refine the orchesrtrator prompt Active to move the entity_type check and title/metadata generation logic to the orchestrator so that the Template Builder and Page Builder agents can focus only on their core tasks. This should at least help fix the latter issue.

The issue with 4.1 is a bit more complicated . I've often observed that it claims it’s going to use a tool, but never actually does.One option is to use the check added in ✨ Add possibility to check if a tool was run Active . However, that may not be the optimal solution, because we can't always guarantee that the orchestrator or any sub-agent should use a specific tool.

For example:

• The user might just ask a question like “What can you do?” - no tool is required.
• Or the user might say “Add an accordion component to the page”, but if there is no accordion component available, the agent shouldn’t invoke a tool in that case either.

Created https://www.drupal.org/project/canvas/issues/3554026 📌 Canvas AI: Refine the orchesrtrator prompt Active as per #11

akhil babu → created an issue.

Created https://www.drupal.org/project/canvas/issues/3553998 📌 Canvas AI: Template builder agent should be aware of existing components in regions Active as a follow-up as per #52.

🤔 The tests appear to run correctly locally.

akhil babu → created an issue.

akhil babu → created an issue.

Verified using card components with required and optional image props, and it works as expected.

Thanks for the feedbacks narendrar,

Should create title and metadata if not exist for page as done for page builder earlier

- Done
Create a template for the homepage of an ice cream shop website: It is observed that it did not place button in Empty button slot for hero component. Also content slot is empty in spacer and desired content is placed below spacer.
This can be enforced from the component description settings form.

Create a template for the homepage of an ice cream shop website, with proper header and footer: If content in header/footer region exists, it places duplicate components also.

If user explicitly asks to generte headera and footer, agent will generate contents even if those regions already have components. I think we could open a followup to optimize this behaviour

Ready for review

Now that 🐛 XB AI: User selection should be taken into account while building page Active is merged, this needs some updates

Ready for review

So, if a user selects a component that has slots and asks the agent to “add a card to the slots of this component,” the agent will go ahead and add the card, even if the card isn’t a suitable child component for that slot.

The suitable child components for each slot can be specified in the slot description from Canvas AI Component Description Settings. But that’s only taken into account when the agent designs the component structure.
For example, in the Starshot demo theme, if we ask the agent to add a hero banner suitable for the landing page of a streaming platform, it will add a hero banner with Paragraph and Button components in the slots. But if a user manually adds a hero banner, clicks on it, and then asks the agent to add another hero banner to the slot, the agent will still go ahead and add it. It depends on how the user phrases the request.

That said, if such validation is really required, it should be possible with some prompt adjustments. But I think that can be handled as a separate issue.

Also, We need to pass proper keywords as instructions. Eg instead of Can you add some cards to show case the product highlights?, if we ask Using page builder can you add some cards to show case the product highlights? or Can you add some cards to show case the product highlights on this page?, orchestrator will pass the request to correct agent for selected component and it will work as expected for scenario 1.

The orchestrator agent invokes the page builder agent correctly only when user inputs are in the format “Add something to the page” or when inputs are explicitly prepended with “Using page builder, do something.”
A direct prompt like “Can you add some cards to showcase the product highlights?” often confuses the orchestrator. In such cases, it may invoke the component agent instead and create a code component.

akhil babu → created an issue.

This is now ready for review.

Resloved the merge conflicts and verified in local. Moving back to RTBC

Ready for review

Thanks for the detailed analysis @tedbow . I have tested the latest changes in AiResponseValidator and confirming that its working when image data is passed in the below format

reference_nodepath: [0, 0]
placement: above
components:
  - sdc.canvas_test_sdc.card:
      props:
        heading: "Delicious Cookie Selection"
        content: "Indulge your sweet tooth with our mouthwatering variety of freshly baked cookies! Choose from chocolate chip, oatmeal raisin, snickerdoodle, and more—all made in-house every morning."
        footer: "Order today for same-day pickup or delivery!"
        loading: eager
        sizes: "auto 50vw"
        image:
          src: /modules/contrib/experience_builder/tests/modules/canvas_test_sdc/components/card/balloons.png
          alt: 'Hot air balloons'
          width: 640
          height: 427

I have updated the MR with following changes
1. Removed the previously added get_media_id tool and related changes as its no longer required
2. Updated the page builder agent's prompt so that it returns the image data properly
3. Updated Drupal\canvas_ai\CanvasAiPageBuilderHelper::processSdc to propely get the detault values for SDCs as described in #11

Out of all 4 sub agents of orchestrator the following only work with Canvas page

Drupal Canvas Metadata Generation Agent
Drupal Canvas Title Generation Agent
Drupal Canvas Page Builder Agent

The remaining agent is the Drupal Canvas Component Agent, which doesn’t care about the entity type. So I don't understand why we shouldn’t use the get_entity_info tool in the orchestrator and ask it to decline the request if the entity type is not canvas_page.

It has 2 Advantages
1. One API request can be saved, which also reduces cost. (For example, if the request is for a page building task, when the page builder agent gets invoked, the entire component context would get passed along with the system prompt, which is very costly.)
2. The user would get a faster response.

The only issue with this approach is when the user is exactly on the /canvas page, i.e., there is no entity_type value. In that case, the orchestrator could ask the user to create a page first before attempting to generate metadata, titles, or add components. Then later we can add new tools to agents to create 'Canvas page' entities programatically when entity_type is empty.

Significant changes have been made to the page builder agent in 🐛 XB AI: User selection should be taken into account while building page Active . Can we hold this issue until that gets merged?

akhil babu → created an issue.

Previously, page building was supported in nodes as well. Now you’ve added a prompt to restrict that to the canvas_page entity type only. So all operations, except code component generation and editing, are now supported only in the canvas_page entity type. So, I think we don’t need to invoke any subagents at all if the entity_type is not canvas_page. That’s why I thought filtering this at the orchestrator level would be more efficient.

If we are restricting all operations to the Canvas Page entity type, wouldn’t it be better to handle this at the orchestrator level by adding a prompt like the following:

If the task is anything other than code component generation or editing, check the entity_type value from the context. Use the corresponding sub-agents if and only if the entity_type is exactly 'canvas_page'. If the entity_type is 'node' or empty, respond with the message: "This operation is only valid for Drupal Canvas pages."

Have we considered refinements to the header and footer here at all? For example, "I'd like to add a 'Start free trial" button in the header that links to #" or "I'd like to add a 'Contact' link that links to '/contact' to the header".

If the prompt is like Create a homepage template for a pizza shop website with a proper header and footer. Add a ‘Start free trial’ button in the header that links to #. Also add a ‘Contact’ link that links to ‘/contact’ in the header, then the Template Builder agent being developed here would be able to place all those components.

If the template has been generated with a prompt like Create a homepage template for a pizza shop website with a proper header and footer, and the user makes a follow up request such as Add a ‘Contact’ link that links to ‘/contact’ in the header, this will be delegated to the Page Builder agent added in ✨ Introduce AI Agents and tools to Build Pages from Single Directory Components Active .

Also, what would happen if user provided a prompt "Make the header sticky so it stays at the top on scroll"? This seems like a common use case which we should think through. Usually you'd probably have some kind of header component which the AI probably should modify in this scenario. Alternatively if the header is an SDC, it could create a new component that wraps the existing component to add the sticky header behavior.

At the moment, this isn’t possible, as both AI agents can only add new components. We have to implement uupdating already placed components in a followup issue

As mentioned in #5, image props returned by AI should have values in the format prop_name: media_id to pass validation. So, I added a get_media_id tool that returns the media ID of the first image item in the database. If none is found, it creates a media item with a placeholder image, which is also included in this PR.

However, components won’t look good when placed with placeholder images. So for now, I am filtering out image props from the frontend so that components use the example image from their definitions.

We can later extend this tool to search for relevant images from the database or external services and return those, and remove the 'filtering' code

The validations will pass if the media ID of an image is given as the value for the image prop.

reference_nodepath: [0, 0]
placement: above
components:
  - sdc.canvas_test_sdc.card:
      props:
        heading: "Delicious Cookie Selection"
        content: "Indulge your sweet tooth with our mouthwatering variety of freshly baked cookies! Choose from chocolate chip, oatmeal raisin, snickerdoodle, and more—all made in-house every morning."
        footer: "Order today for same-day pickup or delivery!"
        loading: eager
        sizes: "auto 50vw"
        image: 1 // Media ID of an image

Thanks @Marcus Johansson for this finding.
We should add a ‘Media Finder’ tool to the Page Builder, which for now would return the first media ID it finds in storage. If no media ID is found, it will create a placeholder media item.

akhil babu → created an issue.

akhil babu → created an issue.

Checked this issue and its not longer reproducible.

Kindly review

akhil babu → made their first commit to this issue’s fork.

Thanks @narendrar. I have added it

This is now ready for review. I have also removed the use of navigate from operations handler as user was getting redirected form the page once components were placed

Linking to current blockers

Linking to current blockers

akhil babu → created an issue.

akhil babu → created an issue.

🐛 Canvas AI: Agent should pass values to image props correctly Active is a blocker now.

🐛 Canvas AI: Agent should pass values to image props correctly Active is a blocker now.

akhil babu → created an issue.

akhil babu → changed the visibility of the branch 3543744-xb-ai-create to hidden.

akhil babu → changed the visibility of the branch 3543744-xb-ai-create to active.

akhil babu → changed the visibility of the branch 3543744-xb-ai-create to hidden.

I have tested this using the content type agent and its working as expe

Overrode the existing description of the Create content type tool and its name and label arguments from the UI.

Put a breakpoint in OpenAI\Resources\Chat::create, and the tool uses the overridden descriptions instead of the default ones.

Moving to RTBC

I have tested this with the math reasoning example given here.

Created a Math tutor agent with instruction 'You are a helpful math tutor. Guide the user through the solution step by step'
Enabled structured output option and gave the following schema

{
  "name": "math_reasoning",
  "schema": {
    "type": "object",
    "properties": {
      "steps": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "explanation": { "type": "string" },
            "output": { "type": "string" }
          },
          "required": ["explanation", "output"],
          "additionalProperties": false
        }
      },
      "final_answer": { "type": "string" }
    },
    "required": ["steps", "final_answer"],
    "additionalProperties": false
  },
  "strict": true
}

Then using Agents explorer, asked the agent to solve 'what is x and y, if 3x + 2y = 36 and 2x - y = 17'. It gave the output in the proper json format.

{
  "steps": [
    {
      "explanation": "First, let's write down the system of equations: 3x + 2y = 36 (1), and 2x - y = 17 (2). Let's solve equation (2) for y.",
      "output": "2x - y = 17 => y = 2x - 17"
    },
    {
      "explanation": "Now substitute y = 2x - 17 into equation (1): 3x + 2y = 36 becomes 3x + 2(2x - 17) = 36.",
      "output": "3x + 4x - 34 = 36"
    },
    {
      "explanation": "Combine like terms: 3x + 4x = 7x, and move -34 to the other side: 7x = 36 + 34 = 70.",
      "output": "7x = 70"
    },
    {
      "explanation": "Solve for x by dividing both sides by 7.",
      "output": "x = 10"
    },
    {
      "explanation": "Now we substitute x = 10 into y = 2x - 17 to find y.",
      "output": "y = 2*10 - 17 = 20 - 17 = 3"
    }
  ],
  "final_answer": "x = 10, y = 3"
}

This is technically ready, but some extra changes are needed once ✨ Introduce AI Agents and tools to create entire page templates using available component entities Active gets merged. I’ll push them after that is merged, as it is more urgent

I could recreate it by asking the agent to create any component with a link prop.

'Create a cta code componet with a heading and link'
'Create a ContactUs block Next.js component with text and link props'

I tried to recreate this issue with 4.1 but couldn’t. The model directly created the component and also edited the components when follow-up questions were asked. However, the prompt changes should enforce this further.

This is now ready for review. It extends on the code changes made in ✨ XB AI: Create a common service for validations Active . So that issue must be merged first before reviewing this

I have tested this with prompts like Create a product card component with props for the title, image, and cost and the css and js are no longer printed in the response.

Ready for review

Minor change, Ready for review

Just noticed that addNewComponentToLayout still supports adding a 'to' param in the payload. So, only this change has to be reverted