🇮🇳India @Akhil Babu

This is now unblocked as ✨ Introduce AI Agents and tools to Build Pages from Single Directory Components Active has been merged.

#41 is also fixed in the current MR. I think we could merge this now.

Created ✨ XB AI: Add enum validation Active

akhil babu → created an issue. See original summary → .

Edit: Reverting the enum validation added in #48 as it breaks tests.

If enums prop is not required, maybe we can just removed non valid option, instead of regenerating the response again, also we can enter into infinite loop if LLM is stubborn

The max_loop setting in agents helps prevent them from going into an infinite loop. Simply removing the incorrect value might not be the best solution here, as I think we may still face the same "component failed to render" issue if the prop is required (though I haven’t tested this yet).

I think it’s better to handle this in a follow-up issue, where all of us can discuss our ideas for a proper fix.

Thanks for the feedbacks @heyyo.
#45 was due to a bug in the code. I have fixed that in 37631920 . It should work now.

#44
Urls are correctly getting added if the prop type for the url field is string. Eg:

    url:
      type: string
      title: Url
      description: URL for the link button.
      examples:
        - 'https://drupal.org

It might not be working correctly for 'uri' type props. I think we could support that in 🐛 Impossible to set a URL or SRC of an SDC with AI Active

#46
Added enum validation in b17588fd . This increases the response time as it forces the AI to regenerate the entire response if it returns a non-existing enum value. But at least we shouldn't face a "component failed to render" error.

Tested current MR with 🐛 AI-created new components cause loss of DeepChat message history Active . The final message from the page builder agent is not getting printed in the chatbot interface 😫 . We would have to create a followup issue for that as well

Created ✨ XB AI: Centralized Management of Component Descriptions, Props, and Slots Active as per #32

akhil babu → created an issue.

Created a followup issue ✨ XB AI: Simplify metadata generation Active for #21

akhil babu → created an issue.

Thanks for the explanation @utkarsh_33. IMHO, this can be very simplified, in fact, merged with the metadata agent created in 📌 Generate content metadata using AI Active . That agent is solely for generating the description field value for the XB page entity type based on the page content or a given topic. The same agent could generate the title as well.

The get_entity_information tool can be updated to give the data in following format

entity_type:
entity_id:
entity_fields:
   title:
   description:
   content: {{ The layout we send from ui should go here }}

And the ai_agent_create_field_content, ai_agent_edit_field_content and ai_agent_add_metadata tools can be combined into a single tool
ai_agent_set_metadata which accepts 3 arguments

'title'
'description'
'entity_id'

and returns these details in yml format which can be sent to the UI.

We could use a prompt like

You are a Drupal agent specializing in generating SEO-friendly titles and descriptions for xb_page entity types. You are a looping agent. You can run multiple times until the task is completed.

You will be provided with the entity_type, id, field values of the entity user is trying to modify. The field values may contain information like title, description and content of the entity. The content section will include details of various components placed on the page along with the content in their props and slots. Your task is to generate a title or description suitable for the page as per the user's request.

If the entity_type is not xb_page, simply respond that I can only generate metadata for 'XB Page' entities.

If the request is for xb_page entity type, try to understand the user's intent. If the user is asking to create or update title/description for a specific topic, prioritize that over the field values in the entity. Otherwise, generate a suitable title/description based on the field values of the entity including the page content.

Use the set_page_metadata tool to set the description or title. Use the id of the xb_page entity as the 'entity_id' parameter

Not sure if I'm missing something here. Would love to hear your thoughts.

akhil babu → created an issue.

📌 Generate content metadata using AI Active has already added a metadata generation agent for creating/editing page descriptions. So, couldn't we use the same agent to generate the title as well? Is it really necessary to create a new agent just for generating the title?

Updated the code as per the PR feedbacks. Kindly review again after taking latest pull and importing configs, Thanks

First, I created a code component using the prompt "Could you create a hero banner component with some text, call-to-action button, and background colour red".
Then I added the following prompt to test this MR functionality: "Create a Hero banner showing three key features of Drupal and Create a section showing three key features of PHP"
It created a HeroBanner component and 2 sections - one for Drupal and one for PHP. This seems incorrect to me.

Did it create the code component again? Also after generating the hero anner, was it added to the component library using the 'Add to components' option?
Code component generation and page building do not work together as of now. This means you cannot use a single prompt like: `Create a hero banner with text and a CTA, and place two banners on the page: one with content about Drupal and another with content about PHP.`

The orchestratior agent may sometimes delegate the task to the code component creation agent if the prompt contains words like 'Create'. So, to test page building, we should either start the prompt with 'Add/place a hero banner...' or at the end of the prompt add '..using the page builder agent'

The components are not getting placed in the canvas when they are nested. Moving back to active to check this

akhil babu → created an issue.

@marcus_johansson, Sure, I’ll go ahead and create a new issue for this. The upcasting feature sounds really useful. If I understood it correctly, when a parameter like node:1 is passed, we automatically get the corresponding node entity in the tool: similar to how routes work.

Just out of curiosity, what should the 'data_type' be in such cases? I tried using entity:node, but it didn’t seem to work, because of this check in Drupal\Core\Plugin\Context\ContextDefinition::__construct()

assert(!str_starts_with($data_type, 'entity:') || $this instanceof EntityContextDefinition);

Not sure if it's an actual bug
In ✨ Introduce AI Agents and tools to Build Pages from Single Directory Components Active , I have created a tool that accepts a yml string from the ai agent.
The context definition is added like this

  context_definitions: [
    'component_structure' => new ContextDefinition(
      data_type: 'string',
      label: new TranslatableMarkup("Component structure in yml format"),
      description: new TranslatableMarkup("The component structure to store, in yml."),
      required: TRUE,
    ),
  ],

$component_structure = $this->getContextValue('component_structure'); always returns the yml string. But now, Drupal\ai\PluginManager\AiDataTypeConverterPluginManager::convert() converts it into an array as it calls Drupal\ai\Plugin\AiDataTypeConverter\YamlDeserializer::appliesToDataType(). Since the data_type is set as string in context definition, this results in error 'This value should be of the correct primitive type.'

akhil babu → created an issue.

akhil babu → created an issue.

Most of the feedbacks have been addressed. Kindly review the changes again, Thanks

Since we added the possibility to step through the loops it would be nice if the Orchestration Agent splits up the requests to the Page Builder Agent into loops even if this takes 3-5% extra time to produce, because then for a request like "Create me three components on the page for my pizza shop Marios Pizza", would create the first component and have a loader for that and you would see it, then it would start on the next etc. - so you see progress. #3531000: Integrate incremental agent loop execution in XB AI will be needed for this.

Something like this would definitely improve the user experience. I haven't checked 📌 Integrate incremental agent loop execution in XB AI Active yet, but I'm going to give it a try.

Thank you for the encouraging feedback @marcus_johansson.

I'm guessing there is another issue for editing components and their values?

Yes. Currenly only component addition is implemented. New issues will be created for updating/rearranging existing components placed in a page.
Eg: User clicks on a paragraph component and asks the agent to summarize/elaborate/replace it's content and props.

The YAML structure should be as minimal as possible (maybe already is) - the wait time is 99% invested on generating tokens, and the less tokens it needs to generate the better.

The front end only needs 2 options to add a component to the page
1. The component object
2. nodePath (An array represnting the position of the component: [0,0] -> First component, [0, 0, 0, 0] -> First component in the first slot of the first component 😬)

Initially, I tried asking the agent to generate the nodepath, but it started hallucinating when components had to be nested deep. So, in the current approach we ask the agent to provide a yml in the following format

    reference_nodepath: []  # Nodepath of any existing component placed in the page.(Agent gets it from the current layout json provided as context)
    placement: below                           # 'above' or 'below'
    components:
      - sdc.component.id:
          props:
            prop_name: "value"
          slots:
            slot_name:
              - sdc.nested.component:
                  props:
                    nested_prop: "value"
    message: "Concise 1-2 sentence summary of changes made"

and then generate the nodepath from this yml. The data for each component in the yml contains its id, prop names and values, slot names and child components.
The front end gets these details -> Loads the component objects based on ID -> Replaces the prop values and places each component at the specified nodepath.
So the yml contains only the required data.

Thanks for reviewing @marcus_johansson. I’ll go through your feedbacks and get back to you.

Moving this back to Active for the following updates

1. Create a form that allows site builders to customize the description of SDC components and their props/slots. Upon submission, this description will be sent to the agent as context.

2. Extend the same form with a checkbox list that lets site builders choose which enabled component entities (SDCs, Blocks, Code components) should be included as context for the LLM. Currently only SDCs are allowed

The page builder AI agent needs both the curent layout structure (Provided by the get_current_layout tool) of the page and the details of avaialble SDCs (Provided by get_sdc_info tool). Default information tools option helps to send these direct;y along with the Agent instructions as context. So it saves two extra AI calls.

The changes aren’t about creating or editing components. They’re focused on adding already existing components (Only SDC as of now) to the canvas with dynamic content to help build pages. For example: “Create a section showing three key features of Drupal” or “Add a button with the text ‘Read more.’”
In my opinion, creating entire page templates (with header, body, footer, etc.) can be handled in a separate issue after the current approach is reviewed and accepted, since we’ll need to reuse a lot of the existing logic for that.

Also, have you run npm run build after pulling the latest changes? If yes, are you seeing any errors in the console?

Currently there are 2 issues

1. The messages from the page_builder_agent are not rendering in the chatbot window.
2. Once a component is selected from the canvas, the screen automatically refreshes while typing the next message. It seems that a request is somehow being triggered to the endpoint /xb/api/v0/form/content-entity/{entity_type}/{entity}/{entity_form_mode}

akhil babu → created an issue.

Thanks @annmarysruthy. Just checked again. getReadableOutput has been moved to Drupal\ai\Base\FunctionCallBase. So this issue no longer exists. Closing this issue

akhil babu → created an issue.

This is working fine now. So merging the code

Ready for review

Tested with
AI Core 1.0.5
AI Agents 1.0.2
Gemini provider 1.0.0-beta1

Steps to use Gemini Provider

• Add your Gemini API key in Configuration > System > Keys
• Go to Configuration > AI > Provider Settings > Gemini provider and select the added key
• Go to Configuration > AI > AI Default Settings
• Select 'Gemini' as 'Default Provider' and 'Gemini 2.0 flash' as 'Default model' for 'Chat' and 'Chat with Complex JSON' operations
• Now, the assistant would work with the Gemini provider

akhil babu → created an issue.

akhil babu → created an issue.

akhil babu → made their first commit to this issue’s fork.

Thanks @wim leers. I have updated GeneratedFieldExplicitInputUxComponentSourceBase::validateComponentInput() and added a check for unexpected props.

However, no exception was thrown for the curl request mentioned in #5, and the heading component was still added to the canvas. The expected error only appeared when publishing the page (Because ValidComponentTree constratint was invoked)
Shouldn't the error have been thrown at the time of the request?

I also tried to validate 'missing' props but that resulted in many test failures as some props are not always part of user input (Like 'attribute' prop in sdc.experience_builder.my-hero component)

Thanks @wim leers🙏. I just tested this using the 'Heading' SDC, by adding an extra property 'textUnwanted' when making a curl request to the endpoint /xb/api/v0/layout/xb_page/5 and the changes were successfully saved. There were no validation errors regarding the unexpected prop

curl --location 'https://starshot.ddev.site/xb/api/v0/layout/xb_page/5' \
--header 'accept: */*' \
--header 'accept-language: en-US,en-GB;q=0.9,en;q=0.8' \
--header 'content-type: application/json' \
--header 'origin: https://starshot.ddev.site' \
--header 'priority: u=1, i' \
--header 'referer: https://starshot.ddev.site/xb/xb_page/5/editor/component/fdc18261-ef6e-41bd-bdba-99ecc72bf359' \
--header 'sec-ch-ua: "Chromium";v="136", "Google Chrome";v="136", "Not.A/Brand";v="99"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "Linux"' \
--header 'sec-fetch-dest: empty' \
--header 'sec-fetch-mode: cors' \
--header 'sec-fetch-site: same-origin' \
--header 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36' \
--header 'x-csrf-token: PIqRn04kuCGgm5leY3w_pc0Oq9ZGv0mzWXds0SE7cmg' \
--header 'Cookie: SSESS1a128ff57fee5bd2b5663cea7bd52db5=n4epfmoesptq7m91dcj6skrvtl' \
--data '{
  "layout": [
    {
      "nodeType": "region",
      "id": "content",
      "name": "Content",
      "components": [
        {
          "slots": [],
          "nodeType": "component",
          "type": "sdc.experience_builder.heading",
          "uuid": "fdc18261-ef6e-41bd-bdba-99ecc72bf359"
        }
      ]
    }
  ],
  "model": {
    "fdc18261-ef6e-41bd-bdba-99ecc72bf359": {
      "name": "Heading",
      "resolved": {
        "text": "A heading element new text",
        "style": "primary",
        "element": "h1",
        "textUnwanted": "An unwanted heading" // Invalid prop
      },
      "source": {
        "text": {
          "expression": "ℹ︎string␟value",
          "sourceType": "static:field_item:string",
          "value": [
            {
              "value": "A heading element new text"
            }
          ]
        },
        "textUnwanted": { // Invalid prop
          "expression": "ℹ︎string␟value",
          "sourceType": "static:field_item:string",
          "value": [
            {
              "value": "An unwanted heading"
            }
          ]
        },
        "style": {
          "expression": "ℹ︎list_string␟value",
          "sourceType": "static:field_item:list_string",
          "value": [
            {
              "value": "primary"
            }
          ],
          "sourceTypeSettings": {
            "storage": {
              "allowed_values": [
                {
                  "value": "primary",
                  "label": "primary"
                },
                {
                  "value": "secondary",
                  "label": "secondary"
                }
              ]
            }
          }
        },
        "element": {
          "expression": "ℹ︎list_string␟value",
          "sourceType": "static:field_item:list_string",
          "value": [
            {
              "value": "h1"
            }
          ],
          "sourceTypeSettings": {
            "storage": {
              "allowed_values": [
                {
                  "value": "div",
                  "label": "div"
                },
                {
                  "value": "h1",
                  "label": "h1"
                },
                {
                  "value": "h2",
                  "label": "h2"
                },
                {
                  "value": "h3",
                  "label": "h3"
                },
                {
                  "value": "h4",
                  "label": "h4"
                },
                {
                  "value": "h5",
                  "label": "h5"
                },
                {
                  "value": "h6",
                  "label": "h6"
                }
              ]
            }
          }
        }
      }
    }
  },
  "entity_form_fields": {
    "langcode[0][value]": "en",
    "revision_log[0][value]": "",
    "title[0][value]": "Untitled page",
    "path[0][alias]": "",
    "path[0][source]": "/page/5",
    "path[0][langcode]": "en",
    "image[media_library_selection]": "",
    "description[0][value]": ""
  }
}'

Just to confirm, does this mean validateComponentInput() currently allows unexpected properties (i.e., garbage values) for SDCs?

Could you please give an example for 'garbage value'?

I have created an MR based on the patch in #4 with some additional changes
1. Got the following error for blocks
When a #lazy_builder callback is specified, no properties can exist; all properties must be generated by the #lazy_builder callback. You specified the following properties: #prefix, #suffix, #access

So updated the lazybuilder callback to include the #access, #prefix and #suffix properties. But the Drupal\Tests\experience_builder\Kernel\Plugin\ExperienceBuilder\ComponentSource\BlockComponentTest::testRenderComponentLive fails as #prefix and #suffix are not included in the assertion. Should the test be updated, or is there a better approach?

akhil babu → made their first commit to this issue’s fork.

Please review

akhil babu → created an issue.

I have tested the changes in https://git.drupalcode.org/project/ai_agents/-/merge_requests/124/diffs?... and https://git.drupalcode.org/project/ai_agents/-/merge_requests/124/diffs?... and its working fine.

• [ai_agent:agent_instructions] token is properly added for content type and taxonomy agents
• The [agent_instructions] token is correctly updated to [ai_aent:agent_instructions] in previously created agents, after running drush updb

MR 1008 adds an error message when the code component does not exist.

However user can't navigate to a new code component from the left side bar. The URL changes, but the code editor is not renderd unless 'Try again' is clicked. The error boundary needs to be reset. I belive 'resetBoundary' should be used, but it didn't work (I probably implemented it incorrectly. still learning react 😅)

akhil babu → created an issue.

akhil babu → made their first commit to this issue’s fork.

Looks like this is already being implemented in ✨ Add the actual agents as configs Active

Please review

akhil babu → created an issue.

Will try to create the agents

Looking at the available tools, all tools mentioned in the issue summary are already there

'List Bundles' tool already provides bundle types for an entity type.
'Get Content Type Info' provides information about any node types.
'Create Content Type' for Creating content types
'Edit Content Type' for editing content types
'Delete Content Type' for deleting content types.

Please review

I was able to reproduce this issue. When the page title is updated, the entity_form_fields store the correct value, but the HTML section in the autosave data does not get updated.

Rather than adding '/' to paths that dont start with it, we should add validation to the entity form fields

Validation for entity fields are not yet impletemted
file: experience_builder/ui/src/components/form/inputBehaviors.tsx

  const validateNewValue = (e: React.ChangeEvent, newValue: any) => {
    // @todo Implement this.
    return { valid: true, errors: null };
  };

This should be updated to

  const validateNewValue = (e: React.ChangeEvent, newValue: any) => {
    const target = e.target as HTMLInputElement;
    if (target.id === 'edit-path-0-alias' && typeof newValue === 'string') {
      if (newValue && !newValue.startsWith('/')) {
        return {
          valid: false,
          errors: [
            {
              keyword: 'value',
              instancePath: '',
              schemaPath: '',
              params: {},
              message: 'The alias path has to start with a slash',
            },
          ],
        };
      }
    }
    // @todo Implement for other entity fields.
    return { valid: true, errors: null };
  };

hestenet → credited akhil babu → .

"#2 is no longer valid.

The module's architecture has completely changed. It now leverages utility functions defined by the XB UI, such as addNewComponentToLayout, to place components rather than updating the autosave data. This is inspired by the code of the ConceptProvider extension added in XB.

The current implementation has only one agent with two sub-agents. The new version has the following agents:

• Main Agent – An orchestration agent that parses the user request and assigns the task to the corresponding sub-agents.
• Template Designer Agent: Used as a tool by the main agent. Responsible for building an entire page (e.g. a template for a news article).
• Layout Editor: Used as a tool by the main agent. Responsible for adding new components to an already created page (e.g. "Add a paragraph with text explaining 5 new features of Drupal.")
• Contextual Editor: Used as a tool by the main agent. Can update content or child components of a selected component placed on the page (e.g. the user selects a 2 column layout and asks, "Add 2 images to both columns").

The module now supports creating code components from Figma design, using the Figma MCP Server. These new agents have been created for that:

• Figma Agent: Creates Preact code from Figma data.
• Figma Image Decoder Agent: Used as a tool by the Figma agent when the code component should use the exact images present in the Figma design, if any."

These are not yet published.

I have raised a PR. Please review

akhil babu → created an issue.

Please review

akhil babu → created an issue.