🇮🇳India @utkarsh_33

This is ready for a review/testing.

@tedbow i tested the MR and here are my findings:-

1. The page builder works fine and the errors reported to the AI makes the required changes and there is no error reported on the UI side.For this i tested the same scenario which i mentioned in #12, with all the changes mentioned in #12.
2. There seems to be some problem related to the validation for the component builder. I made the following changes in PropsSchema.json :-
   

   {
       "name": "Image with example value",
       "type": "object",
       "$ref": "json-schema-definitions://experience_builder.module/image",
       "example": {
         "width": "1200",
         "height": "900",
         "alt": "Example image placeholder"
       },
       "id": "4ea2a14d-7828-467c-a505-063f006f1ebf",
       "derivedType": "image"
     },

   use this image prop instead of the existing image prop to feed the wrong example to the AI about the prop structure so that it produces wrong props.

   When we do the above change and ask the AI to create a hero component using Create a hero component prompt and then wait for it to do all the validation and return the valid prop structure it does return if we add a breakpoint in CreateComponent but proceeding further it gives an error in the UI An error occurred while processing your request. Please try again. and also you can see the errors in the reports which are basically the validation error. I didn't dig deep into it what is causing the issue but i have described the exact steps that took me to the error.

Tested this and it fixes the component creation task using AI. Marking it RTBC.

Re #7 :
The

No prompt

issue is fixed also the LLM asks relevant questions not just related to the users.

I totally agree with

I don't think we should assume, uploading an image, means creates a code component.

therefore it asks the user questions related to what task it should perform based on it's current capabilities. As the capabilities grow then it might present those additional options as well to the user.

Marking it NR

utkarsh_33 → created an issue.

Tested this manually and now the image upload feature works much better. This is ready for review.

🐛 AI should use only necessary history/comments for each task Active is fixed now and this is actionable.

Thanks @tedbow for the work. I tested it manually and here is my insights after testing:-

1. I get the correct structure once the structure is validated and the feedback(in the form of exception) is given back to the AI related to the problem in the prop structure. The exception is processed by the AI and the required changes are made to generate a valid structure.
2. One thing that AI does great is, say if we have 3 heading components in a page layout and somehow there was a issue with props structure of heading component and we give AI the feedback that the problem exists then it refactors all the heading components at once thus reducing our looping through other heading components as well(This might be expected behaviour as well but just documenting it as a win)
3. Another scenario that i tested was with a combination of components namely :- heading and hero-component. I intentionally told the AI the wrong structure for both the component by adding
   

   *  **CRITICAL** : If the component is *sdc.xb_test_sdc.heading* component then never include element props in it.
           *  **CRITICAL** : If the component is *sdc.xb_test_sdc.my-hero* component then use then never add cta1href props in it.
   

   these 2 rules in my page builder agent which lead to the wrong generation of the component structure.
   I gave the prompt Add 3 heading component and 2 hero component and then i got the error for the heading component which fixed the issue for all the 3 headings and in another round of validation it fixed for the hero component by adding the required props back to the structure. For now i think we can go with validating 1 component at a time and maybe open a discussion with @laurii and @tim.plunkett related to whether we want to validate all components at once. Not sure if this might require some changes in prompting or will it be taken care off just like it is happening now.

4. I had a question related to validation error @tedbow, are the errors descriptive enough to provide the AI with sufficient details about the reason for failure? In the above 2 cases it was able to tell AI that the 2 fields that i deliberately told not include were actually required. I will try to figure out scenarios related to wrong props structure and maybe test it against them, in the mean time if you have one in mind please let me know i can test that as well.

For now that's all i have in my mind.Thanks!

I think we should wait for 🐛 AI should use only necessary history/comments for each task Active to go in first, so postponing it on that.

utkarsh_33 → made their first commit to this issue’s fork.

Tested most of the scenarios and all the responses are much better then previous ones and the history is smartly taken care off now. Happy to mark RTBC based on the current state.
Anything else related to history could be fixed in a better way in Create ShortTermMemoryPlugin → .

This still has some work left to do and the code in this is not review ready so assigning it to me for now.

So here is what I tried. In the ‘set_component_structure’ tool where we get the component structure generated by the AI I tried to pre-validate the components that are created by the AI using ApiAutoSaveController::post where I edited the `field_xb_demo` that is a part of the $all_auto_saves’s data array. This is by default populated from the UI changes that we make(example if we add 2 components in the UI then those 2 components are a part of this array).
The array structure looks something like :-

[
   'uuid' => 'da3155db-cda8-4b1a-81e1-5cf501c2065b',
   'inputs' => '{"text":"Press","href":      {"uri":"https:\\/\\/www.drupal.org","options":[]}}',
   'component_id' => 'sdc.xb_test_sdc.my-cta',
   'component_version' => '9454c3bca9bbbf4b',
   'parent_uuid' => null,
   'slot' => null,
   'label' => null
]

Where I tried to validate the props for the elements that we create and randomly coding rest of the values as arbitrary values.
What I tried was add a list of arrays with there props set against the input value in the array structure defined above and then letting the auto save do the rest of the stuff.
The response for the `post` method that I always got was {"message":"Successfully published 0 items."} which should not be the case as I am populating wrong prop data as a part of 1 component in the $all_auto_saves’s data array.
Now the real question is am I going in the right direction or I need to try something else? I don’t have much idea about how the Autosave manager works so it would be helpful to get some insights for it.
Also we might need or use any existing api which can validate the component structure before returning the response to the client side as it would not be possible for us to re-iterate in case we get some error on the client side. If we can do that in the backend then it would be easier for use to re-iterate.
Just for the context we get the following structure from the AI tool in ‘set_component_structure’/SetAIGeneratedComponentStructure.php file:-

reference_nodepath: [1, 0]
placement: above
components:
 - js.text:
   props:
    text: "Subscribe to our newsletter and be the first to know about exclusive deals and new arrivals."
    textColor: "Dark"
    textSize: "Normal"
 - js.button:
   props:
    text: "Subscribe Now"
    variant: "Solid"
    link: "#"

utkarsh_33 → created an issue.

Tested this locally and it now works as expected. Marking it RTBC.

I have a small suggestion to this. As you can see in the image → we can add red color to the text to distinguish it from the normal message.
I can push the changes related to this in a commit if it is an improvement.

Changes looks good to me and now we get a better message in the Deepchat. Also all the feedbacks are addressed so marking it RTBC.

So now we get all the prop's structure correctly which are supported by experience builder and select the correct props for creation(except image) of a component.
I will attach a video demonstrating how it works now for component creation. Also marking this MR to NR, in the mean time i can try to work with image prop.

utkarsh_33 → made their first commit to this issue’s fork.

I just tried reproducing the issue on the latest 0.x, but was not able to reproduce this. Scrolling is working for me.
Attaching the recording.
If i am missing some steps then please let me know or update the steps to reproduce demonstrating the scenario in which this issue was occurring.

Addressed feedbacks.

I have merged all the new changes and also tested that no existing scenario is breaking and the new once are working as expected. Marking this NR.

Uploading the SS showing how the image upload icon looks like.

I have created a follow-up issue which enables the client side validation of size for image uploads 📌 Client-side validation preventing large image uploads to LLM Active .

utkarsh_33 → created an issue.

Tested the MR by setting different times for timeout and things works as expected. Also the code looks good to me so marking it RTBC.

Merging the latest changes from 0.x.

So the current MR fixes the issue mentioned in the IS and the code looks good to me so this can be RTBC'd.
As far as the issue mentioned by @balintbrews in #9 🐛 [PP-1] Chat history not getting preserved for Deepchat on re-rendering Active is something that i can see in the network tab once the component is created but not when i try to edit the component using the code editor or using the AI bot.
It might be because of following code:-

const component = message.component_structure;
    await createCodeComponent(component).unwrap();
    navigate(`/code-editor/component/${component.machineName}`);

Upon quickly debugging i found that component.status is false, but didn't investigate much in detail as this can be fixed as a separate issue.
@balintbrews if you think that it's not a hard blocker for this issue then i can open another issue for the point you mentioned in #9 and we can keep this issue moving forward.

The current state of the MR works fine for creating simple code components. but there are some issues when there is an image which are as follows:-

• When a user uploads an image as mentioned in #8's prompt 2 📌 [PP-1] Image upload feature in AI Chatbot Active It makes it hard for the ai to generate exact similar doodle. It identifies that a doodle is present but does not generate one which is exact similar.
• When a user uploads an image like this → then it does insert an image which is close to this(in this case it inserted and image of forest without the human)

It does create simple components such as this → one very well. Also as discussed with @laurii that if we tell the AI bot to create a component and also which image to insert in the images section then it does take care of that as well.

@Marcus_Johansson, @laurii we can discuss further what all needs to be taken care of in this issue.

Thanks for the review @Marcus_Johansson. I have made the changes which does not require inputs from @laurii.Remaining things can be done once we get the confirmation from @laurii.

This change is not related to this MR but since @AkhilBabu made a valid point that we don't need that if we are using it in default information so removing it as a part of this issue only.If someone thinks that it should be done in a follow-up then happy to do that as well.This does not affect any existing functionality so i thought it's better to do that in this issue only.

I'll assign it to me as this still needs some more work.

I have rebased it with the changes from HEAD.

utkarsh_33 → made their first commit to this issue’s fork.

Since the follow-up is created and the current functionality reported in the issue is also working fine and the code also looks good to me, marking it RTBC.

When a user refreshes the page, the panel’s state (open or closed) and its content should be preserved. If the panel was open before the refresh, it should remain open with its content intact after the refresh. The history should only be cleared when the user manually closes the panel, not upon a page refresh.

I have tested to best of my knowledge and everything seems to be working fine.Marking it NR.

Here is the video demonstrating how the title is generated based on the content present on the page.

Marking this NW for a small left over.

I tested this and i can confirm that the message chat history is cleared when the AI panel is closed.I noticed an issue while testing this:-

Processing state on 0.x:-

Processing state on this MR:-

The processing states are not correct.

Handled the problem reported in #9 in this commit.
Now things are working as expected so marking it NR for an initial round of testing or Code reviews.

There is a know problem in this which is this comment. It's because we only want the creation and editing flow to work with the xb_page entity for now as discussed with @lauriii.But the everting works as expected if we are on the xb_page entity.
Will try to get a workaround for this ASAP.

So here's the update on how am i testing it along with the results:-
Editing flow:-

1. Lengthen/shorten title's content:-Working as expected
2. Fix grammar for title:- Working as expected
3. Elaborate title:- Working as expected
4. Rephrase title:- Working as expected without further questions to the user.

Content creation flow:-

1. Create content on topic Drupal and insert in title field:- working as expected
2. Add title on cricket/ Add content on title field for cricket :- working as expected (which was not working before)
3. Generate title on topic Drupal:- working as expected

I am not adding the component creation flow as those are already working.

Re #6 : This problem of failing to edit content now is fixed and it almost(what ever i tested) works as expected.

It would be great if someone can test this MR now.

So here's what the current state looks like:-

1. The component creation and editing flow works fine(which includes the editing of the props for the components as well)
2. The content creation flow also works 10/10 times and it creates the content correctly on any given topic and insert it correctly in the title field.
3. The content editing flow only works 8/10 times which is causing this to push back a bit.I have attempted changing the prompt(and still trying it) so that before calling the editing tool it always get's the content related to the field(in our case it will always be title) that user want's to edit, but sometimes it fails.

I'll keep trying to fix the issue that's coming while editing the title and will post any update soon.

I tested the MR and here are my observations from it:-

• The history of messages remains intact even after navigating to the other page.
• The issue with this is that even after refreshing or closing the chatbot the history still remains intact.Does it makes sense to add a Clear chat button to reset history?

Rest everything looks good apart from small nits.Marking it NW for small changes.

utkarsh_33 → created an issue.

So as per the discussion with @lauriii we agreed to the point that we only want to update the title field for the xb_page entity type.So this issue is scoped at updating the title field with the value generated by AI agent.
Also we might need to make changes in setPageData according to what in @balintbrews mentioned in this comment 📌 Generate structured content with AI Active .

Also i want to highlight that this MR is capable of generating content for multiple fields at a time as demonstrated in #4 📌 Generate structured content with AI Active but for now we might just use this to update the title that to only for the xb_page entity.

There are a lot of changes that are common in 📌 Generate structured content with AI Active and this MR, so postponing it on 📌 Generate structured content with AI Active .
Once 📌 Generate structured content with AI Active is in we might need to do a little bit of refactoring and test this to move this ahead.

utkarsh_33 → created an issue.

So here's what we assumed and are trying to do:-

1. When we say we want to insert the data generated by AI in body and title it could be both the node entity type as well as xb_page.I have pushed the code to update the page data in this commit after which tried to update both types of entity but its not working.
2. I also tried to update different values as we are doing in some tests using
   

   const newerTitle = { title: [{ value: 'Newer title' }] };
       store.dispatch(setPageData(newerTitle));

   but unfortunately it also didn't work.

I think we are targeting to update the entity fields as a part of this issue for now not the component's base field.We can have a follow-up for that if we want.

I also tried to see the form state (for node entity) and the key, values for the form are a bit different from what's in the xb_page entity.There also i tried to update with some hardcoded values but it didn't update the values in the entity fields.

If i am missing something please let me know how can i start digging more into the issue.

This is the structure of the form for node entity.

{
    "changed":,
    "title[0][value]": "",
    "form_build_id": "",
    "form_token": "",
    "form_id": "",
    "field_image_0_upload_button": "",
    "field_image[0][fids]": [],
    "field_image[0][display]": "1",
    "body[0][summary]": "",
    "body[0][format]": "",
    "body[0][value]": "",
    "field_tags[target_id]": "",
    "menu[enabled]": false,
    "menu[title]": "",
    "menu[description]": "",
    "menu[menu_parent]": ":",
    "menu[weight]": "",
    "path[0][alias]": "/1",
    "uid[0][target_id]": "",
    "created[0][value][date]": "",
    "created[0][value][time]": "",
    "promote[value]":,
    "sticky[value]":,
    "advanced__active_tab": ""
}

The following is the structure for xb_page entity:-

{
    "form_build_id": "",
    "form_token": "",
    "form_id": "",
    "title[0][value]": "",
    "path[0][alias]": "",
    "image[media_library_selection]": "",
    "image-media-library-update": "",
    "image-media-library-open-button": "",
    "description[0][value]": "",
    "advanced__active_tab": ""
}

I assume this issue is only for adding content on entity fields (including body), we can have a separate issue for components when needed. @tim.plunkett and @lauriii can you please confirm before I make further progress on this?

So here is the update what's happening in this MR at this point.
When a user gives a prompt like

Create content on topic dogs and insert it in body and title field

Then the response that comes to the front-end is:-

{
    "status": true,
    "message": "Array\n\nI\u0027ve created comprehensive content about dogs for both the title and body fields. The content includes:\n\n1. An engaging title that introduces the topic: \u0022Understanding Dogs: A Complete Guide to Man\u0027s Best Friend\u0022\n2. A detailed body section that covers:\n   - The historical relationship between dogs and humans\n   - Dogs\u0027 social nature and communication abilities\n   - Five key aspects of responsible dog ownership\n   - Concluding thoughts on dog care\n\nThe content has been successfully added to both fields and is formatted appropriately with proper paragraphing and structure. The information is both informative and engaging, suitable for a website article about dogs.",
    "created_content": [
        {
            "title": "Understanding Dogs: A Complete Guide to Man\u0027s Best Friend",
            "body": "Dogs have earned their title as \u0027man\u0027s best friend\u0027 through thousands of years of companionship, loyalty, and unwavering devotion to their human families. These remarkable animals have evolved from wild ancestors into the diverse array of breeds we know and love today.\n\nAs social creatures, dogs possess an incredible ability to understand human emotions and communicate through both verbal and non-verbal cues. From the tiniest Chihuahua to the massive Great Dane, each breed brings its own unique characteristics, temperaments, and care requirements.\n\nKey aspects of responsible dog ownership include:\n\n1. Proper Nutrition\nFeeding your dog a balanced diet appropriate for their age, size, and activity level is crucial for their health and longevity.\n\n2. Regular Exercise\nDogs need regular physical activity to maintain their physical and mental well-being. The amount and type of exercise needed varies by breed and age.\n\n3. Veterinary Care\nRoutine check-ups, vaccinations, and preventative care are essential for keeping your dog healthy and happy.\n\n4. Training and Socialization\nEarly training and proper socialization help develop well-behaved, confident dogs who can adapt to various situations and environments.\n\n5. Love and Attention\nDogs thrive on human interaction and require regular attention, affection, and mental stimulation.\n\nWhether you\u0027re a first-time dog owner or an experienced pet parent, understanding these fundamentals of dog care will help you provide the best possible life for your canine companion. Remember that every dog is unique, and taking the time to understand your pet\u0027s individual needs is key to a successful and fulfilling relationship."
        }
    ]
}

The message part of the response will go in the chatbot that user interacts with.
As we can see that the response contains the content along with the field name with which it is associated or needs to be inserted.

Now the question is how can we update the UI.Considering the example of the two fields that are there in the response i.e body and title how can we insert the content in respective fields.

utkarsh_33 → created an issue.

This is ready for review.

🐛 External AI Chatbot Functionality Active is fixed now so removing Postponed status from the issue.

This is built on top of 🐛 External AI Chatbot Functionality Active .

utkarsh_33 → made their first commit to this issue’s fork.

Closed 📌 [PP-1] Add a new subagent for moving the components using experience builder AI. Active follow-up as i have fixed the issue in this issue only.Moving the components only make sense when we have multiple components(that we plan to add while adding the page builder agents) and this MR allows us to create and edit single component.

🐛 External AI Chatbot Functionality Active adds the ability to create and edit single component, so we can can't move the single component around.This issue is fixed in 🐛 External AI Chatbot Functionality Active only so we don't need this follow-up now, thus closing it.

Here is the video demonstrating the component creation and editing using xb_ai module.

utkarsh_33 → created an issue.

I tried reverting the changes in this commit but still the tests were failing.I am wondering that could this be because of some random wait?I see that explicitly the wait was set to 40000.
I am unable to get the Cypress running on my local(DDEV) machine so it's hard for me to figure out what's causing this to fail.

utkarsh_33 → created an issue.

utkarsh_33 → created an issue.

utkarsh_33 → created an issue.

I tried using Vite as per request in this comment but I am unable to get the CI passing for the UI checks.

utkarsh_33 → created an issue.

Created a follow-up issues in which we can add tests [PP-1] Add test coverage for Experience builder AI submodule. 📌 [PP-1] Add test coverage for Experience builder AI submodule. Active .

utkarsh_33 → created an issue. See original summary → .

utkarsh_33 → made their first commit to this issue’s fork.

Re#9 ya i just noticed that the positioning was a bit off.I have fixed that positioning.Marking it NR again.

I can confirm that the issue is solved with this Fix.

Re #6 I can see the SS that you attached still have the issue that's reported here.The width of the installed image and the drop button are still different,You can see the SS in #3 how it looks after the fix.

Adding #3511104: The dropbutton has extra width when opening in the modal as could haves as this is a UX improvement with which users might interact frequently.

Merging the latest changes.

Ready for reviews.

utkarsh_33 → made their first commit to this issue’s fork.

The tests are now passing.

utkarsh_33 → created an issue.

Also as i mentioned earlier in #8 regarding the use of toggles instead of plain checkboxes, I am attaching the SS of how it looks after using toggle.
SS:-

What do others think about this change?