Prompt Create a header for a restaurant site with logo, search bar and main navigation menu created component instead of template
See:
But similar prompt for footer created correct template Generate a footer for a restaurant site with location/hours, menu links, social media, and reservation button
Test is failing at https://git.drupalcode.org/issue/canvas-3533079/-/jobs/7002673#L2055
Re,
#5
🐛
AI fails to resume component generation after a prerequisite error is resolved
Active
,
When I added second prompt(I have enabled it), without enabling it, it neither resume operation, nor it validated that Linkset is not enabled
Test fixed and handled in CanvasAiPageBuilderHelper.php. Moving to RTBC.
narendrar → made their first commit to this issue’s fork.
This seems duplicate of 📌 "Thinking" status persists after a process terminates with an error Active which is already fixed. Please re-open if this is a different issue.
This polling behaviour is intentional. Polling begins as soon as a chat request is sent to /admin/api/canvas/ai, in order to track the request status and retrieve information as soon as it's available. Currently, maxPolls is set to 500 (about 16 minutes), and we can make this configurable in the future.
Tested manually and this fixes PHPStan issue on Canvas
Fixed in https://www.drupal.org/project/canvas/issues/3540211#comment-16236980 🐛 XB AI: User selection should be taken into account while building page Active
Can you please create a merge request for this? This will automatically run the created test and make it easier to review.
Also, I think these tests should not interact with the LLM directly; we should mock the response instead.
See:- tests/src/Playwright/ai.spec.ts
Tested it manually and below are some observations:-
- Should create title and metadata if not exist for page as done for page builder earlier
Create a template for the homepage of an ice cream shop website:It is observed that it did not place button in Empty button slot for hero component. Also content slot is empty in spacer and desired content is placed below spacer.Create a template for the homepage of an ice cream shop website, with proper header and footer: If content in header/footer region exists, it places duplicate components also.
Tested it with below prompts and LLM is using packages from custom library:
Create a reusable button component. Define variants for:
intent: 'primary', 'secondary', 'danger', 'ghost'
size: 'xs', 'sm', 'md', 'lg', 'xl'
fullWidth: true/false
Set defaults: intent='primary', size='md', fullWidth=false
Apply appropriate Tailwind classes for each variant combination.
Create a component that displays a blog post excerpt. The component should accept props for title and htmlContent. Style it as a card with padding and shadow.
Create an Alert component with multiple variants (info, success, warning, error) and sizes (sm, md, lg). When the alert is an error AND large, make the text bold with a thicker border. When it's a success AND small, make the text italic. Include areas for icon, title, and description.
Build a card component that animates into view with fade and slide effects when it appears. Add hover effects that scale or transform the card.
Tested it manually by reducing http_client_options for timeout and now this timeout is getting considered.
Code changes looks good to me. Moving it to RTBC.
Tested it manually and it is working as expected for most of the scenarios.
One issue which I found is that, Scenario 2 from issue summary is not covered here. So basically, if a user has selected hero component which has an empty slot for buttons and asks “Can you add some cards to show case the product highlights to page?” it will add components to it(instead of adding below), without considering slot type.
Also, We need to pass proper keywords as instructions. Eg instead of Can you add some cards to show case the product highlights?, if we ask Using page builder can you add some cards to show case the product highlights? or Can you add some cards to show case the product highlights on this page?, orchestrator will pass the request to correct agent for selected component and it will work as expected for scenario 1.
Code changes looks good to me. Also tested and verified that this MR fixes the PHP warnings.
Component editing is working as expected, selected component context is getting passed correctly. Moving it to RTBC
narendrar → made their first commit to this issue’s fork.
Re, #43 📌 Integrate incremental agent loop execution in XB AI Active ,
- 🐛 Add AiAgentFinishedExecution to all finished execution Needs review will fix it automatically in next tagged release.
- 📌 Orchestrator and sub-agents produce duplicate final messages Active
- 📌 Brief loader flicker before "Thinking" status appears on prompt submission Active
- 📌 "Thinking" status persists after a process terminates with an error Active
Tested it manually and it is working as expected. Now I am getting AiAgentFinishExecution for other agents as well. Moving it to RTBC
Tested it manually with given prompt and it is working as expected.
narendrar → changed the visibility of the branch 3531000-polling-status to hidden.
narendrar → changed the visibility of the branch 3531000-integrate-incremental-agent to hidden.
Tested this with LiteLLM and can confirm that the issue is now fixed.
Tested manually and on saving agent config form again, values can be exported in the desired format. Marking as RTBC.
Everything looks good to me, except for some minor textual changes.
narendrar → made their first commit to this issue’s fork.
I manually tested this by repeatedly calling ::getAllComponentsKeyedBySource and found that subsequent response times were significantly reduced.
RTBC +1
It is working as per all 4 cases defined in issue summary. Moving it to RTBC
narendrar → made their first commit to this issue’s fork.
Tested and changes seems to be working as expected.
#[Hook('js_settings_alter')] used to add permission
Changes looks good to me. Moving it to RTBC.
Displaying so much information is not required which creating menu. Please see attached image.
Tested it again and I am not able to reproduce issue mentioned in #9 🐛 XB AI: AI can hallucinate the prop names Active . Moving it to RTBC.
Changes looks good to me. Moving it to RTBC.
narendrar → changed the visibility of the branch 3543592-component-agent-should to hidden.
narendrar → changed the visibility of the branch 3543504-do-not-include to hidden.
Moving it to RTBC.
Cover 🐛 Canvas AI: Agents fail due to missing entity type and entity ID parameters Active in this issue. I believe that 'Should verify AI panel API request and response payload' could have prevented this problem.
Changes looks good to me.
Re
#4
🐛
Canvas AI: Agents fail due to missing entity type and entity ID parameters
Active
, This navigate was added in
🐛
XB AI: The selected component context should not be passed after page creation task.
Active
to fix context related issue. We should try to update navigate('/editor') to maybe just reload the page, once all operations are performed.
I have suggested some changes. I will test manually once these are fixed.
Tested it manually and component is getting created for content type and fields. Moving it to RTBC
narendrar → changed the visibility of the branch 3533449-users-without-use to hidden.
narendrar → changed the visibility of the branch 3543786-handle-parseexception-in to hidden.
Tested prompt from #6, and it is giving same issue when prompt is used third time.
Looks good to me. Just couple of suggestions.
Cover 🐛 Handle ParseException in XbBuilder Controller Needs work in this issue.
Thanks @Akhil, I was able to reproduce this from first prompt. Verified that this MR fixes the issue. Moving it to RTBC.
Re #4 🐛 Handle ParseException in XbBuilder Controller Needs work , It is possible but writing test for page building task will be covered in 📌 Expand Playwright test coverage for XB AI module Active .
narendrar → created an issue.
I tried to test it with below prompts, but not able to reproduce the original issue.
Could you create a new hero banner component with some text, call-to-action button, and a background image
Create a code component that has a form with yellow background, which takes input of user name and age and on click on submit button gives a success message
The change looks good to me, but it would be helpful to have a prompt which can reproduce this issue, so as to validate the fix.
Did small change in prompt and tested this manually and the response is now less verbose. Marking as RTBC, +1.
narendrar → made their first commit to this issue’s fork.
Although I am not able to test this manually, the prompt looks good to me. It would be helpful if someone with access to OpenAI GPT-4.1 could test and verify this change before moving it to RTBC.
Overall looks good. Some feedback to address.
Also, Input field error message and description is contradicting. See attached SS.
Tested it manually and it is creating component as expected. Moving it to RTBC.
narendrar → created an issue.
This issue still occurs intermittently (about 3 out of 10 times). See attached screenshot:
I also noticed that the generated component sometimes includes unused imports, for example: import Image from 'next-image-standalone';
Tested with the following prompt:
Create an Alert component with a message and an optional type prop (info by default).
Used indexedDB instead
narendrar → made their first commit to this issue’s fork.
Tested manually and it is working as expected. Validation errors are logged and proper structure is generated. Moving it to RTBC.
Re
#6
🐛
Uncaught exceptions when determining solvability of AI request
Active
, I mean to say that we are not getting expected exception when quota is exhausted. We are not getting AiQuotaException instead we get AiRequestErrorException due to which we will get 'Something went wrong' in response which will be not that useful.
Tried to test it manually, but I am getting below error:
Error: Call to undefined method Drupal\xb_ai\XbAiPageBuilderHelper::extractComponentIds() in Drupal\xb_ai\Plugin\AiFunctionCall\SetAIGeneratedTemplateData->execute()
I tested this manually and found that an AiRequestErrorException is thrown when the limit is reached. As a result, it is caught by the generic catch (\Exception $e) block, which is not very helpful in this scenario.
Duplicate of 🐛 Uncaught exceptions when determining solvability of AI request Active
narendrar → created an issue.
Tested it couple of times and it is creating new component once and unable to determine the selected component. See attached SS.
narendrar → made their first commit to this issue’s fork.