AI agent sometimes fails to modify code components immediately after creation

Created on 9 July 2025, 10 days ago

Overview

It was discovered in 🐛 [PP-1] Chat history not getting preserved for Deepchat on re-rendering Active that the AI agent, in some cases or under certain configurations, is not able to make changes to a code component directly after creating it.

I'm consistently able to reproduce the issue using OpenAI Provider as my only provider and using the default settings at /admin/config/ai/settings that are offered after installing it.

⚠️ @utkarsh_33 and @kunal.sachdev haven't been able to reproduce the problem using other providers than OpenAI.

Steps to reproduce

  1. Install the OpenAI Provider module, configure an API key at /admin/config/ai/providers/openai.
  2. Go to AI Settings (/admin/config/ai/settings) and make sure OpenAI is configured as the default provider for the following operation types with the respective default models:

    • Chat with Complex JSON: gpt-4o
    • Chat with Structured Response: gpt-4.1
    • Chat with Tools/Function Calling: gpt-4.1
  3. Open XB and the AI sidebar, enter a prompt to create a new code component. E.g.: Build a new hero component with a heading and subheading.
  4. After the component gets created, ask for a change. E.g.: Change the background to a solid blue color.
  5. Notice that the agent is not able to determine the component for which it is supposed to make the change. This is usually indicated by its response, but sometimes it goes ahead and creates a new component. I believe the latter can be traced back to the same reason: the agent is getting confused about what component to work with.

The same issue doesn't happen when an existing component is opened and changes are asked from the agent.

Proposed resolution

TBD

User interface changes

n/a

🐛 Bug report
Status

Active

Version

0.0

Component

AI

Created by

🇳🇱Netherlands balintbrews Amsterdam, NL

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @balintbrews
  • 🇮🇳India Akhil Babu Chengannur

    akhil babu made their first commit to this issue’s fork.

  • @akhil-babu opened merge request.
  • 🇮🇳India Akhil Babu Chengannur

    The agent uses the ai_agent_get_js_component tool to load existing components. However, it was using the component’s name directly, and if that name contained uppercase letters, the tool returned null because component IDs are always lowercase.

    It's working now.

  • 🇮🇳India Akhil Babu Chengannur
  • 🇮🇳India narendraR Jaipur, India

    These changes look good and are harmless, but I’m not sure why this issue occurs in the first place for OpenAI. The selected_component (as machineName) is already passed as context to the agent, so the agent should use the correct machine name from the context.

    Perhaps we should consider updating the system_prompt for ai_agents.ai_agent.experience_builder_component_agent.yml (Case: The tools you have to use for editing components are:) to ensure the provided context is explicitly taken into account.

  • 🇮🇳India Akhil Babu Chengannur

    Yes, we could update the prompt and give explicit instructions to use the selected_component value for loading the component.

    Currently, there’s no way for the agent to understand the ID of the code component. For example, I created a 'PieChartComponent' and asked the agent to modify it. In that case, the instruction given by the orchestrator agent to the code component builder agent was:

    Task Title: 
    Task Author: 
    Task Description:
    Change the text color of the PieChartComponent to yellow. Make sure to update the Tailwind CSS classes accordingly in the component's code to reflect this change.
    --------------------------

    So the model called the tool like this:

    Tool: ai_agent_get_js_component
    component_name: PieChartComponent

    We could try using the [xb_ai:selected_component] token in the code component generator agent as well.

    That said, I’d still recommend keeping the code changes, as they help ensure nothing breaks even if the model hallucinates and uses the wrong ID.

Production build 0.71.5 2024