Add an setting that is required to set to override prompts

Created on 13 January 2025, 8 days ago

Problem/Motivation

The assistant is dependent on three different components for its system message.

  • The system prompt, which is the parent component that is always running and loading the two texts under and also giving some context like the user role, current path etc.
  • The pre-action prompt which are instructions on the tools/agents/RAG/plugins the assistant can use. The only are available on the pre-action stage.
  • The instructions, which are extra instructions on top of both this, that should always run.

Out of these the system prompt and the pre-action prompt are set to a fairly working default text when you create new assistant and also already today requires you to change a settings value to be able to edit at all. Our estimation is that most people will not touch this at all.

The instructions, together with the tools it can use, are what makes the assistant specific for what it is doing in term of language, formatting, length of answers etc.

The other two prompts are however part of the configuration that useally goes into a recipe, like the AI recipe in Drupal CMS. This means that if we add a fixed version there, we will not be able to change it on the systems that have already installed it. This was never the purpose.

The is rather that people that know how to write prompts, can override also the system prompt and pre-action prompt, while most sites would be ok with having prompts with incremental improvements.

The solution for this would be the same as with the setting for modifying this and basically having a setting for running the overriden pre-action prompt and system prompt. This means that people that are comfortable with changing in settings.php will still have this option.

Proposed resolution

  • Add a settings called ai_assistant_custom_prompts that has to be set to TRUE to run the overriden prompts.
  • Add a description for this.
  • Document it on https://project.pages.drupalcode.org/ai/
  • Make sure that the pre-action prompt and system prompt that is located in text files, otherwise runs from cache.
Feature request
Status

Active

Version

1.0

Component

AI Assistants API

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @marcus_johansson
  • First commit to issue fork.
  • Pipeline finished with Failed
    7 days ago
    Total: 160s
    #395805
  • 🇬🇧United Kingdom justanothermark

    Created an MR for this.

    I also want to suggest creating a follow-up ticket to remove the `$include_pre_prompt` parameter from `AssistantMessageBuilder::buildMessage()`. We already pass the AIAssistant to this parameter so the Message Builder can check this directly to see whether or not to include the pre prompt. In turn, this means we can simplify `AiAssistantApiRunner::process` and keep all references to the prompt files in a single place. However, this would introduce backwards compatibility issues so I haven't included it as part of the MR.

  • 🇩🇪Germany marcus_johansson

    Thanks Mark - I did realize one thing that I forgot to put in the proposed resolution that we should add here,

    If they actual are overriding already, we can assume at the moment that they are using a custom prompt. We should create a update hook that outputs a warning about it, so they have seen it? Or some other way of letting them know that this has happened, otherwise I'm not sure we can add it into 1.0 branch, since it is changing running sites.

  • 🇬🇧United Kingdom justanothermark

    Ah yeah, that makes sense @marcus_johansson. I've added an update hook and implemented hook_requirements to provide warnings with links to the documentation if the site is using advanced mode but doesn't have custom prompts enabled.

  • Pipeline finished with Failed
    6 days ago
    Total: 162s
    #396581
  • 🇮🇳India akulsaxena

    The changes LGTM
    But there seem to be some PHPCS error in the file. Seems like a single small error.
    can you please fix it, moving it to NW for now

  • 🇬🇧United Kingdom justanothermark

    My mistake, I'd fixed that but not pushed the change. Updated now.

  • Pipeline finished with Success
    6 days ago
    Total: 347s
    #396662
  • 🇮🇳India akulsaxena

    Looks good now
    Setting to RTBC+

Production build 0.71.5 2024