Contrib.social

Create a generic plugin layer for CKEditor

Open on Drupal.org →
Created on 19 July 2024, 9 months ago
Updated 13 August 2024, 8 months ago

Problem/Motivation

The CKEditor code from OpenAI worked to an extent but falls short on maintenance and extendability. Part of what I was waiting on was to see if the CKEditor API introduced better UI layers for enhancing what you see, better dialogs, better ways of selecting options, dialog handling, etc. As it stands, the CKEditor UI API seems too complex for fast changes.

What we should have is a generic plugin layer that lets you register editor functionality via PHP plugins, and their actions open in Drupal dialogs and returned via AJAX. This would reduce the code needed for pure CKEditor JS and move it back to Drupal which is more native for this audience.

That way, we can provide the current actions as PHP plugins and the community could extend it further. Also, we could get the ability to do 'retry' or 'regenerate' responses if the initial answer for an action is not satisfactory.

Proposed resolution

Implement a plugin layer that can be used. The default plugin should pre configure and handle Drupal dialog handling so any implemented plugins just need to provide the form and submission action.

Here is partial WIP code of what a plugin form would do.

When selecting an action in the CKEditor dropdown in the editor, the JS fires a callback to a Drupal route (provided by the plugin layer somehow). The response is a Drupal form within the dialog.

This is the Tone action converted to a FormBase form:

  public function buildForm(array $form, FormStateInterface $form_state) {

    $form['tone'] = [
      '#type' => 'select',
      '#title' => t('Choose tone'),
      '#description' => t('Selecting one of the options will adjust/reword the body content to be appropriate for the target audience.'),
      '#options' => [
        'friendly' => t('Friendly'),
        'professional' => t('Professional'),
        'helpful' => t('Helpful'),
        'easier for a high school educated reader' => t('High school level reader'),
        'easier for a college educated reader' => t('College level reader'),
        'explained to a five year old' => t('Explain like I\'m 5'),
      ],
    ];

    $form['actions'] = [
      '#type' => 'actions',
    ];

    $form['actions']['submit'] = [
      '#type' => 'button',
      '#value' => $this->t('Submit'),
      '#ajax' => [
        'callback' => [$this, 'ajaxSubmitForm'],
      ],
    ];

    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {

  }

  /**
   * {@inheritdoc}
   */
  public function ajaxSubmitForm(array &$form, FormStateInterface $form_state) {
    $response = new AjaxResponse();
    $response->addCommand(new EditorDialogSave([
      'attributes' => [
        'prompt' => 'Change the tone of the following text to be more ' . $form_state->getValue(['tone']) . ' using the same language as the following text: ',
        'endpoint' => 'api/openai-ckeditor/completion',
        'options' => [
          'model' => 'gpt-3.5-turbo',
          'max_tokens' => 2048,
          'temperature' => 0.4,
        ]
      ],
    ]));
    $response->addCommand(new CloseModalDialogCommand());
    return $response;
  }

$response->addCommand(new EditorDialogSave (https://api.drupal.org/api/drupal/core%21modules%21editor%21src%21Ajax%21EditorDialogSave.php/11.x) is the key part that puts the response directly back into the editor. 

/**
   * Command to save the contents of an editor-provided modal.
   *
   * This command does not close the open modal. It should be followed by a
   * call to `Drupal.AjaxCommands.prototype.closeDialog`. Editors that are
   * integrated with dialogs must independently listen for an
   * `editor:dialogsave` event to save the changes into the contents of their
   * interface.
   *
   * @param {Drupal.Ajax} [ajax]
   *   The Drupal.Ajax object.
   * @param {object} response
   *   The server response from the ajax request.
   * @param {Array} response.values
   *   The values that were saved.
   * @param {number} [status]
   *   The status code from the ajax request.
   *
   * @fires event:editor:dialogsave
   */
  Drupal.AjaxCommands.prototype.editorDialogSave = function (
    ajax,
    response,
    status,
  ) {
    $(window).trigger('editor:dialogsave', [response.values]);
  };

At this point it would not be a stretch to add additional dialog action buttons for Re-generate, Save to (prompt entity?), etc. This should help drive innovation around this instead of force everyone to learn CKEditor 5 JS API layer which is not really happening at large.

Remaining tasks

tbd

User interface changes

The CKEditor AI drop down would look the same, but the option list would be derived from registered plugins and not hardcoded.

API changes

tbd

Data model changes

tbd

✨ Feature request
Status

Fixed

Version

1.0

Component

Code

Created by

🇺🇸United States kevinquillen

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @kevinquillen
  • Comment 9 months ago →
    🇺🇸United States kevinquillen
  • Comment 9 months ago →
    🇺🇸United States kevinquillen

    Initially, we need:

    • Generic plugin manager
    • Generic plugin
    • Way of communicating what provider that plugin uses
    • Create plugin configuration from CKEditor admin config screen
    • Possibly make those plugins permissions-enabled
  • Comment 9 months ago →
    🇺🇸United States kevinquillen
  • Comment 9 months ago →
    🇺🇸United States kevinquillen
  • Comment 9 months ago →
    🇩🇪Germany marcus_johansson

    Maybe I'm abstracting to much here and without knowing exactly how CKEditor returns the output (at cursor, over marked field etc), my guess is that its better if the plugin manager or an abstract base class takes care of the actual AjaxResponse and the plugin just needs to responde with some normalized outputs that could take text, html and potentially even images and files.

    If there are some specified way of doing the output, like replace marked text or add at cursor, it could then be set a attributes or in some method of the plugin. That way, whoever creates a module doesn't need to learn how the different types of AjaxResponse works together with CKEditor.

    So you basically have:
    * Form method
    * Form Validation method
    * Form Submit method, that should return a object that can include text, html or image.

    I'm not sure if there is some nifty way to also move marked text or marked image etc, into the form via context somehow. That would be cool if it was an easy solve.

    Off tangent but what would also be awesome - if it is possible on top of that to have multiple plugin discovery in one plugin via a method, you can then expose the disposable entities from the Automator via a bridge module, so you can have very specific solution setup in minutes or via recipes. I could help you with this.

    If that exists what we can do is (I hope its possible to follow):
    1. Enable AI CKEditor, AI Automator, Unstructured and AI Automator CKEditor
    2. Create a Automator entitiy bundle that you set to disposable (as in that its deleted after it runs) that has a file field, a text long and a formatted text long field.
    3. Setup a rule on the text long field with the Automator "Unstructured: File to Markdown" so it fetches the content.
    4. Setup a rule on the formatted text long field with the Automator "LLM: Prompt" where it prompts it to extract any tables and create it as HTML table markdown.
    5. Then there is a configuration for CKEditor Automator where you choose that you want to expose this Automator Chain, you set that the input is the file field and the output is the formatted text field. Give it a title (and icon?) and save.
    6. Go to CKEditor and its available to use, a form popups with the file form, you upload a PDF/JPG/Word file that has some table in it and it takes this and puts it formatted in the CKEditor field where the marker is.

    (And this is just one out of probably 1000s of plugins you could dynamically setup without a single line of code)

  • Comment 9 months ago →
    🇧🇪Belgium wouters_f Leuven

    I love the idea and it absolutely makes sense.

  • Comment 9 months ago →
    🇧🇪Belgium wouters_f Leuven

    I think the same should also apply for the assistants in the sidebar. (ai_content).

  • Merge request !28Resolve #3462674 "Create a generic plugin layer for CKEditor" → (Merged) created by scott_euser
  • First commit to issue fork.
  • Status changed to Needs review 8 months ago
  • Comment 8 months ago →
    🇺🇸United States kevinquillen

    I think this is in a pretty good spot to review. Some plugins may not have configuration, like Help, but I think that is okay.

  • Comment 8 months ago →
    🇧🇪Belgium wouters_f Leuven

    I would like to apply this too for https://www.drupal.org/project/ai/issues/3462089 ✨ AI powered Spelling Fixer in ckeditor Needs review
    but currently I don't know how to add my plugin.
    Is there any docs I can add to the readme or something?

  • Comment 8 months ago →
    🇺🇸United States kevinquillen

    @Marcus I think we are ready here and it is in good enough shape. I left some comments and made some final additions. One annoying thing is that the entity autocomplete element in Drupal allows multiple inputs even if you limit it to 1, but that seems to be how core behaves.

  • Comment 8 months ago →
    🇺🇸United States kevinquillen

    #12 plugins can no longer be made in native CKEditor JS. You would have to review this MRs PHP plugins for an example. The closest one is probably the Completion or Summarize plugin in src/Plugin/AiCKEditor.

  • Comment 8 months ago →
    🇩🇪Germany marcus_johansson

    @kevinquillen - Looks good, feel free to merge with dev and set the ticket to fixed.

  • Pipeline finished with Skipped
    8 months ago
    #234092
  • Comment 8 months ago →
    System Message
  • Status changed to Fixed 8 months ago
  • Comment 8 months ago →
    🇺🇸United States kevinquillen
  • Comment 8 months ago →
    🇧🇪Belgium wouters_f Leuven

    I've changed my spellcheck plugin for this new code.
    But while testing I must say that personally the user interface of the new layer is substantially less easy to use than what is previously was.
    The amount of clicks needed just to fix my spelling has tripled.
    Also it does not seem to be streaming anymore?

  • Comment 8 months ago →
    🇧🇪Belgium wouters_f Leuven

    Also the newlines in the suggested text are removed when clicking "save changes to editor" resulting in one big blob of text.

  • Comment 8 months ago →
    System Message
  • Comment 8 months ago →
    🇳🇱Netherlands jurriaanroelofs

    Amazing work on this, with this API it will be easier to create and extend plugins.

    I do agree with @wouters_f that the decision to load the selectbox with plugins in a popover is not a good user experience. It would be better to load those options in the native CKEditor dropdown as before, and when clicking them we load the Drupal popover. Did you try that and run into any trouble doing it this way @kevinquillen?

  • Comment 8 months ago →
    System Message
  • Comment 8 months ago →
    System Message
  • Comment 8 months ago →
    🇺🇸United States kevinquillen

    I generally don't care to write JS for CKEditor, thats why. I put a POC in ✨ Add new CKEditor plugins to the drop down button in CKEditor toolbar Active , but it will need more work to be completed.

  • Comment 8 months ago →
    🇳🇱Netherlands jurriaanroelofs

    Thx I'll take a look today!

  • Comment 8 months ago →
    System Message

    Automatically closed - issue fixed for 2 weeks with no activity.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024