Incorrect use of FormattableMarkup in logger messages

Created on 6 February 2023, almost 2 years ago
Updated 11 April 2023, over 1 year ago

Problem/Motivation

\Drupal\ckeditor5\SmartDefaultSettings is using FormattableMarkup with log messages, I don't think this is correct.

By using FormattableMarkup the params are being combined into the message and can no longer be accessed by loggers.

Steps to reproduce

Proposed resolution

Remove the use of FormattableMarkup in log messages, pass the message as the first param and the params as the second.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

๐Ÿ› Bug report
Status

Fixed

Version

9.5

Component
CKEditor 5ย  โ†’

Last updated 1 day ago

Created by

๐Ÿ‡ฆ๐Ÿ‡บAustralia larowlan ๐Ÿ‡ฆ๐Ÿ‡บ๐Ÿ.au GMT+10

Live updates comments and jobs are added and updated live.
  • Novice

    It would make a good project for someone who is new to the Drupal contribution process. It's preferred over Newbie.

Sign in to follow issues

Comments & Activities

  • Issue created by @larowlan
  • Assigned to royalpinto007
  • Status changed to Needs work almost 2 years ago
  • @royalpinto007 opened merge request.
  • Status changed to Needs review almost 2 years ago
  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia royalpinto007

    I just wanted to inform you that I have removed the use of FormattableMarkup in log messages and have instead passed the message as the first parameter and the parameters as the second. This change ensures a more consistent and efficient logging system. Let me know if there are any questions or concerns.

  • ๐Ÿ‡ซ๐Ÿ‡ทFrance andypost

    probably it needs some test or runtime assertion to catch it

  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia royalpinto007

    It fails for 1 test, how to fix that?

  • First commit to issue fork.
  • Issue was unassigned.
  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    Hi @royalpinto007!

    I've added the string messages with parameters into $this->t() function. It seems like if we use it directly into the loggers, it will break the tests.

  • Status changed to Needs work almost 2 years ago
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom longwave UK

    The logger methods should take an untranslated string as the first argument and the context array as the second argument; the untranslated string and context is stored, and translation is handled at display time, so the logs can be viewed in different languages.

  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    Thanks @longwave for the feedback!

    It seems like this approach is currently used in CKEditor5.

    The only logger method that is not being passed the FormattableMarkup, is using it.

    See: https://git.drupalcode.org/project/drupal/-/blob/9.5.0/core/modules/cked...

  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    To give more context on this, I was debugging the test() in SmartDefaultSettingsTest class, and when we use the logger method passing the string as the first argument and the context as second, the context arguments are not being transformed into the string when they are being called in the tests.

    This makes me think that by not using the FormattableMarkup, the messages are being stored in the watchdog table as untransformed strings. But I don't really understand why...

  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    You are right @gxleano, the messages are stored as they are, without the replacements applied. See the "log" method here: https://api.drupal.org/api/drupal/core%21modules%21dblog%21src%21Logger%...

    I think, in this case, the test might need to be updated too as part of this issue, as it assumes that the stored value in DB has the replacements, and that's not correct. The test checks the array entries "expected_db_logs" with the context already replaced.

    So, the way I see it, the initial fix from @royalpinto007 was valid, but it also needed to have the tests updated. Maybe you want to give it a go @gxleano? Note that the $this->t commit will need to be reverted.

  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    Thanks @fjgarlin for the feedback!

    Sure, I will try to rework the "expected_db_logs" and revert $this->t() changes.

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    Woah, great catch! Looks like this is getting close ๐Ÿ˜Š

  • Status changed to Needs review almost 2 years ago
  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    I've updated the kernel test in order to do it compatible with the logger context.

  • Status changed to Needs work almost 2 years ago
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom longwave UK

    I think the test cases should stay the same, otherwise we are losing coverage of the contextual values; the test should format the message and then check that the final string is as expected.

  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    the test should format the message and then check that the final string is as expected.

    @longwave - in this part of the test https://git.drupalcode.org/project/drupal/-/blob/8aaf79d53b2251e8bd3d9b2... we are storing the "message" column as it is in the database, without replacements.

    Wouldn't it make sense to compare the same thing (aka without the replacements applied) in the tests?

    Otherwise, I guess we'd need to apply the replacements in those lines (getting the variables), and then leave the strings in the test unchanged. Is that the approach that you suggest?

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom longwave UK

    Yes, I mean to apply the replacements - the select statement already reads the variables column, but then does nothing with it.

  • First commit to issue fork.
  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    @longwave

    I've tried to apply the replacements with the variables column, but there is a problem. The values coming from the variables column are different than the provided data in the test.

    For instance:

    • value coming from variables: Basic HTML
    • expected value: Basic HTML

    So, the test are trying to compare the next:

    --- Expected
    The CKEditor 5 migration enabled the following plugins to support tags that are allowed by the Basic HTML text format: Code (for tags: <code>). The text format must be saved to make these changes active.

    +++ Actual (With replacements applied)
    The CKEditor 5 migration enabled the following plugins to support tags that are allowed by the Basic HTML text format: Code (for tags: ). The text format must be saved to make these changes active.

  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    Which code are you using to do the replacements?

  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres
        $db_logged = $this
          ->database
          ->select('watchdog', 'w')
          ->fields('w', ['message', 'variables', 'severity'])
          ->condition('type', 'ckeditor5')
          ->orderBy('wid')
          ->execute()
          ->fetchAll();
    
        $type_to_status = [
          6 => 'status',
          4 => 'warning',
        ];
        $db_logs = [];
        foreach ($db_logged as $log) {
          $message = $log->message;
          $variables = unserialize($log->variables);
          foreach ($variables as $key => $value) {
            if (str_contains($message, $key)) {
              $message = str_replace($key, $value, $message);
            }
          }
          $db_logs[$type_to_status[$log->severity]][] = $message;
        }
    
  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    But on my eyes, it seems to be normal, because of it is storing the format name, without html.

    '%text_format' => $editor->getFilterFormat()->get('name'),

  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    Try something like this better (not tested):

          $variables = unserialize($log->variables);
          $message = new FormattableMarkup($log->message, $variables);
    

    You'll need to add use Drupal\Component\Render\FormattableMarkup; to the top of the test file.

    % or @ have different meanings when replacing, and the class should take care of it, instead of doing a "str_replace".

  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    Thanks @fjgarlin for the hint.

    I've refactored this piece of code and test are passing now.

    $variables = unserialize($log->variables);
    $message = new FormattableMarkup($log->message, $variables);
    $db_logs[$type_to_status[$log->severity]][] = (string) $message;
  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    Great to hear that! Things are looking better and better.

    You might need to change your editor autosaving/autoformatting to _not_ change the other parts of the code that are not affected / relevant that were triggering the coding standards. Ideally, the MR should only reflect the changes needed to fix the issue.

    Is the (string) casting needed for the tests to pass?

  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres

    I have reverted the unnecessary changes.

    And yes, the casting (string) is necessary to make the test pass.

  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    Awesome, tests are running as we type and the MR looks much cleaner now. Great work!

  • Status changed to Needs review almost 2 years ago
  • ๐Ÿ‡ช๐Ÿ‡ธSpain gxleano Cรกceres
  • Status changed to RTBC almost 2 years ago
  • ๐Ÿ‡ช๐Ÿ‡ธSpain fjgarlin

    I checked the code and it all looks good, and also tested everything manually via Drupalpod as follows:
    - I uninstalled ckeditor5, enabled ckeditor.
    - Then installed again ckeditor5 and assigned it to a text format (to trigger all the logging messages).
    - Went to the log messages section and the messages showed up as expected, with the replacements properly parsed.

    See the two images below:

    Marking this as RTBC.

  • Status changed to Fixed almost 2 years ago
  • ๐Ÿ‡ฆ๐Ÿ‡บAustralia larowlan ๐Ÿ‡ฆ๐Ÿ‡บ๐Ÿ.au GMT+10

    Thanks folks, committed to 10.1.x and backported to 10.0.x and 9.5.x for consistency as the risk of disruption is low.

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • Status changed to Fixed over 1 year ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bnjmnm Ann Arbor, MI

    Adjusting issue credit as credit is not provided for button-click rebases without additional contribution.

Production build 0.71.5 2024