HTML in email body is not rendered => one time login link gets messed up

Created on 7 February 2024, over 1 year ago

Before the update to Drupal 10, we were using the simple_html_mail module, and had to switch to the symfony_mailer module since the former is no longer maintained.

The customers reported that they couldn't use the "Password forgotten" feature anymore, as the link they recieved is not valid:

This is the text as seen in the email:

<a href="http://10.0.0.107:10501/user/reset/6139/1707234465/8Q2sf1WdhTOJs-0elC5f…;

this is the html behind this:

&lt;a href="<a href="http://10.0.0.107:10501/user/reset/6139/1707234465/8Q2sf1WdhTOJs-0elC5fHgbcWgRcemLOiiz56KZ8_EM&quot;>http://10.0.0.107:10501/user/reset/6139/1707234465/8Q2sf1WdhTOJs-0elC5fHgbcWgRcemLOiiz56KZ8_EM</a><br/><br/&amp;gt">http://10.0.0.107:10501/user/reset/6139/1707234465/8Q2sf1WdhTOJs-0elC5f…</a>;

The expected behaviour should be this:

The text as seen in the email:

http://10.0.0.107:10501/user/reset/6156/1707298374/VhTohKaT_6qmfpltMkU6…

this is the html behind this:

<a href="http://10.0.0.107:10501/user/reset/6156/1707298374/VhTohKaT_6qmfpltMkU6AJkkLEiFa6qI28WUgkDSABo">http://10.0.0.107:10501/user/reset/6156/1707298374/VhTohKaT_6qmfpltMkU6…</a>

I found a fix for this behaviour at line 785 of user.module file:

BEFORE

  $message['subject'] .= PlainTextOutput::renderFromHtml($token_service->replace($mail_config->get($key . '.subject'), $variables, $token_options));
  $message['body'][] = $token_service->replace($mail_config->get($key . '.body'), $variables, $token_options);

AFTER

  $message['subject'] .= PlainTextOutput::renderFromHtml($token_service->replace($mail_config->get($key . '.subject'), $variables, $token_options));
  $message['body'][] = PlainTextOutput::renderFromHtml($token_service->replace($mail_config->get($key . '.body'), $variables, $token_options));

And so I created a patch that applies this change and linked it to this issue.

πŸ› Bug report
Status

Active

Version

10.1 ✨

Component
User moduleΒ  β†’

Last updated about 13 hours ago

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Status changed to Needs review over 1 year ago
  • last update over 1 year ago
    29,722 pass
  • Status changed to Needs work over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Will need a test case showing the issue.

    Also can the issue summary be converted to use the standard template please.

  • First commit to issue fork.
  • πŸ‡«πŸ‡·France prudloff Lille

    I am getting a similar problem when using symfony_mailer and sending a password reset mail containing the [site:name] token.
    The LegacyEmailBuilder from symfony_mailer seems to assume that all core email are plain text.
    But user_mail() adds an HTML body to the email (because it uses Token::replace() and not Token::replacePlain().

    When using core to send emails you don't notice it because PhpMail::format() then removes the HTML.
    But when using symfony_mailer, this HTML is double encoded.

    LegacyMailerHelper::formatBody() from symfony_mailer converts the message body to a #processed_text element without specifying a format, so the default format is used.
    On most websites, the default form will be plain_text with the FilterHtmlEscape filter enabled.
    This means if the website name contains a ', user_mail() will convert the token to a string containing &#039; and FilterHtmlEscape will double escape it as &amp;#039;.

    Is symfony_mailer correct to assume all core emails are plain text? If so, user_mail() should add a plain text body to the message.

    @ot1201 instead of using PlainTextOutput::renderFromHtml($token_service->replace()), it is probably cleaner to use $token->replacePlain().

  • Merge request !10611Keep user emails plain text β†’ (Closed) created by prudloff
  • Pipeline finished with Success
    5 months ago
    Total: 359s
    #372824
  • First commit to issue fork.
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    Started work on test coverage. Pushed to the MR.

  • Pipeline finished with Failed
    5 months ago
    Total: 183s
    #373233
  • Pipeline finished with Failed
    5 months ago
    Total: 254s
    #373249
  • Pipeline finished with Success
    5 months ago
    Total: 1603s
    #373261
  • Pipeline finished with Failed
    5 months ago
    Total: 645s
    #373279
  • Pipeline finished with Canceled
    5 months ago
    Total: 187s
    #373288
  • Pipeline finished with Success
    5 months ago
    Total: 1385s
    #373290
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    Test coverage is in place, removing tag.

    The test-only test is passing. This suggests that there are no HTML entities contained in the reset password email. The issue description states,

    Before the update to Drupal 10..

    That implies that the issue is fixed in Drupal 11.x..

  • Pipeline finished with Success
    5 months ago
    Total: 3036s
    #373334
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    It seems there is no problem with the password reset email text in current Drupal with default email setup. The problem is when symfony_mailer module (as mentioned in the issue summary) is installed and configured for email. This needs to be made clear in the 'steps to reproduce'.

  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    @prudloff Can you please specify which version of Drupal and Symfony mailer you were using in #5. If possible can you please provide screenshots to compare with my own.

  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    Setting issue status to 'Needs review'. Unless the test is deemed useful as a guard against future problems with email as described in the issue summary I think the reviewer may consider postponing this issue for further information or closing it since it appears to be already fixed in core.

  • πŸ‡«πŸ‡·France prudloff Lille

    I am using Drupal 11.0.9 with symfony_mailer 1.5.0.
    I don't have the problem with the reset link reported by @ot1201. But I have a double encoding problem when the site name contains a ' character.

    I am attaching screenshots of the problem.

  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    @prudloff Ah thanks! In that case I think we can broaden the scope of this issue a bit. The existing issue summary does provide before and after code which shows that a test for HTML entities in the body of the reset email would cover the issue. But the 'trigger' for the issue seems to have changed. It is not triggered by the link in the email body any more.

    Now it is more of an edge case. As you mention, it is triggered by the single quotation character. So I think the way ahead is to change the test so it generates a test email with a body containing a quotation mark. Would be worth including a few other similar characters like double quotation marks, things that the exiting fix in the MR will convert from HTML entities back to the plain text versions.

    Changing status to 'Needs work'.

  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • Pipeline finished with Failed
    5 months ago
    Total: 162s
    #375815
  • Pipeline finished with Failed
    5 months ago
    Total: 966s
    #375823
  • Pipeline finished with Success
    5 months ago
    Total: 1041s
    #375850
  • Pipeline finished with Success
    5 months ago
    Total: 890s
    #375868
  • Pipeline finished with Failed
    5 months ago
    Total: 238s
    #375887
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    Manual testing in D11 without the fix: screenshot of html entities in email β†’

  • Pipeline finished with Success
    5 months ago
    Total: 3951s
    #375896
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    Change to 'needs review', hope to also get advice on test coverage.

  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡¬πŸ‡§United Kingdom oily Greater London

    Improved the issue summary, removing the tag. Pipeline is running all green.

    Set to needs review. Reviewer may be able to suggest how to make the test-only test fail without the Symfony mailer module installed.

  • πŸ‡¬πŸ‡§United Kingdom oily Greater London
  • πŸ‡«πŸ‡·France prudloff Lille

    prudloff β†’ changed the visibility of the branch 11.x to hidden.

  • Pipeline finished with Failed
    3 months ago
    Total: 184s
    #448452
  • Pipeline finished with Success
    3 months ago
    Total: 416s
    #448458
  • πŸ‡«πŸ‡·France prudloff Lille

    @oily thanks for the tests!

    I don't think it is useful to duplicate the whole test scenario in the FunctionalJavascript test. This one should only test AJAX-specific behavior.
    IMHO the current Functional test is enough.

    But the test will always pass until tested with the Symfony mailer contributed module installed and configured. It is necessary to find a way of applying a test-only test that fails.

    I added a lower level test that can fail even without symfony_mailer.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Seems close! Left some comments on the MR

    If you are another contributor eager to jump in, please allow the previous poster(s) at least 48 hours to respond to feedback first, so they have the opportunity to finish what they started!

  • Pipeline finished with Success
    2 months ago
    Total: 928s
    #454865
  • πŸ‡«πŸ‡·France prudloff Lille

    I refactored the tests to avoid avoid having methods that are only called once.

  • πŸ‡«πŸ‡·France prudloff Lille

    Renamed to better correspond to the current issue summary

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Feedback appears to be addressed.

  • πŸ‡«πŸ‡·France arousseau

    A patch working for D10.4 containing only the one line fix. Could be of use to someone.

  • The Needs Review Queue Bot β†’ tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide β†’ to find step-by-step guides for working with issues.

  • πŸ‡«πŸ‡·France prudloff Lille

    I think the bot tested the patch instead of the MR.
    The MR does not have conflicts.

    • catch β†’ committed 0c336f0c on 11.x
      Issue #3419707 by oily, prudloff, ot1201, smustgrave: user module sends...
  • πŸ‡¬πŸ‡§United Kingdom catch

    Committed/pushed to 11.x, thanks!

    This doesn't apply to 10.5.x, so moving to 'to be ported'.

  • Status changed to Downport 11 days ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    10.5 window still open

  • Pipeline finished with Failed
    10 days ago
    Total: 150s
    #502931
  • Pipeline finished with Failed
    10 days ago
    Total: 107s
    #502946
  • Pipeline finished with Success
    10 days ago
    Total: 404s
    #502947
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Clean backport.

Production build 0.71.5 2024