Add option to set an inital charset value

Created on 12 January 2024, 10 months ago
Updated 1 February 2024, 10 months ago

Problem/Motivation

We are switching from Swiftmailer to PHPMailer and after installing and configuring of the module, we tested if all mails there send correctly like before. We recognized an issue with German Umlauts in the subject header. The Content of the body was fine.

subject: =?iso-8859-1?Q?Neuer_Post_eingetroffen_-_bitte_pr=C3=BCfen!?=

After some digging, where the charset is set for the mail, I found out that the 'PHPMailer' Class initialize the with public $CharSet = self::CHARSET_ISO88591;

In the PhpMailerSmtp->mail function, there is a check to determine the charset to use for the message. It will look at the $headers['content-type'] value and use the charset, if found. If not, the charset keeps the default value of PHPMailer::CHARSET_ISO88591.

We checked our code and there were some modules there the 'Content-Type' value was simple 'text/html'. Others contains charset definitions like 'text/html; charset=UTF-8;'.

The Problem is now, that most of the modules expect the use of utf-8 but don't declare it at the point of sending a message.

We found that after sending a mail the reset() function is called, where the charset will be set to 'utf-8' which it was not at the beginning. So there seems to be no real reason for iso-8859-1.

Proposed resolution

Add the option to set the initial charset and set the PHPMailer->Charset on creation. The logic to check the content-type header will still work. It will only affect the Fallback if there is no declaration on the message.

Remaining tasks

Apply provided patch.

User interface changes

Add smtp_init_charset field in the "advanced SMTP" settings form and set it to default 'utf-8'

Data model changes

Add and smtp_init_charset field to the schema.

Feature request
Status

Fixed

Version

2.2

Component

Code

Created by

🇩🇪Germany LudwigJP

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @LudwigJP
  • Status changed to Postponed: needs info 10 months ago
  • 🇦🇺Australia imclean Tasmania

    Thanks for the detailed report and patch.

    In the PhpMailerSmtp->mail function, there is a check to determine the charset to use for the message. It will look at the $headers['content-type'] value and use the charset, if found. If not, the charset keeps the default value of PHPMailer::CHARSET_ISO88591.

    Yes, the place to set the charset is in the content-type header.

    We checked our code and there were some modules there the 'Content-Type' value was simple 'text/html'. Others contains charset definitions like 'text/html; charset=UTF-8;'.

    The Problem is now, that most of the modules expect the use of utf-8 but don't declare it at the point of sending a message.

    That is indeed the problem. The sending modules which rely on a specific charset need to set the header correctly, which is the correct solution.

    This module responsible for formatting and sending emails based on the parameters supplied. The intent is not to have config options for every possible SMTP and MIME settings so I'm not keen on adding this.

    That said, I'd be open to setting a default of UTF-8 (as is the case with other headers) while honouring any content-type header. Would this resolve the issue for you?

  • 🇩🇪Germany LudwigJP

    Thank you for your quick reply.

    That said, I'd be open to setting a default of UTF-8 (as is the case with other headers) while honouring any content-type header. Would this resolve the issue for you?

    Yes, that would resolve our problem. Instead of hard-coding to 'utf-8' I added, in my patch, an option to override this setting in the configuration (which installs and defaults to 'utf-8') so a user could change it to a different char set encoding when needed.

  • 🇦🇺Australia imclean Tasmania

    Yes I know what your patch does, but I don't think we should add a new setting for this. The correct way to resolve the issue would be to set the content-type header, not provide a config option to get around problems with other modules.

  • 🇩🇪Germany LudwigJP

    https://www.drupal.org/node/8408

    This article points out that 'utf-8' is the standard charset for all drupal data and should not be changed.

    With this information, i think it would actually be better to initialize PHPMailer hard with 'utf-8' and not to add an setting. So all modules can expect that, if not otherwise specified, that all data is processed as 'utf-8'.

    I'm not quite sure if the assumption that everything is 'utf-8' unless otherwise specified is universal, but I think other modules don't specify 'charset=utf-8' because it's redundant for them because it's the standard value for them.

    The hardcoded 'utf-8' charset would be viable for us. On second thought, I don't think there will be a need to change it to something other than that.

  • Status changed to Needs review 10 months ago
  • 🇦🇺Australia imclean Tasmania

    OK see if this works for you.

  • 🇦🇺Australia imclean Tasmania

    This is better.

  • Status changed to Postponed: needs info 10 months ago
  • 🇩🇪Germany LudwigJP

    Thank you for your patience.
    We tested #7 and it works great.

    • imclean committed aa68e063 on 2.2.x
      Issue #3414290 by imclean, LudwigJP: Add option to set an inital charset...
  • Status changed to Fixed 10 months ago
  • 🇦🇺Australia imclean Tasmania

    Thanks, committed to latest dev.

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024