Fix outdated Unicode requirements check

Created on 19 March 2015, over 9 years ago
Updated 14 June 2023, about 1 year ago

Problem/Motivation

Mixing encodings is a bad idea, as it can lead to bugs, garbled input/output, storing bad data and security issues, so we want to use UTF-8 everywhere and check for this in the Unicode requirements check.

Over in #2332295: Unicode requirements check not working with PHP 5.6 β†’ we added support for http_input and http_output for < 5.6 in Unicode::check() be we also want to add support for the newly added internal_encoding, input_encoding and output_encoding for 5.6+.

Per the RFC the precedence of settings for the mb_*, iconv_*, htmlentities functions is:
default_charset < php.* < mbstring.*/iconv.* < encoding specified by functions

So the values must be set as follows:

PHP <5.6

  • mbstring.http_input = pass (set in our .htaccess)
  • mbstring.http_output = pass (set in our .htaccess)
  • mbstring.internal_encoding = irrelevant, we set it ourselves
  • iconv.internal_encoding = empty (default) or utf-8
  • iconv.output_encoding = empty (default) or utf-8

PHP >=5.6

Same as before, except now mbstring.* and iconv.* are deprecated so they may also be empty. Newly introduced variables:

  • input_encoding (misnamed in PHP 5.6/7, the RFC says php.input_encoding) = empty or utf-8
  • internal_encoding (misnamed in PHP 5.6/7, the RFC says php.input_encoding) = empty or utf-8
  • output_encoding (misnamed in PHP 5.6/7, the RFC says php.input_encoding) = empty or utf-8

@see

  1. https://wiki.php.net/rfc/default_encoding
  2. http://php.net/manual/en/mbstring.configuration.php#ini.mbstring.interna...

Proposed resolution

Update Unicode::check() to also test for the newly introduced *_encoding options in PHP 5.6+

Remaining tasks

Review patch

User interface changes

n/a

API changes

n/a

πŸ“Œ Task
Status

Needs work

Version

11.0 πŸ”₯

Component
BaseΒ  β†’

Last updated less than a minute ago

Created by

πŸ‡¦πŸ‡ΊAustralia benjy

Live updates comments and jobs are added and updated live.
  • Needs issue summary update

    Issue summaries save everyone time if they are kept up-to-date. See Update issue summary task instructions.

  • Needs change record

    A change record needs to be drafted before an issue is committed. Note: Change records used to be called change notifications.

  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024