Language detection not working correctly for most Chinese readers (and add a user interface for all browser language mappings)

Problem/Motivation

Browsers understand language codes differently to Drupal. The issue was originally created to because this causes serious problems for Simplified Chinese. IE and Firefox will send 'zh-cn', but Drupal understands this language as 'zh-hans'. However this applies to other languages as well.

Proposed resolution

A mapping system to map external language codes to internal language codes and default mappings for some common situations.

This has been committed and pushed to 8.x (in #365615-99: Language detection not working correctly for most Chinese readers (and add a user interface for all browser language mappings) → , #365615-103: Language detection not working correctly for most Chinese readers (and add a user interface for all browser language mappings) → and #365615-231: Language detection not working correctly for most Chinese readers (and add a user interface for all browser language mappings) → ) and is being backported to 7.x.

Steps to reproduce

From Firefox, go to the following Menu progression:
View > Character Encoding > Auto-Detect > Simplified Chinese
Make sure "Simplified Chinese" has a check next to it.
There are multiple Simplified Chinese encodings. The step above SHOULD detect them all (). To test the individual Simplified Chinese encodings, take the following menu progression:
View > Character Encoding > More Encodings > East Asian
This will lead you to a list of multiple encodings for Simplified Chinese, along with other East Asian languages. Choose the encoding you wish to test.

Once you've adjusted your language detections settings, be sure you have an up-to-date install of Drupal 7.x. See the Drupal Core Git instructions section on 'Getting ready to create or apply patches' for details on updating your installation.

Once your install is updated, log in and be sure the "Locale" module is active. Go to "DrupalSite/admin/modules" OR from the administration menu at the top of the screen, click "Modules". The Locale module is part of the Core group. Module names are listed in alphabetic order. Be sure the box next to "Locale" is checked, then scroll to the bottom of the page and click the "Save Configuration" button.

Next, go to your Drupal 7.x installation and go to "admin/config/regional/language/configure". Here, be sure the "Browser" detection method is enabled and drag it to the top of the list of detection methods. Click the "Save Settings" button at the bottom of the screen. Nothing should happen.

Next, apply the patch from comment 139.

Description of what should happen after the patch is applied.

Remaining tasks

Reroll. Review. Commit.

User interface changes

A new user interface is added to edit mappings of external vs. internal language codes.

D8 screenshot in #92
D7 screenshot in #131
Before and after screenshots of entry point in D7 in #132

Original report by yang_yi_cn

language.inc, line 95, in function language_from_browser() :

change from

  foreach ($browser_langs as $langcode => $q) {
    if (isset($languages['1'][$langcode])) {
      return $languages['1'][$langcode];
    }
  }

to

  foreach ($browser_langs as $langcode => $q) {
    if (($langcode == 'zh-cn') && isset($languages['1']['zh-hans'])) {
      return $languages['1']['zh-hans'];
    }
    if (isset($languages['1'][$langcode])) {
      return $languages['1'][$langcode];
    }
  }

As you can see, for Simplified Chinese langugage, both IE and Firefox will send 'zh-cn' as the preferred language code, which is different from Drupal's 'zh-hans' code, thus the browser detection will not work for Simplified Chinese.

Maybe there's also some other languages having this issue, but this is what I got now.