Wrong language field labels after `drush cr` because of Drush language negotiation

FTR: I am quite sure that there are multiple reasons that this can happen.

IIRC this issue hass settled on drush cr and language negotiation.

I found another one of them in the related issue.

We just experienced this and these are our steps to reproduce it:
1. Have a multilingual site with EN and DE.
2. Set EN as the default language.
3. Configure interface and content language detection to be based on URL.
4. Configure that for DE the path prefix is / and for EN it is /en.
5. Place breakpoints in
5.1. \Drupal\Core\Entity\EntityFieldManager::getFieldStorageDefinitions()
5.2. \Drupal\language\Config\LanguageConfigFactoryOverride::loadOverrides()
6. Execute drush cr with xdebug enabled
7. Observe how the entity field manager now uses "de" for building the cache ID since without passing url to drush the url negotiator returns "de". 8. Observe how but when it goes to load the overrides the language config factory override uses "en" to load the overrides since its constructor takes the site's default language.

Clearing the caches over the UI no matter if on the EN or DE path prefixes does not seem to be causing issues. It breaks only when drush is involved into the cache rebuild.

I also noticed that LanguageRequestSubscriber does not get triggered at all as no KernelEvents::REQUEST event gets dispatched during drush cr.

In contradiction to #32 🐛 Wrong language field labels after `drush cr` because of Drush language negotiation Needs work for me the MR solution worked but the PHP_SAPI check did not work.

Inspired by both approaches I am providing a mixed alternative that hopefully should be then covering all the cases for everyone and not only the ones we are observing in the entity field manager.

The last submitted patch, 35: 3221375-35.patch, failed testing. View results →

Thanks for #35 details, it looks exactly the same reason as discussed in 🐛 Installing a module causes translations to be overwritten Fixed

Patch failed only temporarily

This was tagged for tests which appear to still be needed. Question with 🐛 Installing a module causes translations to be overwritten Fixed being committed did it fix the issue here at all?

Don't think it's related but there is also 🐛 Creating a new translation may delete translations with drafts Fixed may be worth mentioning.

Reading through this issue, just some thoughts for now...

1. It looks like we have two different issues here:

* Drush not triggering / respecting language negotiation / not settling on a language early enough
* Bad caching, not respecting language either or mixing config language and caching language.

2. Which language should Drush select? Not clear for me... is there a request / a URL / a language prefix ... ? Since this may quickly get really confusing I think we should aim at settling on a sensible default, also something that won't mess with deployments on different environments... which is I think the main use of drush... This should be IMO the Site Default language, and that's it. Easy and clear.

3. Whatever the final patch is, triggering language negotiation mid-request doesn't look like a good idea to me... So either we set it during initialization or we don't.... Again, 2) may be a sensible default.

4. So... I see two ways to fix this issue: A) Proper language / initialization with Drush to (whatever makes sense) or B) Fix caching, aka.. make config language consistent with caching language (whatever it is) or maybe skip persistent cache.

I've done some more tests with this one and Drupal 10.1.x...

As @nanak #22 mentions, behavior is changed after cache rebuilding was fixed. But we can reproduce it with the setup in the issue description and forcing the field rebuilding with drush, try:

drush cr && drush php-eval "\Drupal::service('entity_field.manager')->getFieldDefinitions('node', 'article');"

Some more debugging shows also the already mentioned mismatch #35 in getFieldDefinitions(), when running with drush, while current language is the one set as custom language (de), the config override language is the default one (en). So we can see why the patch here, which fixes that mismatch works https://git.drupalcode.org/project/drupal/-/merge_requests/1361

However, I don't think that is the best place to switch configOverrideLanguage... or maybe just if we switch it on a method call, we better switch it back before returning, but not sure about this...

What I think we need to do is fixing the internal consistency of getFieldDefinitions() using current language bug calling other methods that get configuration that may not be using 'current language', but 'config override language' instead...

And as a general rule, if we want some consistent API, there shouldn't be any function that does language dependent caching without ensuring anyting it calls is using the same language...

Thinking a bit more...

* There's no easy way to use language consistently through the API, different languages (current, configuration) are retrieved from the container everywhere... that's dependency injection I guess ...
* Inconsistencies happen at other levels too. String translation is not properly initialized with drush either.

// This one uses default language
drush php-eval "print t('User name');"

// This one triggers language negotiation and uses defined language.
drush php-eval "print t('User name', [], ['langcode'=> \Drupal::languageManager()->getCurrentLanguage()->getId()]);"

So yes, just aiming for consistency we need to fix the language initialization, not only config override language, bug locale language too.

And this is where everything happens, but not when running drush:

// class Drupal\language\EventSubscriber\LanguageRequestSubscriber
  private function setLanguageOverrides() {
    $this->negotiator->setCurrentUser($this->currentUser);
    if ($this->languageManager instanceof ConfigurableLanguageManagerInterface) {
      $this->languageManager->setNegotiator($this->negotiator);
      $this->languageManager->setConfigOverrideLanguage($this->languageManager->getCurrentLanguage());
    }
    // After the language manager has initialized, set the default langcode for
    // the string translations.
    $langcode = $this->languageManager->getCurrentLanguage()->getId();
    $this->translation->setDefaultLangcode($langcode);
  }

This patch is a simple and rough version of #40 / 4B: Just use default language for drush - and CLI - *always*

It will fix the issue for most people, while also keeping consistent string translations. Just keep in mind the language when running drush will be *consistent* but maybe not the one you like.... as it's always the site default language.

(Not intended for committing, of course, just setting to needs review to trigger testing and see what happens)

Experiencing the issue as well. Basically breaks every deployment, because of `drush cr`. In our case also the "Follow the user's language preference." language detection mode is affected.

Patch #35 has been solving it.

We are experiencing this also during deployments. At first it seemed like the patch from issue #35 works and after applying the patch, drush cr seemed to set the language correctly. But it still seems there is a difference when running cache clear with drush and through the UI.

For example today we noticed that some taxonomy term values were in English language when viewing the site in Finnish. After clearing the caches in the UI it changed the language correctly to Finnish.

At the moment we have a deployment process which uses drush deploy during the deployment.

It seems like this has something to do with the language negotation because we started experiencing the issues after we changed some of the language negotiation settings and changed the Selected language to fallback to Finnish. Our site's default language is English. We have enabled the URL language detection from the language prefix and also the Selected language negotiation is set to fallback to Finnish. Other language negotiations are disabled.

Here is our current language.negotiation.yml with the site experiencing the issue.

_core:
  default_config_hash: uEePITI9tV6WqzmsTb7MfPCi5yPWXSxAN1xeLcYFQbM
session:
  parameter: language
url:
  source: path_prefix
  prefixes:
    en: en
    fi: fi
    sme: sme
    smn: smn
    sms: sms
    '': null
  domains:
    en: ''
    fi: ''
    sme: ''
    smn: ''
    sms: ''
selected_langcode: fi

Here is the language.types.yml example:

_core:
  default_config_hash: dqouFqVseNJNvEjsoYKxbinFOITuCxYhi4y2OTNQP_8
all:
  - language_interface
  - language_content
  - language_url
configurable:
  - language_interface
negotiation:
  language_content:
    enabled:
      language-interface: 0
    method_weights:
      language-content-entity: -9
      language-url: -8
      language-session: -6
      language-user: -4
      language-browser: -2
      language-interface: 9
      language-selected: 12
  language_url:
    enabled:
      language-url: 0
      language-url-fallback: 1
  language_interface:
    enabled:
      language-url: -20
      language-selected: -15
    method_weights:
      language-user-admin: -16
      language-url: -20
      language-session: -19
      language-user: -18
      language-browser: -17
      language-selected: -15

We also have another site with a similar problem but there we are trying to also follow the user's profile to set the site's language so that might play some part in the problem because language-user-admin is enabled.

Here is the language.negotiation.yml from that site:

_core:
  default_config_hash: J6QBFdEYFecHpT05_8KmHuTn9k9ELLdiLJO-fQS1nDk
session:
  parameter: language
url:
  source: path_prefix
  prefixes:
    en: en
    fi: fi
    '': null
  domains:
    en: ''
    fi: ''
selected_langcode: fi

And also the language.types.yml:

_core:
  default_config_hash: dqouFqVseNJNvEjsoYKxbinFOITuCxYhi4y2OTNQP_8
all:
  - language_interface
  - language_content
  - language_url
configurable:
  - language_interface
  - language_content
negotiation:
  language_content:
    enabled:
      language-url: -19
      language-selected: -14
    method_weights:
      language-content-entity: -20
      language-url: -19
      language-user: -18
      language-session: -17
      language-browser: -16
      language-interface: -15
      language-selected: -14
  language_url:
    enabled:
      language-url: 0
      language-url-fallback: 1
  language_interface:
    enabled:
      language-user-admin: -19
      language-url: -18
      language-selected: -15
    method_weights:
      language-user: -20
      language-user-admin: -19
      language-url: -18
      language-session: -17
      language-browser: -16
      language-selected: -15

One detail to add is that we are currently using both the patch #35 from this issue and the patch #9 from issue #3114824.

At least with the patch from this issue seemed to give me some results. In my local environment when I was clearing caches with drush without the patch from this issue, it seemed like the language for labels was in wrong language. With the patch from this issue it seemed to work correctly when I cleared caches with drush and then immediately viewed the page in the correct language.

But in our production environment there are still cases where we get client error reports that the labels are in wrong language. Not 100% understanding all the mechanism around this issue but my guess would be that it might be also related when the actual node gets cached for the first time. So in my test cases it was first viewed with the correct language prefix so it cached correctly. But if someone for example with a wrong administration interface language views the same node for the first time, could this result in some cache pollution situation that it doesn't anymore use the correct language for from the prefix?

Currently it seems like our language negotiation configs should prefer language-url based on the above configurations but I wouldn't be surprised if there was some weirdness with language negotiation, fallback language and default language given how complex the language negotiation logic can be.

Very probably this 2 related issues will solve it

We discussed this issue in Slack and I tested the latest patch from 🐛 Configuration being imported by the ConfigImporter sometimes has stale original data Fixed against 10.2.6 to solve the issue. At least in our staging environment which was suffering from this issue, the patch seems to now solve it. Our symptom was that taxonomy term and select list values in frontend were showing in wrong language after Drush deployment. I was also able to remove the patch from 🐛 Add multilingual support for caching basefield definitions Needs work without it having a visible regression.

Unfortunately my comment from #51 was not the final verdict. We have still seen this issue after deployments. Mostly the problem still happens with the multisite where term names and field labels are in wrong language after deployment. Everytime a manual cache clear from the UI fixes the issue. It seemed like the patch from 🐛 Configuration being imported by the ConfigImporter sometimes has stale original data Fixed might have even made the problem more visible because with that patch we were also seeing wrong labels in the default site when usually they are limited to the multisite instance.

I'll report more findings if we are able to track down the issue further. Of course any debug or test instructions that we can run and report here would help.

I'll report more findings if we are able to track down the issue further. Of course any debug or test instructions that we can run and report here would help.

Heikki, I can see that both sites of yours have their default language as something else than English. Have you already tried changing it to English? My colleagues said that this change fixed the translation issue. Meaning:

1. default_langcode: en in system.site.yml
2. selected_langcode: site_default in language.negotiation.yml
3. Make sure that every configuration is in English

I need to make this work on one of the sites as well and will report how this workaround went.

Hi @yevko, nice to see you.

No, both sites have the default language as English.

Our system.site.yml has

_core:
  default_config_hash: yXadRE77Va-G6dxhd2kPYapAvbnSvTF6hO4oXiOEynI
langcode: en
uuid: 01602f5c-032b-4c67-a1da-b7bba1207a45
name: 'Example site'
mail: 'noreply@example.site'
slogan: ''
page:
  403: ''
  404: ''
  front: /node/1
admin_compact_mode: false
weight_select_max: 100
default_langcode: en
mail_notification: noreply@example.site

But indeed the language negotiation has been set so that the site by default opens the Finnish site and not the English one in language.negotiation.yml:

_core:
  default_config_hash: uEePITI9tV6WqzmsTb7MfPCi5yPWXSxAN1xeLcYFQbM
session:
  parameter: language
url:
  source: path_prefix
  prefixes:
    en: en
    fi: fi
  domains:
    en: ''
    fi: ''
selected_langcode: fi

My feeling is that because the site's default language is set to English sometimes Drush takes that as the default language and changes Finnish labels into English. This does not happen the other way around that English site shows Finnish labels. The problem still gets fixed by clearing all caches in the admin UI.

@heikkiy , we have noticed exactly what you described.

The patch from comment #50 fixes the problem.

Correction, we're actually using a github patch for drush.
https://patch-diff.githubusercontent.com/raw/drush-ops/drush/pull/5755.p...