Track 17: Multilingual Roadmap

Some random thoughts:

- Should it be possible for a site building to re-run the recipe multiple times to add multiple languages?
- Installing and configuring TMGMT could be part of an "advanced" translation recipe. The events recipe has already triggered a discussion about how Drupal CMS can present optional recipes
- I wonder if there is some way other tracks can specify what fields should be translatable. For example, in an event content type I would say that typically the "where" (date and time) and "where" (location or link) fields don't need to be translated

I agree with the translation settings:

Translation settings:

• Entities Types such as Nodes, Taxonomy Terms, Media Types, will be translatable.
• Basic fields such as, Textfields, Textareas, will be translatable
• Entity reference fields such as reference fields to taxonomy terms or media types, will not be translatable.

This is how we at 1xINTERNET build translation, too.

Maybe your recipe could work in a generic way that works by field type. Then your recipe could be applied to arbitrary content types. I guess this could work, if we agreed on a field naming convention. Maybe this could also solve your consideration regarding SEO fields. If they followed a naming convention, they would automatically be configured correctly.

For our project we always keep English as the default language, as a matter of fact we prevent changing the default language to something other than English. This has saved us a lot of time, because it prevents configuration chaos, when you introduce configuration translation.

It sounds like language detection will be based on URL, which we do as well as a default. However, we always configure with language prefix also for the default language. So for a German / English page we would have /de for the German pages and /en for the English pages and not / for the English pages in the default language.

Regarding your question about TMGMT I would suggest to leave it out of scope. We use it in many projects because it is awesome, but it is not a common thing we installe. We only install it for large clients, that have advanced translation requirements. However, having this as a separate recipe that we can use would of course be nice :-)

However, I think Interface translation and Configuration translation should be in scope. Usually, when clients request multilingual pages, they expect the interface for editors also to be translatable. Also without Configuration translation you end up having views with labels in wrong languages for site visitors. I guess when your recipe enables Interface translation and Configuration translation it does not solve the problem, that the interface and the configuration has to be translated. And who is responsible for this. Do the recipe creators for a content type that ships a view have to provide translations for their configuration? I am not sure how to best approach this.

Quoting the issue summary:

It will ask the user for input to select the language they want to use for translations.

This could be a bit of a technical bugaboo, because right now the recipe input system has no way to programmatically come up with a list of choices. You'd probably have to hard-code them all into recipe.yml, and that absolutely sucks; or you'd have to expect the user to enter a langcode.

This might, however, be a case where core could justifiably add that feature to the recipe system.

The bigger issue here is probably that you cannot generate dynamic config entity IDs. So if the user says they want to use the fr langcode, via an input, you cannot do this in config actions:

config:
  actions:
    language.entity.${langcode}:
      createIfNotExists: ...

Inputs are not allowed in keys because they make the recipe less predictable. This would make recipes act more like functions, which they aren't.

I don't mean to throw water on the parade here; but I want to make sure you can actually build what you need, and I'm not entirely certain that core yet provides the tooling that will fulfill the purpose. I'd be delighted to be wrong.

My 2 cents:

I like most of what I read here, I think it's a very good place to start from.

I do agree with breidert on the configuration and interface translations being included. I don't know how much we would need to have configured by default, but they should be looked at IMO.

As for mandclu's comment about fields like "where" and "when" fields, I believe we should not make any field type translatable unless we're very sure what it does (e.g. only text and text_area and nothing else). IMO it is better to have a field not translatable that should be in stead of the other way around (because of the note in the edit form).

I agree with ignoring content moderation at this point.

I think metatag translations should be something for the SEO recipe. The most important ones will be generated based on fields anyway, and these fields (or their entities) will be translatable (title, description, image).

One other thing that might need to be discussed is the entity translation default settings (showing language select or not, interface language by default vs default language vs a language selected on recipe install).

Agreed on interface translation (even non-EN single language sites need that and it's automatically installed when you install with non-EN), but it's basically just enabling the module. Same for config translation, that doesn't really need configuration, each config entity defines what's translatable, everything else is generic. (Minus the complexities and bugs and problems when working with config then, but recipes can't solve that)

I'm not sure why adding a specific language would need to be a recipe, configurable or not. The recipe is the ability to have multiple languages and necessary configuration for that. The specific languages can then be added through the regular UI, that seems like a non-issue to me. It's a single step, the recipe could display a message to guide the user to the next step or so? This might be a general thing that I haven't really seen yet, that we tell the user what the next step is after a certain recipe.

URL language negotiation mostly configures itself once enabled, the prefix automatically defaults to the language code, the only thing I'd recommend is that the default language is also set to a prefix, because by default it doesn't have that. And that doesn't play well with browser language negotiation and caching.

The main complexity here is IMHO content_translation configuration, that's where the bulk of complexity is. Because it basically adds an extra dimension to each content type/bundle that's set up. The issue summary talks about entity type and field type settings, but that's not how translatable config is stored, it's stored per bundle and per field. Each bundle needs:

* the opt-in to enable content translation (new config entity with a few settings like default langcode and show/hide untranslatable fields, which also relates to content_moderation, which enforces that). Not every entity type/bundle needs to be translatable, depending on your use case and scenario (e.g. comments and users can be translatable, but most projects won't need or want that, you might have node types that aren't translatable)
* each field needs to define whether or not it's translatable. this is base config, while not exposed in the UI, each recipe can be configured to set sensible defaults (e.g. body translatable and a tags field isn't). But this setting also exists for base field like changed date and title, those are then stored as field config overrides.
* Some field types like image support a mix, so the field as a whole is translatable, but the file reference is shared/synchronized and the title/alt text is translatable/different. That's a third party setting on the field config entity, depending on content_translation, so can't be prepared.
* Depending on the use case, there are different ways to filter views. Either you only show translated content, or you show all content once, with a translation if it exists. That means different filters on views that by default like won't have them. As a default, all views should probably default to showing the default translation only, that shouldn't affect single-language sites and then sites can customize from there.
* Translation permissions, although this can also be done fairly well with a global permission at this point (translate editable entities)

Constraints:
* Multilingual content translation recipe can't depend on all other content type recipes.
* This set up should IMHO work independently of recipe install order. We can't expect that sites will first enable all the content type recipes and then go multilingual. They might later on add an event recipe and that should then also be multilingual.

I can vaguely think of 3 general approaches to that:

1. Generic code/magic/assumptions. That sounds more like a new (core?) feature than a recipe. Something like a checkbox somewhere that says that all node node types (setting per entity type? or maybe inverted, something like a checkbox to customize the configuration) and their fields should make sensible default assumptions about translatability based on field type. But I expect there will be things that are off, and if it's like the mentioned customize page, once it's off, it's off.
2. An additional foo_translatable recipe for each foo recipe that depends on foo and multilingual. That will need a lot of additional recipes and IMHO would require some kind of suggestion system like "You just applied the content translation recipe, you might also want to apply Article Translation, Page translation, Tag translation, ..." and also in reverse, "You just applied the event recipe, (since content translation is enabled) you also might want to apply Event translation.
3. Optional/conditional config and config actions in a recipe, similar to optional config in a module that gets applied once all dependencies are met.

Both 2 and 3 would require a system that keeps track of applied recipes to make suggestions/apply their optional parts once the requirements are met. For 3, that would means enabling content translation would revisit a lot of previously applied recipes and set up the necessary extra config entities and third party entries. Option 1 might sound like the easiest, but it's a new thing that AFAIK doesn't exist yet and I would expect that there will always be exceptions where the generic assumptions won't work.

For our project we always keep English as the default language, as a matter of fact we prevent changing the default language to something other than English. This has saved us a lot of time, because it prevents configuration chaos, when you introduce configuration translation.

I can see how that does make sense as an agency, but that's not really something that can be enforced IMHO. Sometimes you start with a single-language project that's not English and then you need another language. The installer asks for the language when you install, I don't think we can or want to disallow that. That process requires that even for single-language sites, you need to create everything EN first and then translate it. Drupal also needs to be available for users that don't speak English or at least not good enough.

Ah, and as maintainer of TMGMT, definitely +1 on not including it for now. Happy if people use and like it, but as mentioned, it's targeted towards more complex sites and needs integration with a translation service providers to be really useful.

The basic page recipe is now updated, so I think we can get started on the recipe?

Does this track include also the translation of the installer? Currently if you switch the language in the installer, only existing translated Drupal strings are actually shown in the desired language. Custom strings from Drupal CMS are not translated. Or is there an issue for this? If not, I can create one. Thanks!

Does this track include also the translation of the installer?

No, it doesn't. Feel free to create the issue but please mark it 'Postponed' as the installer text is not finalised. We will aim to have it finalised by RC.

For the reasons already outlined in some of the comments here, we have decided to postpone this feature until it can be scoped properly as a recipe that delivers more than just the basics. Although we could provide this, and we even have a version of it from the prototype, we don't want to set an expectation with users with a recipe that only does a few relatively simple things.

Considering this space is very complex and overlaps with many other areas, it needs a lot of planning and consideration. So we will probably try to recruit a few people who can come up with a proposal, most likely that will require additional features either in recipes (or elsewhere) to provide a more comprehensive solution.

If anyone who has commented here is interested in that, please let me know :)

Atm, not having translation support in Drupal CMS is quite unfortunate - it's a big, important feature that seems "unsupported" for evaluators.

I'm not sure why adding a specific language would need to be a recipe, configurable or not. The recipe is the ability to have multiple languages and necessary configuration for that. The specific languages can then be added through the regular UI, that seems like a non-issue to me.

I very much agree with that. If we pair this with the sensible defaults as described by breidert:

Translation settings:

Entities Types such as Nodes, Taxonomy Terms, Media Types, will be translatable.
Basic fields such as, Textfields, Textareas, will be translatable
Entity reference fields such as reference fields to taxonomy terms or media types, will not be translatable.

we could quite easily make recipe that turns on translation for content+config+interface. Yes, it's not perfect, but it's starting point + it shows off that Drupal can do it. I think that would be much better than having it "seem unsupported".

First point: The installer. This belongs in a separate issue. I know that this would also go beyond the scope. If there is already an issue for this, please link to it on the right-hand side.

Nevertheless, I would like to briefly note here that the installer is the first point of contact with Drupal for many users. If we continue to allow the display language to be mixed between English and other languages (e.g. when selecting a time zone [country/city/state]), then users are likely to continue to wonder why, on the one hand, we advertise with first-class multilingual support but, on the other hand, show during installation that this does not always work. Everything else belongs in a separate issue.

Second point: Configuration translations (primarily views but also menu labels and descriptions).

As far as views and basically all other configuration translations are concerned, I understand that we are currently displaying the English originals here, even if a website is installed in a language other than English, but as an end user, I simply do not accept this.

If I put myself in the shoes of an international agency, I would say something like the following:
“We want labels of views, titles, filter options and the like to continue to be available in English even on sites installed in Spanish, for example, because our multinational employees must be able to make configurations on a site whose default language is Spanish. Therefore, it would be good if we could first import English-language values here, which would then have to be translated manually.”
I have read this elsewhere on Drupal.org, unfortunately, I can no longer find the reference issue.

From an agency's point of view, this is certainly true. However, as an end user, I will contradict this statement and say something like the following:

1. I have installed Drupal in my native language so that I understand what I am triggering when I activate or deactivate an option in Drupal.
2. I activate a module or recipe for Drupal that adds a new view, a new menu or a new dashboard and notice that I, as an administrator, or even at the frontend (for example, when a view does not yet contain any content or I activate a filter within the view), am suddenly shown English texts.

a. I don't understand this.
i. For one thing, I didn't install the site in English.
ii. Furthermore, there is a translation for the Drupal user interface in my language
iii. Furthermore, the translation of the relevant character string is available there.
iv. Nevertheless, the view provided by the newly added recipe/module is still partly in English.

1. My conclusion is: Drupal's multilingual support is not as good as it is made out to be.
2. So I'll just sit down and translate such standard messages and button texts myself, so that Drupal at least looks like it supports multilingualism perfectly on my website.
3. Now I have to spend time on something that the responsible translators have already done. What's the point? Why is that so?
Example: This page does not contain any content, yet. (Text if there is no Content in a View.
Add a Blog Post
Filter --> Button Label (Apply) ect. (Filter options in general, Buttons for exposed Filters, ect.)

This is exactly what many end users experience when they first start using Drupal.
This applies to view components, menu names, and much more in Drupal.

Guys, I understand both worlds. Really!

But we have to do something about it. From the view point of an end unser it is unacceptable for Drupal to display English strings in the user interface when it is displayed in a different display language.

Especially not if translations are already available for a particular part of Drual on localize.drupal.org. We as a community are no longer being properly informed about the fact that there could be a plausible reason for this. This is due to the expectations of many end users, who in the age of AI simply assume that it should no longer be the case that they encounter situations within a software in which there is a mix of different languages in the display language of a user interface, if this has not been explicitly specified by the end user.

Similarly, this can be applied, for example, to menu names and link labels within menus.

If a menu is named Welcome, as in the case of the Drupal CMS Dashboard recipe, or in the case of Drupal Corr Main Navigation, then as an end user I expect the name of the respective menu and its description to be in the default display language of the site that was selected when the website was installed.

Unfortunately, Drupal still doesn't do this well, even in version 11. We need to be careful here and write tests to help developers identify such problems early on.

Third and last point: We urgently need guidelines for developers on how to write translatable original strings, as well as technical control routines.

If you take a look at the translation server, you'll find a number of original strings that look something like this:

<div class=“messages messages--info”>Your CivicTheme (for Starshot) version: @version</div>

The point is that we do not get such translations translated because of the API function
https://api.drupal.org/api/drupal/core%21modules%21locale%21locale.modul...
not translated.

These are rejected when importing translations into Druapal because of the above-mentioned function that simply does not allow div tags. This means that Drupal is not 100% translatable by default, for the sake of security.

Developers who are not aware of this may inadvertently cause such translations to end up on the server. If we want to keep Drupal future-proof, then we need a solution that checks submitted original strings against the tags contained in the function locale_string_is_safe at the latest in the publication process of a module or theme and rigorously advises developers to modify the affected strings so that they are translatable before the POT file for a module can be deployed to the translation server.

This is half the battle with this problem. The other half is to find such strings in modules that have already been published and to point out to the maintainers which of your modules or themes contains original strings that could be improved.

At least since Drupal 7, these errors have become commonplace and as much as I like translating Drupal to make it more accessible to newcomers and interested parties, it is becoming increasingly difficult to keep track of which project maintainer has been informed that there is an incorrectly written original string in their module and whether or not the problem has been fixed. I can only speak for myself, but I can't do it in my free time. There are simply too many of the same kind of issues to manually track them.

I can imagine that especially the last point, which tends to fall into the areas of infrastructure and documentation teams, does not necessarily fit into this track. But I don't know where or with whom I should talk about it if I don't bring it up here. This is a big problem and one that we have been putting off since Drupal 7 and have not been able to get under control at any level. That's why I don't want to leave it unmentioned here.