Require `langcode: โ€ฆ` only for simple config that contains translatable values

Created on 13 March 2024, 8 months ago
Updated 3 September 2024, 3 months ago

Problem/Motivation

Context: This was discovered in ๐Ÿ“Œ Mark layout_builder.settings fully validatable Needs review , because langcode: en had to be added to the new layout_builder.settings config for it to pass validation for type: config_object. @alexpott said it should NOT need langcode. And then this happened:

Quoting @alexpott from Drupal Slack:

FWIW this is going to lead to something great
Maybeโ€ฆ
Because the real rule around langcode is it is required when the config has translatable data in it.
And maybe we can write a constraint for that

๐Ÿ’ฏ

Steps to reproduce

  1. Remove the langcode: en from any fully validatable default simple config without translatable strings
  2. Re-save that simple config
  3. Expected: success, actual: validation error

Proposed resolution

  • โœ… New LangcodeRequiredIfTranslatableValues validation constraint.
  • Discuss whether to consider superfluous langcode key-value pairs invalid.
  • โœ… Update existing default config in core
  • โœ… Refine LocaleConfigManager::updateDefaultConfigLangcodes() to no longer blindly update ALL shipped config to have langcode: <default langcode>. See #5.
  • โœ… Update path + test to add/remove langcode: โ€ฆ to existing config objects. See #6.

Remaining tasks

TBD

User interface changes

None.

API changes

None.

Data model changes

None.

Release notes snippet

N/a

๐Ÿ› Bug report
Status

Active

Version

10.3 โœจ

Component
Configurationย  โ†’

Last updated about 3 hours ago

Created by

๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @wim leers
  • Pipeline finished with Failed
    8 months ago
    Total: 546s
    #118135
  • Pipeline finished with Failed
    8 months ago
    Total: 519s
    #118142
  • Pipeline finished with Failed
    8 months ago
    Total: 493s
    #118154
  • Pipeline finished with Failed
    8 months ago
    #118165
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    Wow, it took me ~1 hour to figure out why/how the hell mergedConfigExport was pre-set to some value โ€” turns out I am to blame, due to #2949021: Deprecate schema fallback in ConfigEntityType::getPropertiesToExport โ†’ ~6 years ago ๐Ÿ˜… Untangling that is out of scope here, so just updating that existing test fixture.

  • Pipeline finished with Failed
    8 months ago
    Total: 586s
    #118244
  • ๐Ÿ‡จ๐Ÿ‡ญSwitzerland berdir Switzerland

    What about things like third party settings which may contain translatable strings even if the original config entity definition does not?

  • Pipeline finished with Failed
    8 months ago
    Total: 468s
    #118253
  • Pipeline finished with Failed
    8 months ago
    Total: 495s
    #118266
  • Pipeline finished with Failed
    8 months ago
    Total: 481s
    #118281
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    Finally found why I was under the impression that all config automatically got a langcode upon saving: \Drupal\locale\LocaleConfigManager::updateDefaultConfigLangcodes() does that!

    And locale_config_batch_set_config_langcodes() means that all default configuration gets updated to have a langcode.

    Interestingly, \Drupal\locale\LocaleConfigManager::getTranslatableData() already has logic similar to what LangcodeRequiredIfTranslatableValuesConstraintValidator is adding!

  • Issue was unassigned.
  • Status changed to Needs work 8 months ago
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ
    1. CR created: https://www.drupal.org/node/3427629 โ†’
    2. Per #5, we now need an update path for all config on a site. All config on a site will have to be checked against its config schema (if it has it) to ensure that langcode is present if and only if it it contains translatable data. For config without a config schema, we should accept either. The update path test should start from a fixture containing one config object that has translatable values but is missing langcode, and one config object that has langcode but has no translatable values. That test should then fail.

    The MR is now 99% green. Unassigning to let somebody else push this to the finish line.

  • Pipeline finished with Failed
    8 months ago
    Total: 608s
    #118299
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    #4: very interesting question! ๐Ÿค” I think we can do something similar to what we have for \Drupal\locale\LocaleConfigManager::updateDefaultConfigLangcodes() but then performed by the system module whenever modules are installed or uninstalled, to automatically add/remove langcode as needed.

  • Pipeline finished with Failed
    8 months ago
    Total: 204s
    #118321
  • ๐Ÿ‡จ๐Ÿ‡ญSwitzerland berdir Switzerland

    #7: Follow-up question then: Is that complexity really worth not needing a langcode key? ;)

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    @Berdir: I actually agree with you ๐Ÿ˜‡ IMHO just adding langcode: โ€ฆ to all config is totally fine and reasonable.

    But @alexpott disagreed over at #3426429-11: [PP-1] Mark layout_builder.settings fully validatable โ†’ ๐Ÿ™ˆ

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States smustgrave

    @Wim Leers can you resolve the phpcs issue, very curious to see the tests run.

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    @smustgrave See #6, I can't. But 99.9% of tests are green already, see the last full CI run ๐Ÿ˜„

  • Pipeline finished with Failed
    8 months ago
    Total: 184s
    #118968
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom alexpott ๐Ÿ‡ช๐Ÿ‡บ๐ŸŒ

    RE #4 - third party settings are not for simple config and config entities always have a langcode.

    I think this issue is valuable because adding a langcode to simple configuration when there's nothing translatable in it is confusing. What does it mean, what happens if I change it. I've already seen a site where people used language overrides to override things that are not translatable. The problem with doing this is the moment you save that config via the UI your overrides will disappear. (Now we're getting on to the subject of validating language config translation overrides - definitely one for the future!)

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    D'oh, right, I wish I'd thought of that, @alexpott! ๐Ÿ˜„ Clarified issue title.

    Now we're getting on to the subject of validating language config translation overrides - definitely one for the future!

    ๐Ÿค“๐Ÿค“๐Ÿค“ Is there an issue for this already? ๐Ÿ˜„

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    I don't have a ton of complaints here!

    But we still need to get everything green, as well as that update path, per #6.

  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    omkar.podey โ†’ made their first commit to this issueโ€™s fork.

  • Pipeline finished with Failed
    8 months ago
    Total: 174s
    #137358
  • Pipeline finished with Failed
    8 months ago
    Total: 204s
    #137439
  • Pipeline finished with Failed
    8 months ago
    Total: 190s
    #137531
  • Pipeline finished with Failed
    8 months ago
    Total: 572s
    #137535
  • Pipeline finished with Success
    8 months ago
    Total: 630s
    #137567
  • Pipeline finished with Canceled
    8 months ago
    Total: 96s
    #137590
  • Pipeline finished with Canceled
    8 months ago
    Total: 214s
    #137592
  • Pipeline finished with Canceled
    8 months ago
    Total: 243s
    #137602
  • Pipeline finished with Canceled
    8 months ago
    Total: 57s
    #137609
  • Pipeline finished with Success
    8 months ago
    Total: 1297s
    #137612
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    Alright - I've written an update path and a test for it.

  • Pipeline finished with Failed
    8 months ago
    Total: 178s
    #137719
  • Pipeline finished with Failed
    8 months ago
    Total: 1169s
    #137727
  • Pipeline finished with Canceled
    8 months ago
    Total: 615s
    #137752
  • Pipeline finished with Success
    8 months ago
    Total: 1140s
    #137761
  • Status changed to Needs review 8 months ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States smustgrave

    So locally I added a langcode to a random settings config file that didn't have any translation.
    Applied the MR and ran the update hook
    Hook ran fine and exported config and see the langcode has been removed.

    Question on the MR since we are going to 11.x should we not do the trigger_error first instead add the violation.

  • Status changed to Needs work 8 months ago
  • The Needs Review Queue Bot โ†’ tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide โ†’ to find step-by-step guides for working with issues.

  • Pipeline finished with Canceled
    8 months ago
    Total: 565s
    #141930
  • Pipeline finished with Success
    8 months ago
    Total: 962s
    #141939
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    I wrote the logic. @phenaproxima was happy with that.

    I knew an update path (+ tests) was necessary, but I don't have the time to do that right now. @phenaproxima did ๐Ÿฅณ

    Then @phenaproxima wrote the update path + tests. I just reviewed it. It looks GREAT! I found only a few things:

    1. One comment nit: https://git.drupalcode.org/project/drupal/-/merge_requests/7018#note_295315
    2. I think rather than deleting the body of a post-update function we should use hook_removed_post_updates(): https://git.drupalcode.org/project/drupal/-/merge_requests/7018#note_295312
    3. We still need the @todo @phenaproxima asked for at https://git.drupalcode.org/project/drupal/-/merge_requests/7018#note_281429
    4. P.S.: I also responded to @smustgrave's and @phenaproxima's questions about historical config system oddities ๐Ÿ‘๐Ÿค“

  • Status changed to Needs review 7 months ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    Welp, removing the defunct (unshipped) post-update from dblog makes tests pass, so I think that's about it. All I gotta do is file a follow-up.

  • Pipeline finished with Success
    7 months ago
    Total: 1107s
    #143957
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    Added ๐Ÿ“Œ Simple config with a `langcode` element, but no translatable elements, should raise a validation error Active to convert the "superfluous langcode" deprecation error into a validation error, and linked it in the code.

  • Pipeline finished with Failed
    7 months ago
    Total: 229s
    #143989
  • Pipeline finished with Running
    7 months ago
    #144000
  • Status changed to RTBC 7 months ago
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    Dear committer, see #20 for why it's okay for me to RTBC this โ€” @phenaproxima reviewed and RTBC'd the logic I wrote, I reviewed and RTBC'd the update path+tests he wrote.

  • Status changed to Needs work 7 months ago
  • ๐Ÿ‡ฆ๐Ÿ‡บAustralia larowlan ๐Ÿ‡ฆ๐Ÿ‡บ๐Ÿ.au GMT+10

    Left some comments on the MR

  • Pipeline finished with Canceled
    7 months ago
    #144359
  • Pipeline finished with Success
    7 months ago
    Total: 993s
    #144370
  • Pipeline finished with Success
    7 months ago
    Total: 1109s
    #144487
  • Assigned to wim leers
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    This is now blocking several config validation issues, so tagging accordingly.

    I've addressed the feedback in the update path (which is the only part of this patch which I wrote); leaving the rest for Wim to do, so I can re-RTBC those parts.

  • Pipeline finished with Success
    7 months ago
    Total: 988s
    #144846
  • Pipeline finished with Failed
    7 months ago
    Total: 989s
    #144940
  • Assigned to phenaproxima
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    I agree with all changes here so far.

    I responded to all MR threads.

    2 MR threads still left:

    1. BC dance: https://git.drupalcode.org/project/drupal/-/merge_requests/7018#note_296154
    2. one doubt about the batch update @phenaproxima added: https://git.drupalcode.org/project/drupal/-/merge_requests/7018#note_296160

    IMHO after those 2 trivial things are fixed, @phenaproxima should re-RTBC. ๐Ÿ˜Š

  • Pipeline finished with Success
    7 months ago
    Total: 987s
    #144983
  • Status changed to RTBC 7 months ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    Okay, I think everything is addressed now. Restoring RTBC as per #26.

  • Pipeline finished with Success
    7 months ago
    Total: 959s
    #145182
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom alexpott ๐Ÿ‡ช๐Ÿ‡บ๐ŸŒ

    Adding issue credit.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom alexpott ๐Ÿ‡ช๐Ÿ‡บ๐ŸŒ

    Committed 83a53ba and pushed to 11.x. Thanks!
    Committed 2c3cb5e and pushed to 10.3.x. Thanks!

    • alexpott โ†’ committed 2c3cb5e8 on 10.3.x
      Issue #3427564 by phenaproxima, Wim Leers, alexpott, omkar.podey, Berdir...
  • Status changed to Fixed 7 months ago
    • alexpott โ†’ committed 83a53ba4 on 11.x
      Issue #3427564 by phenaproxima, Wim Leers, alexpott, omkar.podey, Berdir...
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom alexpott ๐Ÿ‡ช๐Ÿ‡บ๐ŸŒ

    I don't think we need release notes for this because the validation system will tell you what to do and the changes are not mandatory for contrib / custom yet.

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    Wonderful to see this in ๐Ÿฅณ Updated blocked issues.

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    ::validate() should be return typehinted. Opened ๐Ÿ“Œ Fix return typehinting for LangcodeRequiredIfTranslatableValuesConstraintValidator::validate() Active .

  • Issue was unassigned.
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    @mondrake Whoops โ€” sorry. That seems a trivial follow-up fortunately :) Reviewed it.

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • Status changed to Active 5 months ago
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

    This is breaking the ckeditor5 UI in 10.3. I think we should consider entirely rolling back and trying again. See ๐Ÿ› Replace LogicException with trigger_error in LangcodeRequiredIfTranslatableValues constraint RTBC for discussion.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States joshuautley

    Hello all, I've read through this thread and I'm looking for some guidance with an update issue from 10.2.7 > 10.3.1 in which I'm receiving the following (possibly) related error... in which I'm seeking a direction as Google search results have not been too helpful. TR pointed me to this thread (Ref: https://www.drupal.org/project/typed_data/issues/3464120#comment-15701671 ๐Ÿ’ฌ Error: Class name must be a valid object or a string Postponed: needs info ). Thank you in advance for any assistance any of you may be able to contribute.

    An AJAX HTTP error occurred.
    HTTP Result Code: 200
    Debugging information follows.
    Path: /update.php/start?id=383&op=do_nojs&op=do
    StatusText: parsererror
    ResponseText: Error: Class name must be a valid object or a string in Drupal\Core\TypedData\TypedDataManager->createInstance() (line 99 of /chroot/home/oceanbe1/oceanbeachsandiego-rv1.com/html/core/lib/Drupal/Core/TypedData/TypedDataManager.php).

    -----------------------

    Drupal database update

    Error message
    Warning: Undefined array key 1 in Drupal\system\Controller\DbUpdateController->results() (line 432 of core/modules/system/src/Controller/DbUpdateController.php).

    The update process was aborted prematurely while running update # in system_post_update_add_langcode_to_all_translatable_config.module. All errors have been logged. You may need to check the watchdog database table manually.

    The following updates returned messages:
    system module
    Update add_langcode_to_all_translatable_config

    Processed 550 items of 876.

    -----------------------

    Error: Class name must be a valid object or a string in Drupal\Core\TypedData\TypedDataManager->createInstance() (line 99 of /html/core/lib/Drupal/Core/TypedData/TypedDataManager.php)

    #0 /html/core/lib/Drupal/Core/TypedData/TypedDataManager.php(108): Drupal\Core\TypedData\TypedDataManager->createInstance()
    #1 /html/core/lib/Drupal/Core/Config/Schema/ArrayElement.php(155): Drupal\Core\TypedData\TypedDataManager->create()
    #2 /html/core/lib/Drupal/Core/Config/Schema/ArrayElement.php(57): Drupal\Core\Config\Schema\ArrayElement->createElement()
    #3 /html/core/lib/Drupal/Core/Config/Schema/ArrayElement.php(104): Drupal\Core\Config\Schema\ArrayElement->parse()
    #4 /html/core/lib/Drupal/Core/Config/Schema/ArrayElement.php(138): Drupal\Core\Config\Schema\ArrayElement->getElements()
    #5 /html/core/lib/Drupal/Core/Config/Schema/ArrayElement.php(24): Drupal\Core\Config\Schema\ArrayElement->getIterator()
    #6 /html/core/modules/system/system.post_update.php(270): Drupal\Core\Config\Schema\ArrayElement->hasTranslatableElements()
    #7 /html/core/includes/update.inc(236): system_post_update_add_langcode_to_all_translatable_config()
    #8 /html/core/includes/batch.inc(296): update_invoke_post_update()
    #9 /html/core/includes/batch.inc(138): _batch_process()
    #10 /html/core/includes/batch.inc(94): _batch_do()
    #11 /html/core/modules/system/src/Controller/DbUpdateController.php(195): _batch_page()
    #12 [internal function]: Drupal\system\Controller\DbUpdateController->handle()
    #13 /html/core/lib/Drupal/Core/Update/UpdateKernel.php(115): call_user_func_array()
    #14 /html/core/lib/Drupal/Core/Update/UpdateKernel.php(76): Drupal\Core\Update\UpdateKernel->handleRaw()
    #15 /html/update.php(27): Drupal\Core\Update\UpdateKernel->handle()
    #16 {main}

  • ๐Ÿ‡ช๐Ÿ‡ธSpain javier_rey

    After implementing this change, when installing a fresh install in a language other than English and using the โ€œstandardโ€ installation profile, for exampleโ€, I get the following error:

    the configuration objects have different language codes so they cannot be translated
Production build 0.71.5 2024