Fix bugs in update path surfaced by config validation

Created on 15 November 2023, almost 2 years ago

Problem/Motivation

In πŸ“Œ Configuration schema & required keys Fixed , if I revert the newly added \Drupal\FunctionalTests\Update\UpdatePathTestBase::$configSchemaCheckerExclusions, then the tests fail, proving that this is indeed still necessary.

Failures:

  1. Drupal\Tests\views\Functional\Update\ViewsFixRevisionIdUpdateTest::testViewsPostUpdateFixRevisionId
    There should be no errors in configuration 'block.block.claro_help_search'. Errors:
    Schema key 0 failed with: [] 'uuid' is a required key.
    
    Failed asserting that Array &0 (
        0 => '[] 'uuid' is a required key.'
    ) is true.
    

    … even though that block does not exist in the update path fixture, nor is it installed prior to updates getting executed. This makes it highly likely to be caused by help_post_update_help_topics_search(), introduced in πŸ“Œ Merge Help Topics classes into Help with BC layer Fixed .

    Same thing for search.page.help_search.

  2. block.block.stark_testblock, block.block.testblock and editor.editor.test_text_format only exist in the "filled" fixture, and cause problems only for update path tests using that one. The block portion of this is possibly caused by ✨ Prefix block machine name suggestions with the theme machine name Fixed .
  3. Steps to reproduce

    Proposed resolution

    1. Revert
        protected static $configSchemaCheckerExclusions = [
          // The following config trigger config schema validation errors, even after
          // the update. An update path is needed to fix these.
          // @see drupal-9.4.0.filled.standard.php.gz
                // @todo Remove in <follow-up issue to be created>
          'block.block.claro_help_search',
          'block.block.stark_testblock',
          'block.block.testblock',
          'editor.editor.test_text_format',
          'search.page.help_search',
        ];
      
    2. Add the missing update paths

    Remaining tasks

    User interface changes

    API changes

    Data model changes

    Release notes snippet

πŸ› Bug report
Status

Active

Version

11.0 πŸ”₯

Component
HelpΒ  β†’

Last updated 4 days ago

No maintainer
Created by

πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @wim leers
  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί
  • πŸ‡«πŸ‡·France andypost

    Does it mean it needs to update dump or we can just add extra fixtures to existing dumps?

  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    I have just found this issue after updated a project and realized that help_post_update_help_topics_search() created two configs without UUIDd that also made our strict QA pipeline fail, since the exported config in config/sync differs from the active config of clean site created via drush si -y --existing-config.

    $ git diff
    diff --git a/config/sync/block.block.claro_help_search.yml b/config/sync/block.block.claro_help_search.yml
    index aca7524..6695d6d 100644
    --- a/config/sync/block.block.claro_help_search.yml
    +++ b/config/sync/block.block.claro_help_search.yml
    @@ -1,3 +1,4 @@
    +uuid: 17c15c63-085d-4ac2-aad3-a19746c37ccd
     langcode: en
     status: true
     dependencies:
    diff --git a/config/sync/search.page.help_search.yml b/config/sync/search.page.help_search.yml
    index 38d4344..b9b39a9 100644
    --- a/config/sync/search.page.help_search.yml
    +++ b/config/sync/search.page.help_search.yml
    @@ -1,3 +1,4 @@
    +uuid: 6f35532e-b3ab-4004-bfb2-db2b4482f004
     langcode: en
     status: true
     dependencies:
    
    

    This also means that a UUID gets added to these config after a clean site install and can be exported to config/sync, but it would be beneficial if a follow up update hook would be added to fix the could-be-missing UUID problem on existing projects.

  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    UUID generation was missing from there, indeed.
    Now that the update hook is out, it can be fixed, but already updated sites will need a workaround for fixing the missing UUID.

    In our case the CI prevented merging the change without a UUID, but others probably weren't that lucky.

  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    Btw, should not config storage generate a UUID automatically when a new config is saved via standard API calls? Is there an issue for that already?

  • First commit to issue fork.
  • Merge request !11869Fix the fixture. β†’ (Open) created by longwave
  • πŸ‡¬πŸ‡§United Kingdom longwave UK

    Rediscovered this over in πŸ› Display title setting has wrong schema type Needs work where an update hook across all blocks fails because claro_help_search does not have a UUID in the fixture database.

    The MR here fixes that, the uncompressed diff is:

    --- drupal-10.3.0.filled.standard.php	2025-04-17 15:11:52.088063088 +0200
    +++ core/modules/system/tests/fixtures/update/drupal-10.3.0.filled.standard.php	2025-04-17 15:10:53.921921024 +0200
    @@ -2101,7 +2101,7 @@
     ->values(array(
       'collection' => '',
       'name' => 'block.block.claro_help_search',
    -  'data' => 'a:11:{s:8:"langcode";s:2:"en";s:6:"status";b:1;s:12:"dependencies";a:3:{s:6:"module";a:2:{i:0;s:6:"search";i:1;s:6:"system";}s:5:"theme";a:1:{i:0;s:5:"claro";}s:8:"enforced";a:1:{s:6:"config";a:1:{i:0;s:23:"search.page.help_search";}}}s:2:"id";s:17:"claro_help_search";s:5:"theme";s:5:"claro";s:6:"region";s:4:"help";s:6:"weight";i:-4;s:8:"provider";N;s:6:"plugin";s:17:"search_form_block";s:8:"settings";a:5:{s:2:"id";s:17:"search_form_block";s:5:"label";s:11:"Search help";s:13:"label_display";s:7:"visible";s:8:"provider";s:6:"search";s:7:"page_id";s:11:"help_search";}s:10:"visibility";a:1:{s:12:"request_path";a:4:{s:2:"id";s:12:"request_path";s:6:"negate";b:0;s:15:"context_mapping";a:0:{}s:5:"pages";s:11:"/admin/help";}}}',
    +  'data' => 'a:11:{s:4:"uuid";s:36:"356ded34-80b4-451a-bdc4-df8d8c0931e1";s:8:"langcode";s:2:"en";s:6:"status";b:1;s:12:"dependencies";a:3:{s:6:"module";a:2:{i:0;s:6:"search";i:1;s:6:"system";}s:5:"theme";a:1:{i:0;s:5:"claro";}s:8:"enforced";a:1:{s:6:"config";a:1:{i:0;s:23:"search.page.help_search";}}}s:2:"id";s:17:"claro_help_search";s:5:"theme";s:5:"claro";s:6:"region";s:4:"help";s:6:"weight";i:-4;s:8:"provider";N;s:6:"plugin";s:17:"search_form_block";s:8:"settings";a:5:{s:2:"id";s:17:"search_form_block";s:5:"label";s:11:"Search help";s:13:"label_display";s:7:"visible";s:8:"provider";s:6:"search";s:7:"page_id";s:11:"help_search";}s:10:"visibility";a:1:{s:12:"request_path";a:4:{s:2:"id";s:12:"request_path";s:6:"negate";b:0;s:15:"context_mapping";a:0:{}s:5:"pages";s:11:"/admin/help";}}}',
     ))
     ->values(array(
       'collection' => '',
    
  • πŸ‡¬πŸ‡§United Kingdom longwave UK

    Hm actually this isn't quite right, I edited the fixture by hand but I think it should be exported properly, this is what I get from trying to rush this while waiting at the airport gate to fly home from Dev Days.

  • Pipeline finished with Success
    4 months ago
    Total: 368s
    #475932
  • Status changed to Needs work 5 days ago
  • πŸ‡ͺπŸ‡ΈSpain penyaskito Seville πŸ’ƒ, Spain πŸ‡ͺπŸ‡Έ, UTC+2 πŸ‡ͺπŸ‡Ί

    Quite confused by this.

    1. Using 11.x HEAD (9678f6309c8)
    2. Revert \Drupal\FunctionalTests\Update\UpdatePathTestBase::$configSchemaCheckerExclusions as in the IS
    3. Run core/modules/block/tests/src/Functional/BlockWeightUpdateTest.php, which uses drupal-10.3.0.filled.standard.php.gz.

    Test passes. I expected it to fail.

    So:
    1. I go through git history: find that πŸ“Œ Deprecate SyndicateBlock Active regenerated the dump (in May). I verify the uuid isn't there for block.block.claro_help_search.yml.
    2. I add uuid as longwave did and compress again (without regenerating), so I'd expect the test failure now. It passes.
    3. I think the array serialization needs {a:12 for the new element . Still passing.
    4. I append to the test $this->assertSame('356ded34-80b4-451a-bdc4-df8d8c0931e1', Block::load('claro_help_search')->uuid());. I expected this to pass at this point, and it does.
    5. I revert the changes to the gz. It fails: Failed asserting that null is identical to '356ded34-80b4-451a-bdc4-df8d8c0931e1'.
    6. So at this point, means that the config validation is actually passing! ❓
    7. I check uuid core data type schema, and see that Symfony's UuidValidator constraint actually allows null as a valid uuid 🀯
    So not sure what to do next to move this forward. I still feel we need an issue for adding the uuid that went missing on the upgrade path as #5 mentioned.

    I'm wondering if we should add a not null contraint to the config entity uuid schema (maybe not here but in #2161149: Force-create new UUIDs in default config objects during initial import/creation β†’ ?)

    config_entity:
      type: mapping
      mapping:
        uuid:
          type: uuid
          label: 'UUID'
          constraint:
              NotNull: []
    

    Hope this throws some light here, because I was really in the dark. But still don't know what's next.

  • πŸ‡ͺπŸ‡ΈSpain penyaskito Seville πŸ’ƒ, Spain πŸ‡ͺπŸ‡Έ, UTC+2 πŸ‡ͺπŸ‡Ί

    And for the record, we need a explicit test for that uuid upgrade with a php fixture on top of drupal-10.3.0.filled.standard. If we add this directly to drupal-10.3.0.filled.standard.php.gz, any time it's regenerated for whatever reason, we would lose it as will not be hitting that concrete upgrade path referenced in #5.

  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Virtually all validators must allow NULL by default to avoid crashing hard on optional values.

    I think you're right that we're missing a NotNull constraint. That's also what type: _core_config_info etc. do.

    IIRC, alternatively, we could mark type: config_entity fully validatable β€” I vaguely remember that causing the NotNull constraint to be added automatically for required key-value pairs, and any type: mapping automatically gets all key-value pairs without requiredKey: false set to be required.
    Yep, confirmed, see:

          if ($root_type_has_opted_in) {
            $data_definition->setRequired(!isset($data_definition['nullable']) || $data_definition['nullable'] === FALSE);
          }
    

    β€” \Drupal\Core\Config\TypedConfigManager::buildDataDefinition()

Production build 0.71.5 2024