`type: mapping` should use `ValidKeys: <infer>` constraint by default

Created on 25 July 2023, 11 months ago
Updated 11 October 2023, 9 months ago

Problem/Motivation

✨ Add validation constraints to config_entity.dependencies Fixed introduced the ValidKeys constraint for use on type: mapping. This validation constraint has particularly strong test coverage (see \Drupal\KernelTests\Core\TypedData\ValidKeysConstraintValidatorTest).

It allows either specifying an explicit list of allowed keys, or inferring them based on the keys defined for a mapping in the config schema.

It is currently adopted by:

  1. config_dependencies_base β†’ automatically inferred: ValidKeys: '<infer>'
  2. config_dependencies β†’ automatically inferred: ValidKeys: '<infer>'
  3. _core_config_info β†’ NOT automatically inferred: ValidKeys: ['default_config_hash'] (because this is a low-level config schema type that is NOT extensible, although it absolutely could use ValidKeys: '<infer>' too)

It made sense to introduce this constraint first and adopt it narrowly, and to then adopt it more widely. That's what this issue is about.

The whole point of type: mapping is that you have known keys. If you don't know, you have to use type: sequence. But there is no validation happening on this today πŸ™ˆ.

Only the config_test.validation mapping is validated, for testing purposes, in \Drupal\KernelTests\Config\TypedConfigTest::testSimpleConfigValidation(), by \Drupal\config_test\ConfigValidation::validateMapping(). That is essentially a hardcoded equivalent of the ValidKeys validation constraint (introduced in #1928868: Typed config incorrectly implements Typed Data interfaces β†’ and last refined in #2925689: ConfigValidation class contains code that is brittle and changing for every addition β†’ ).

Steps to reproduce

N/A

Proposed resolution

Use this constraint by default for type: mapping:

mapping:
    label: Mapping
    class: '\Drupal\Core\Config\Schema\Mapping'
    definition_class: '\Drupal\Core\TypedData\MapDataDefinition'
+   constraints:
+     # By default, only allow the explicitly listed mapping keys.
+     ValidKeys: '<infer>'

This has zero consequences today because no configuration is validated. (It's only checked if it matches the structure of the config schema, but only fairly loosely.)

The comment captures what it does. It makes type: mapping all across Drupal core actually sanely validatable by default! πŸš€

Remaining tasks

TBD

User interface changes

None.

API changes

None.

Data model changes

type: mapping's implementation now actually matches its documentation: only known keys are allowed.

Release notes snippet

N/A

πŸ“Œ Task
Status

Fixed

Version

11.0 πŸ”₯

Component
ConfigurationΒ  β†’

Last updated about 3 hours ago

Created by

πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @Wim Leers
  • Open on Drupal.org β†’
    Environment: PHP 8.2 & MySQL 8
    51:05
    51:05
    Queueing
  • @wim-leers opened merge request.
  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Clarifying the current status of Drupal core.

  • last update 11 months ago
    29,795 pass, 6 fail
  • last update 11 months ago
    29,796 pass, 6 fail
  • Issue was unassigned.
  • Status changed to Needs review 11 months ago
  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    At this point, we could choose to remove the test-only validation constraint:

    diff --git a/core/modules/config/tests/config_test/config/schema/config_test.schema.yml b/core/modules/config/tests/config_test/config/schema/config_test.schema.yml
    index 2a707facb0..87f84bb9e3 100644
    --- a/core/modules/config/tests/config_test/config/schema/config_test.schema.yml
    +++ b/core/modules/config/tests/config_test/config/schema/config_test.schema.yml
    @@ -172,9 +172,6 @@ system.action.*.third_party.config_test:
     config_test.validation:
       type: config_object
       label: 'Configuration type'
    -  constraints:
    -    Callback:
    -      callback: [\Drupal\config_test\ConfigValidation, validateMapping]
       mapping:
         llama:
           type: string
    

    … because it literally is surfacing the same problem that ValidKeys: '<infer>' surfaces, but with more precision. Applying the above diff would result in

        // 2 constraint violations triggered by the default validation constraint
        // for `type: mapping`
        // @see \Drupal\Core\Validation\Plugin\Validation\Constraint\ValidKeysConstraint
        $this->assertSame('', $result->get(0)->getPropertyPath());
        $this->assertEquals("'elephant' is not a supported key.", $result->get(0)->getMessage());
        $this->assertSame('', $result->get(1)->getPropertyPath());
        $this->assertEquals("'zebra' is not a supported key.", $result->get(1)->getMessage());
    

    keeping this πŸ‘† and removing this πŸ‘‡

        // 1 additional constraint violation triggered by the custom
        // constraint for the `config_test.validation` type, which indirectly
        // extends `type: mapping` (via `type: config_object`).
        // @see \Drupal\config_test\ConfigValidation::validateMapping()
        $this->assertEquals('', $result->get(2)->getPropertyPath());
        $this->assertEquals('Unexpected keys: elephant, zebra', $result->get(2)->getMessage());
    

    However, it's harmless to keep both. And in a way, it serves as a nice example to show that it's always possible to add additional validation constraints. So I vote to keep it as-is.

  • last update 11 months ago
    29,880 pass
  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Thanks to \Drupal\Core\Config\TypedConfigManager::determineType() and ::getFallbackName(), e.g. type: field.storage_settings.[%parent.type] automatically resolves to field.storage_settings.boolean, but then falls back to field.storage_settings.* (because there's no specific settings for the boolean field type).

    However, type: field.storage_settings.* does not specify a concrete mapping … and nor does type: mapping! That's what this was crashing on.

    So either we need to re-create all of the inheritance/extension logic in \Drupal\Core\Validation\Plugin\Validation\Constraint\ValidKeysConstraint::inferKeys() … or we just rely on \Drupal\Core\Config\TypedConfigManager::getDefinitionWithReplacements() doing that for us like it has done for the past ~decade. Then all we need to is ensure that even the root type (type: mapping) has a list of allowed mapping keys specified, even if that list is empty. That way, it is nicely inherited down the chain (type: mapping β†’ type: field.storage_settings.* β†’

    Then everything starts working! πŸ‘

  • πŸ‡§πŸ‡ͺBelgium borisson_ Mechelen, πŸ‡§πŸ‡ͺ

    In comment #4 Wim wasn't sure about keeping the ValidMapping constraint. It is a different implementation, so not completely the same and I agree that it is valuable to keep both. It is a good example to keep.

    I'm not sure I completely understand the comment in #5, but I can see why setting the ValidKeys: '<infer>' on the root mapping type bubbles up, so that makes a lot of sense.

    This is probably the simplest of all the config schema validation mapping issues that was committed over the last few weeks and it provides some additional strictness. It is weird to see nothing in core actually have an error against this, as opposed to some of the other issues, but the test coverage proves this works.

  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    It is weird to see nothing in core actually have an error against this, as opposed to some of the other issues, but the test coverage proves this works.

    It's not that weird, because we're not validating all config in Drupal core yet. πŸ“Œ KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate Fixed would change that, and would likely trigger additional failures.

    Regarding #5: see the test failures for the commits prior to the last, they contain output like this:

    assert(array_key_exists('mapping', $definition)) in /var/www/html/core/lib/Drupal/Core/Validation/Plugin/Validation/Constraint/ValidKeysConstraint.php:93
    

    That means the mapping key-value pair was not defined on the given config schema type definition. So, for example, field.storage_settings.* does not contain it. And hence the assertion failed.

    By adding mapping: {} to type: mapping's definition, we ensure that the mapping key-value pair is always defined for all mappings (and child types such as config_object, as well as grandchild types such as field.storage_settings.*). That's what #5 tried to explain, probably not concretely enough πŸ™ˆ

  • Status changed to RTBC 11 months ago
  • πŸ‡§πŸ‡ͺBelgium borisson_ Mechelen, πŸ‡§πŸ‡ͺ

    Now that I am more confident I'll RTBC this one, let's break πŸ“Œ KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate Fixed even harder.

  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    🀣 Thanks… I guess? :P

  • πŸ‡§πŸ‡ͺBelgium borisson_ Mechelen, πŸ‡§πŸ‡ͺ
  • last update 11 months ago
    29,879 pass, 5 fail
  • Assigned to Wim Leers
  • Status changed to Needs work 11 months ago
  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Did that!

    Tests are still running, but I already see several failures. Yay for πŸ“Œ KernelTestBase::$strictConfigSchema = TRUE and BrowserTestBase::$strictConfigSchema = TRUE do not actually strictly validate Fixed keeping us more honest going forward! 😊

    Will investigate the failures as soon as the full test suite finishes running.

  • last update 11 months ago
    29,884 pass
  • Issue was unassigned.
  • Status changed to Needs review 11 months ago
  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Well that turned out to be very simple! 🀩 Barely any violations in Drupal core! πŸ‘

  • Status changed to RTBC 11 months ago
  • πŸ‡§πŸ‡ͺBelgium borisson_ Mechelen, πŸ‡§πŸ‡ͺ

    This is great, super happy to see that the strict config schema checking found 5 more fails.

    • longwave β†’ committed b90b2e30 on 11.x
      Issue #3376794 by Wim Leers, borisson_: `type: mapping` should use `...
  • Status changed to Fixed 11 months ago
  • πŸ‡¬πŸ‡§United Kingdom longwave UK

    Committed b90b2e3 and pushed to 11.x. Thanks!

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • Status changed to Fixed 9 months ago
  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί
Production build 0.69.0 2024