Create test that reports % of config entity types (and config schema types) that is validatable

The Needs Review Queue Bot → tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

The use_more_text should be label but sometimes it saved as false because it should be translatable

Thank to @chx https://drupal.slack.com/archives/C1BMUQ9U6/p1683905207064219

sync/views.view.apps.yml:      use_more_text: 'See all >'
sync/views.view.content_recent.yml:      use_more_text: More
sync/views.view.release_notes.yml:      use_more_text: 'View All'
sync/views.view.smartsheet_in_the_news.yml:        use_more_text: false
sync/views.view.smartsheet_in_the_news.yml:      use_more_text: 'View All ›'
sync/views.view.smartsheet_in_the_news.yml:        use_more_text: false
sync/views.view.smartsheet_in_the_news.yml:      use_more_text: 'Visit the newsroom ›'
sync/views.view.smartsheet_in_the_news.yml:        use_more_text: false
sync/views.view.smartsheet_in_the_news.yml:      use_more_text: 'View All ›'
sync/views.view.taxonomy_term_content_hub_categories.yml:      use_more_text: 'View All ›'
sync/views.view.taxonomy_term_content_hub_categories.yml:        use_more_text: false
sync/views.view.taxonomy_term_content_hub_categories.yml:      use_more_text: 'View All ›'
sync/views.view.tmgmt_job_overview.yml:        use_more_text: false
sync/views.view.tmgmt_job_overview.yml:      use_more_text: 'See all jobs »'
sync/views.view.tmgmt_job_overview.yml:        use_more_text: false
sync/views.view.tmgmt_job_overview.yml:      use_more_text: 'See all jobs »'
sync/views.view.user_admin_people.yml:      use_more_text: more

Filed child issue 🐛 Convert config schema to label for views use_more_text Closed: works as designed

I think that based on what I did at 📌 Expose validation constraint violations in Config Inspector UI and drush command Fixed , I should be able to simplify things here 🤞

@Wim Leers, can we mark this test as incomplete, so that we can allow this to run on d.o infrastructure and have output from it, without it having to sit in the queue?

https://docs.phpunit.de/en/10.2/writing-tests-for-phpunit.html#incomplet...

Since #12, a handful of crucial issues have landed:

1. type: machine_name: 📌 Add config validation for the allowed characters of machine names Fixed
2. type: _core_config_info: 📌 Add validation constraints to _core_config_info Fixed
3. type: langcode: ✨ Add a `langcode` data type to config schema Fixed

The last 2 both in the past 2 weeks!

These have made the following fully validatable:

1. type: config_object
2. comment.settings
3. media_library.settings
4. menu_ui.settings
5. node.settings
6. system.feature_flags

Rerolled for all that, and compared to ~6 months ago in #3:

🚀 Some real progress! 🤩

#23: agreed, and looking into that 👍

Self-review after ~6 months of absence and progress in Drupal core:

+++ b/core/config/schema/core.data_types.schema.yml
@@ -22,30 +22,45 @@ ignore:
 boolean:
   label: 'Boolean'
   class: '\Drupal\Core\TypedData\Plugin\DataType\BooleanData'
+  constraints:
+    PrimitiveType: []
...
 integer:
   label: 'Integer'
   class: '\Drupal\Core\TypedData\Plugin\DataType\IntegerData'
+  constraints:
+    PrimitiveType: [ ]

This was done because:

Fortunately, \Drupal\Core\TypedData\TypedDataManager::getDefaultConstraints() handles that for us. IMHO that should be moved into the config schema for explicitness, but that's subjective. 🤓

— #6.1

… but as we're trying to get this to a committable state, reverting it.

In addition to #25, I still stand by the rationale in #6.2 to do

mapping:
    label: Mapping
    class: '\Drupal\Core\Config\Schema\Mapping'
    definition_class: '\Drupal\Core\TypedData\MapDataDefinition'
+   constraints:
+     # By default, only allow the explicitly listed mapping keys.
+     ValidKeys: '<infer>'

… but this is doing far more than just "create test". So let's remove that from the scope here and make this about only a test. Created 📌 `type: mapping` should use `ValidKeys: ` constraint by default Fixed for that.

The last submitted patch, 25: 3324984-25.patch, failed testing. View results →

The last submitted patch, 26: 3324984-26.patch, failed testing. View results →

Applying the trick that @borisson_ proposed. That indeed should make tests pass!

If committed, this would:

1. allow us to prevent further regressions: all new config schema types (and new config entity types) will cause this test to fail if they don't have validation constraints
2. track progress of how many config schema types (and config entity types) are fully validatable: any time an additional one becomes validatable, this test will fail and would require an addition to \Drupal\KernelTests\Core\Config\ConfigSchemaValidatabilityTest::FINISHED_CONFIG_SCHEMA_SUBTREES

To get the detailed output (as shown in e.g. #3), you'd have to run it locally and uncomment the two lines that contain this:

$this->markTestIncomplete('Remove this when https://www.drupal.org/project/drupal/issues/2164373 is complete.');

was added in #5 and is obsolete thanks to ✨ Add a `langcode` data type to config schema Fixed .

Marking for:

1. extracting one more piece from this issue into another
2. making sure the logic is as simple as possible — my work on Config Inspector ~2 months ago suggests this can be simplified

1. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +  /**
   +   * {@inheritdoc}
   +   *
   +   * Ironically, strict config schema checking must be disabled for this test to
   +   * run at all, because Drupal core does not have config schema for everything!
   

   Mixing inheritdoc and documentation is not allowed, so we need to copy over the rest of the documentation from the base implementation

2. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +  protected function getRawConfigSchemaDefinition(string $base_plugin_id): array {
   

   this needs docs

3. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +    // @todo support non-tail dynamic types.
   

   We need to file a followup or fix this in this issue as well.

   I'd prefer a followup because this is already very helpful as is.

4. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +    $this->assertCount(594, $incomplete, 'A new config schema type was added without validation constraints.');
   

   This can also be false when a complete schema is fixed; not sure if we need to change the message here.

   I don't think we should, so can be ignored.

5. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +   * (This is considered a single node in the tree.)
   

   brackets are not needed here I think.

6. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +  protected function isBaseType(string $type): bool {
   

   Needs docs

7. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +  protected static function getSubtree(array $definition): array {
   

   needs docs

8. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +  protected function getArrayElementProperties(array $definition): array {
   

   needs docs

9. +++ b/core/tests/Drupal/KernelTests/Core/Config/ConfigSchemaValidatabilityTest.php
   @@ -0,0 +1,668 @@
   +  protected function computeSubtreeValidatability(array $subtree, string $parent_property_path) : ConfigSchemaValidatability {
   

   needs docs

#31: thanks, indeed all those things I still need to address 👍 #30.2 should actually address the majority of those 😊

Meanwhile, 📌 `type: mapping` should use `ValidKeys: ` constraint by default Fixed landed, and led to another big boost! Or … it should have, but the numbers remain identical 🤔 That must be a bug. I do remember working on 📌 Expose validation constraint violations in Config Inspector UI and drush command Fixed and starting from the code I wrote here, but realizing I could massively simplify it and make it more accurate. I'm hopeful that doing #30.2 will also fix that. 🤞

Update after many more issues landed.

Update since ~1 month ago in #24:

🚀 Some real progress! 🤩

Tackling #30 + #31 tomorrow.

This is how the numbers in #33 were computed.

This is an unsustainable core patch until such a time that it lands in core. It'll eventually be useful to do something like this … but only when we actually reach 100%, to avoid regressing.

To be clear, we did make a ton of progress in the past few months:

$ git pull
Already up to date.
$ vendor/bin/drush config:inspect | grep 100
  comment.settings                                       Correct   100%          ✅✅  
  menu_ui.settings                                       Correct   100%          ✅✅  
  node.settings                                          Correct   100%          ✅✅  
  shortcut.set.default                                   Correct   100%          ✅✅  
  system.feature_flags                                   Correct   100%          ✅✅  
  system.maintenance                                     Correct   100%          ✅✅  
  system.menu.account                                    Correct   100%          ✅✅  
  system.menu.admin                                      Correct   100%          ✅✅  
  system.menu.footer                                     Correct   100%          ✅✅  
  system.menu.main                                       Correct   100%          ✅✅  
  system.menu.tools                                      Correct   100%          ✅✅  
  user.mail                                              Correct   100%          ✅✅  
$ vendor/bin/drush config:inspect | grep 100 | wc -l
      12

Twelve fully validatable simple config objects right now!

FYI: Over at 📌 Automated report on core config validatability Needs review , I'm working to generate a nice automated report, which eventually should also include charts 🤓

The very first run is happening right now: https://git.drupalcode.org/project/config_inspector/-/jobs/147889 🤩

On Dec 30, 📌 Automated report on core config validatability Needs review landed. It now generates an up-to-date visualization of core config validatability daily.

It is currently underestimating because 📌 Configuration schema & required values: add test coverage for `nullable: true` validation support Fixed did not update the ones listed/surfaced by #35. Fixing that in 📌 Follow-up for #3364109: opt in already validatgable simple config to FullyValidatable Active .

Thanks to ✨ Add a "Validatable config" tests job to GitLab CI to help core evolve towards 100% validatability Fixed having landed, we don't need this anymore! 🚀

Automatically closed - issue fixed for 2 weeks with no activity.