Recipes' imported config is validated too strictly by default

I believe you captured all that we discussed.

Jealously grabbing this issue for myself. :)

Needs tests, but the overall approach is ready for review.

This is affecting Drupal CMS's recipe construction and testing, so tagging as a Starshot blocker.

One thing I realized we haven't thought about is what to do with lines like:

config:
  import:
    tour: '*'

Should all that config be treated strictly? Or more relaxed?

I wonder if we should do something like this to opt it into strict mode:

config:
  import:
    tour: '!*'

My only concern is that this syntax starts to look arcane and, well, Unix-ey. In a way that is not intuitive for recipe authors. I wonder if we could do something like this, then:

config:
  import:
    tour: '!all' # Or '?all' for the non-strict version

And then later on we could change the '*' token to just 'all', which is a synonym for '?all'.

Thoughts?

One tangential concern I have about this syntax is the !include convention is used by at least one prominent python project in order to include additional files.

I can see wanting to split actions into separate files in the future to keep things organized and it might be nice to match that syntax if a parser is written.

https://community.home-assistant.io/t/use-of-include-of-additional-yaml-... as an example.

I agree this issue is super helpful, just thinking out loud about the use of: !

What if we added another layer
(Sorry on my phone)

Import:
  Strict:
    tour: '*'

Or

Import:
  Lenient:
    tour: '*'

Default or no key could be lenient for backwards compatibility.

But then there is no bash like special characters.

I'm open to considering a different syntax.

The reason I went for ! is because it's reminiscent of !important in CSS. To me, it conveys urgency and primacy. "This is important to me!"

It's the opposite of the question mark, which conveys ambiguity. "Maybe I need this." I think the same reasoning is why PHP has the ? for nullable types, and ?-> as the nullsafe operator.

If recipes ever were able to include other files, then I'm guessing we'd go for a syntax like `@include` or something like that. Dunno.

! And ? Make more sense spelled out but I think nesting like my second comment is more explicit and easier to remember.

My first thought when seeing ! was the not operator so people might think it's an exclusion.

My first thought when seeing ! was the not operator so people might think it's an exclusion.

Oooh, that's a great point - definitely a strong reason to consider some other way to opt into strict comparison.

I think the only benefit for:

import:
  strict:
    tour: '*'

Over

import:
    strict:tour: '*'

Is not having to type strict for each line of you have multiple.

But it's probably safer to need to specify strict on every config you need strict validation.

How about this as an idea:

What if we made the comparisons lenient by default, but then allowed an explicit list of config to be compared strictly? You'd be looking at syntax like this:

config:
  import:
    user: '*'
  strict:
    - user.role.authenticated

This would get us where we need to go, with no special syntax. And it would be extremely simple to implement.

I kinda love this idea.

I think the only benefit for:

import:
  strict:
    tour: '*'

Over

import:
    strict:tour: '*'

Is not having to type strict for each line of you have multiple.

But it's probably safer to need to specify strict on every config you need strict validation.

I like the structure in 16!

Only thought is maybe import and strictImport?

The only argument I can think of against strict_import is that it would apply to stuff in the recipe's config directory too, not just the stuff listed in import. But I don't feel strongly.

One idea: maybe instead of import and strict_import, we could say import (always strict) and ensure (which is lenient)?

Yes and in bash shell there is -f ie --force. Which is pretty self-explanatory. So the command might or might not work but with --force it will work for sure.

I'm not sure ensure is as clear as strict.

Also @oily, force implies that it will impose the structure no matter what, where in this case if it can't it will fail. Also this isn't about running the command so you can pass it, it's for the recipe authors to determine the importance of particular config.

We're sort of derailing the issue, but we've already got plans for what "force" means, and it will be added in another issue.

@phenaproxima May the "force" be with you!

Still needs work.

phenaproxima → credited alexpott → .

@nicxvan, @alexpott, @thejimbirch, and I discussed this in Slack and we found a better syntax. I'm updating the issue summary to reflect the proposal.

We're happy with the design and tests are written (and passing!)...I think this is truly ready for review.

The one thing to note: at the moment, this MR keeps the existing strict-by-default behavior. Should we change that now? Or in a follow-up? It feels a little bit like follow-up material since it is a behavior change and most likely warrants a change record. But, happy to go either way.

Wrote a change record: https://www.drupal.org/node/3478662 →

All threads resolved, well commented and has tests.

This is a great step forward for recipe authors and leniency.

Marking as RTBC.

Added a couple of comments to the MR.

Feedback addressed. Back to Alex.

Committed and pushed f0fda63b58 to 11.x and fcf7d526b3 to 10.4.x. Thanks!

One problem related to this is that 🐛 Drupal does not recognize when the config is identical Active . I have applied a recipe and then immediately applied it again was told that the config was not identical.

There are more cases that need to be handled than just strict or not strict. I wrote about four possibilities, replace, update, ignore, and error, in #3478669-5: Consider making recipes' comparison with existing config lenient by default → .