Many core recipes are not idempotent

Created on 14 June 2024, 7 months ago
Updated 31 July 2024, 5 months ago

Problem/Motivation

Reapplying the core's "standard" recipe throws an error. However tried the same with a couple of other recipes those are not throwing any errors and getting reapplied properly.

Steps to reproduce

  1. Install the site with "minimal" profile "drush si minimal"
  2. Apply the "standard" recipe, ddev exec -d /var/www/html php core/scripts/drupal recipe core/recipes/standard -v
  3. Try to reapply the same "standard" recipe with the same command

Proposed resolution

Change the core recipes in whatever way is necessary to make them idempotent (meaning they can be applied more than once), and test that by making CoreRecipesTest apply each recipe twice.

🐛 Bug report
Status

Fixed

Version

10.3

Component
recipe system 

Last updated 2 days ago

Created by

🇮🇳India prashant.c Dharamshala

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @prashant.c
  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    Thanks for reporting. Added this issue to the Recipes Phase 2 roadmap under "Improve the recipe application process"

    https://www.drupal.org/project/distributions_recipes/issues/3446089#reci...

  • Assigned to phenaproxima
  • 🇺🇸United States phenaproxima Massachusetts

    The word here is "idempotent" :) That's a fancy-pants compsci word for "if you do a thing more than once, the result should be as if you only did it once".

    Core recipes should be idempotent. We could adjust CoreRecipesTest to apply all the recipes twice, which would prove that they are. If they're not, then we have some bugs to fix!

  • 🇺🇸United States phenaproxima Massachusetts

    Moving this to core's queue.

  • Pipeline finished with Failed
    5 months ago
    Total: 557s
    #224983
  • Pipeline finished with Canceled
    5 months ago
    Total: 397s
    #225001
  • Pipeline finished with Canceled
    5 months ago
    #225006
  • Pipeline finished with Failed
    5 months ago
    Total: 483s
    #225009
  • Pipeline finished with Canceled
    5 months ago
    Total: 228s
    #225019
  • Pipeline finished with Canceled
    5 months ago
    Total: 240s
    #225023
  • Pipeline finished with Failed
    5 months ago
    Total: 478s
    #225028
  • Pipeline finished with Canceled
    5 months ago
    #225032
  • Pipeline finished with Failed
    5 months ago
    Total: 480s
    #225039
  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    I checked out 3454603-many-core-recipes
    Installed Drupal using php core/scripts/drupal quick-start minimal
    I then applied the standard recipe using php core/scripts/drupal recipe -v standard
    It was successful.
    I then reapplied the recipe using php core/scripts/drupal recipe -v standard
    I got the following error:

    [Drupal\Core\Recipe\RecipePreExistingConfigException]
      The configuration 'core.entity_form_display.node.article.default' exists already and does not matc
      h the recipe's configuration
  • 🇺🇸United States phenaproxima Massachusetts

    Yeah, still working on this.

  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    Can be re-applied
    ==================
    administrator_role
    audio_media_type
    basic_block_type
    basic_html_format_editor
    basic_shortcuts
    comment_base
    content_editor_role
    content_search
    core_recommended_maintanance
    core_recommended_performance
    document_media_type
    editorial_workflow
    example
    feedback_contact_form
    full_html_format_editor
    image_media_type
    page_content_type
    remote_video_content_type
    restricted_html_format
    standard_responsive_images
    tags_taxonomy

    Can't be re-applied
    ==================
    standard

    The configuration 'core.entity_form_display.node.article.default' exists already and does not match the recipe's configuration

    article_comment

    (same as above)

    article_content_type

    (same as above)

    article_tags

    (same as above)

    user_picture

    (same as above)

    core_recommended_admin_theme

    The configuration 'system.theme' exists already and does not match the recipe's configuration

    core_recommended_front_end_theme

    (same as above)

  • Pipeline finished with Failed
    5 months ago
    Total: 497s
    #225610
  • Pipeline finished with Failed
    5 months ago
    Total: 158s
    #225631
  • Pipeline finished with Failed
    5 months ago
    #225640
  • Pipeline finished with Failed
    5 months ago
    Total: 161s
    #225664
  • Pipeline finished with Failed
    5 months ago
    Total: 174s
    #225680
  • Issue was unassigned.
  • Status changed to Needs review 5 months ago
  • 🇺🇸United States phenaproxima Massachusetts
  • Pipeline finished with Failed
    5 months ago
    Total: 163s
    #225691
  • Pipeline finished with Success
    5 months ago
    Total: 595s
    #225697
  • 🇺🇸United States phenaproxima Massachusetts
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    This looks like really nice work. It is create to see createIfNotExists being used to make recipes re-usable. This is why I added it. Nice one @phenaproxima

    The new actions look good too.

  • Status changed to RTBC 5 months ago
  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    All core recipes can be applied twice! Marking RTBC

  • Status changed to Needs work 5 months ago
  • 🇮🇳India prashant.c Dharamshala

    Thanks for the efforts @phenaproxima.

    But still not working for me.

    Pulled the latest code from 3454603-many-core-recipes, on an already installed site tried to apply the standard recipe

    php core/scripts/drupal recipe core/recipes/standard -v
    

    Getting the following error:

    In ConfigConfigurator.php line 47:
                                                                                                                                             
      [Drupal\Core\Recipe\RecipePreExistingConfigException]                                                                                  
      The configuration 'core.entity_form_display.block_content.basic.default' exists already and does not match the recipe's configuration
    Exception trace:
      at /var/www/html/core/lib/Drupal/Core/Recipe/ConfigConfigurator.php:47
     Drupal\Core\Recipe\ConfigConfigurator->__construct() at /var/www/html/core/lib/Drupal/Core/Recipe/Recipe.php:62
     Drupal\Core\Recipe\Recipe::createFromDirectory() at /var/www/html/core/lib/Drupal/Core/Recipe/RecipeConfigurator.php:60
     Drupal\Core\Recipe\RecipeConfigurator::getIncludedRecipe() at /var/www/html/core/lib/Drupal/Core/Recipe/Recipe.php:242
     Drupal\Core\Recipe\Recipe::validateRecipeExists() at /var/www/html/vendor/symfony/validator/Constraints/CallbackValidator.php:34
     Symfony\Component\Validator\Constraints\CallbackValidator->validate() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:747
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateInGroup() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:592
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateGenericNode() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:101
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validate() at /var/www/html/vendor/symfony/validator/Constraints/SequentiallyValidator.php:36
     Symfony\Component\Validator\Constraints\SequentiallyValidator->validate() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:747
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateInGroup() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:592
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateGenericNode() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:101
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validate() at /var/www/html/vendor/symfony/validator/Constraints/AllValidator.php:43
     Symfony\Component\Validator\Constraints\AllValidator->validate() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:747
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateInGroup() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:592
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateGenericNode() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:101
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validate() at /var/www/html/vendor/symfony/validator/Constraints/CollectionValidator.php:59
     Symfony\Component\Validator\Constraints\CollectionValidator->validate() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:747
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateInGroup() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:592
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validateGenericNode() at /var/www/html/vendor/symfony/validator/Validator/RecursiveContextualValidator.php:101
     Symfony\Component\Validator\Validator\RecursiveContextualValidator->validate() at /var/www/html/vendor/symfony/validator/Validator/RecursiveValidator.php:81
     Symfony\Component\Validator\Validator\RecursiveValidator->validate() at /var/www/html/core/lib/Drupal/Core/Recipe/Recipe.php:192
     Drupal\Core\Recipe\Recipe::parse() at /var/www/html/core/lib/Drupal/Core/Recipe/Recipe.php:58
     Drupal\Core\Recipe\Recipe::createFromDirectory() at /var/www/html/core/lib/Drupal/Core/Recipe/RecipeCommand.php:75
     Drupal\Core\Recipe\RecipeCommand->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:279
     Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Application.php:1029
     Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:316
     Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/symfony/console/Application.php:167
     Symfony\Component\Console\Application->run() at /var/www/html/core/scripts/drupal:30
    
    recipe <path>
    
  • 🇮🇳India prashant.c Dharamshala

    Re-tried by dropping the DB and installing the site first with minimal profile as mentioned in the Steps to reproduce of the issue summary.

    Then applying and reapplying the standardprofile works fine.

    Still have a few issues:

    1. Site branding block getting placed twice, one with the initial installation of Drupal with minimal and the second when applying the standardrecipe.
    2. Getting a few warning messages as well but I do not think these are related to recipes
      Warning: Undefined array key "user_admin_people.page_1" in Drupal\views\Plugin\Derivative\ViewsLocalTask->getDerivativeDefinitions() (line 82 of core/modules/views/src/Plugin/Derivative/ViewsLocalTask.php).
      Warning: Undefined array key "content.page_1" in Drupal\views\Plugin\Derivative\ViewsLocalTask->getDerivativeDefinitions() (line 82 of core/modules/views/src/Plugin/Derivative/ViewsLocalTask.php).
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    @Prashant.c I think the task of applying over an standard profile install is an interest one. I wonder if it is possible to apply over a site installed from the standard recipe.

    I think what @Prashant.c's testing implies is that we need to add a re-application test to \Drupal\FunctionalTests\Core\Recipe\StandardRecipeTest and maybe to \Drupal\FunctionalTests\Core\Recipe\StandardRecipeInstallTest... it could be in \Drupal\Tests\standard\Traits\StandardTestTrait::testStandard...

  • Status changed to Active 5 months ago
  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    Applying the standard recipe over minimal profile, you do get block duplication, which I believe is what we were talking about in https://www.drupal.org/project/drupal/issues/3105597#comment-15601255 📌 Copy block configuration from admin theme when enabling an admin theme Active

    I can confirm, trying to install standard recipe over the standard profile errors. I can step through those recipes.

    This is the initial error.

    [Drupal\Core\Recipe\RecipePreExistingConfigException]
    The configuration 'contact.form.feedback' exists already and does not match the recipe's configura
    tion

  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    feedback_contact_form is the only error. All the rest applied successfully on top of the standard profile.

    Can be applied
    ==============
    administrator_role
    article_comment
    article_content_type
    article_tags
    audio_media_type
    basic_block_type
    basic_html_format_editor
    basic_shortcuts
    comment_base
    content_editor_role
    content_search
    core_recommended_admin_theme
    core_recommended_front_end_theme
    core_recommended_maintenance
    core_recommended_performance
    document_media_type
    editorial_workflow
    example
    full_html_format_editor
    image_media_type
    local_video_media_type
    page_content_type
    remote_video_media_type
    restricted_html_format
    standard_responsive_images
    tags_taxonomy

    Can't be applied
    ================
    feedback_contact_form
    standard

  • Status changed to Needs work 5 months ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍
  • Assigned to phenaproxima
  • 🇺🇸United States phenaproxima Massachusetts
  • 🇺🇸United States phenaproxima Massachusetts

    I can't reproduce @Prashant.c's result in #13. My steps, with this MR checked out:

    drush si -y # I confirmed that this installs Standard
    php core/scripts/drupal recipe core/recipes/standard -v
    

    Worked fine. What am I missing?

  • Pipeline finished with Success
    5 months ago
    Total: 477s
    #226783
  • 🇮🇳India prashant.c Dharamshala

    Thanks @phenaproxima
    Now we can reapply the same recipe any number of times without facing any issues. Looks good to me.

    I think we can move it to NR or RTBC.

  • Issue was unassigned.
  • Status changed to RTBC 5 months ago
  • 🇺🇸United States phenaproxima Massachusetts

    I guess I'll just restore RTBC, then!

    To confirm, I was able to reproduce the problem in #17 with this command sequence:

    composer install
    php core/scripts/drupal quick-start standard
    php core/scripts/drupal recipe -v core/recipes/standard
    

    But that is not an idempotency issue, that's because the contact form recipe hard-codes an email address that's different from the one the Standard profile puts into the feedback form. This will be fixed as part of Make it possible for recipes to prompt for input values Fixed , so it's not in scope here.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Backported to 10.3.x because recipes are not API and the whole thing is not stable yet anyway!

    Committed and pushed 9f3ee1ff55 to 11.x and 3fd22d6b62 to 11.0.x and f734ef114e to 10.4.x and 22972e3764 to 10.3.x. Thanks!

    • alexpott committed 22972e37 on 10.3.x
      Issue #3454603 by phenaproxima, thejimbirch, Prashant.c, alexpott: Many...
    • alexpott committed f734ef11 on 10.4.x
      Issue #3454603 by phenaproxima, thejimbirch, Prashant.c, alexpott: Many...
    • alexpott committed 3fd22d6b on 11.0.x
      Issue #3454603 by phenaproxima, thejimbirch, Prashant.c, alexpott: Many...
  • Status changed to Fixed 5 months ago
    • alexpott committed 9f3ee1ff on 11.x
      Issue #3454603 by phenaproxima, thejimbirch, Prashant.c, alexpott: Many...
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024