Create a source plugin that scans for locally installed recipes

Created on 9 January 2024, 11 months ago

Problem/Motivation

As part of allowing Project Browser to install recipes, we'll need to provide a source plugin that can expose recipes.

Recipes are currently not exposed by drupal.org, or by any API I'm aware of, so for the time being I think it's okay to just scan the local filesystem for recipes, and have the source list those. Recipes will be located in /PATH/TO/DRUPAL/recipes, and will be identifiable by the presence of a recipe.yml file. The machine name of the recipe is probably the name of the directory containing recipe.yml, and the Composer namespace can be read from the composer.json.

I don't think we need to expose this in the UI right now; let's just see if we can create a kernel test which confirms that the source returns Project objects for all recipes which are present in the filesystem.

I'm also aware that a bunch of questions and probable blockers will arise while we work on this, but we'll handle those as they arise. :)

Feature request
Status

Active

Version

1.0

Component

Code

Created by

🇺🇸United States phenaproxima Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @phenaproxima
  • 🇺🇸United States phenaproxima Massachusetts
  • 🇺🇸United States phenaproxima Massachusetts
  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    🤩🤓

  • First commit to issue fork.
  • Merge request !425Resolve #3413567 "Create a source" → (Open) created by srishtiiee
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Custom Commands Failed
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    71 pass, 4 fail
  • 🇺🇸United States phenaproxima Massachusetts

    @srishtiiee had asked me in Slack:

    I need some directions over evaluating the compatibility of a recipe with the site, do you have any thoughts over it? Do we need to check the compatibility of all its dependencies?

    That's a really good question.

    My feeling at this stage of things is that we should assume recipes are compatible if their composer.json says they are. I see that the source plugin we're building here, as it currently exists, asks for data on each individual recipe -- that's good!

    To be compatible, maybe a recipe should:

    • Declare a dependency (in its composer.json's require section) on drupal/core or drupal/core-recommended
    • That requirement must satisfy (according to semver) the current version of Drupal core. So something like this (psuedocode):
      if (\Composer\Semver\Semver::satisfies($core_version_required, \Drupal::VERSION)) {
        $recipe_is_compatible = TRUE;
      }
      

    I'm sure we'll refine this later on, when drupal.org properly exposes recipes. But given the limited information we can get from Packagist, this might be a reasonable approach in the meantime.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    77 pass
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    72 pass, 2 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    72 pass, 2 fail
  • Status changed to Needs review 10 months ago
  • 🇮🇳India srishtiiee

    That's a random test fail, marking NR anyways.

  • Status changed to Needs work 10 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    77 pass
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Custom Commands Failed
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Custom Commands Failed
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Custom Commands Failed
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    77 pass, 1 fail
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    10 months ago
    #91311
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Waiting for branch to pass
  • Pipeline finished with Canceled
    10 months ago
    Total: 47s
    #91322
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 10 months ago
    Custom Commands Failed
  • Pipeline finished with Success
    10 months ago
    Total: 402s
    #91324
  • Status changed to Needs review 10 months ago
  • 🇮🇳India srishtiiee

    Addressed feedback, setting back to NR.

  • Status changed to Needs work 10 months ago
  • 🇺🇸United States phenaproxima Massachusetts

    I think this is a fantastic start, and quite straightforward and generally easy to understand. There are some spots where it needs to be more robust and better-documented, and we need to expand the test coverage somewhat, but I think this is a very strong foundation upon which to iterate. Nice job, @srishtiiee!

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Custom Commands Failed
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Canceled
    9 months ago
    Total: 340s
    #98843
  • Pipeline finished with Success
    9 months ago
    Total: 352s
    #98846
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    9 months ago
    Total: 329s
    #99180
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Failed
    9 months ago
    #99346
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    9 months ago
    Total: 625s
    #99364
  • Status changed to Needs review 9 months ago
  • 🇮🇳India srishtiiee

    Addressed the feedback, setting back to NR

  • Status changed to Needs work 9 months ago
  • 🇺🇸United States phenaproxima Massachusetts

    This looks really good to me. Most of my remaining feedback is quite minor!

  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    9 months ago
    Total: 331s
    #102158
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    9 months ago
    #102229
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    9 months ago
    #102260
  • Status changed to Needs review 9 months ago
  • 🇮🇳India srishtiiee

    @phenaproxima, thanks for the review! I've addressed it and the MR needs another review.

  • Status changed to Needs work 9 months ago
  • 🇺🇸United States phenaproxima Massachusetts

    Few nits, but otherwise I don't see any reason to hold this up!

  • 🇺🇸United States phenaproxima Massachusetts
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 9 months ago
    Waiting for branch to pass
  • Pipeline finished with Success
    9 months ago
    Total: 346s
    #105697
  • Status changed to Needs review 9 months ago
  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    Can we get some screenshots in the issue summary? 😇 Very curious to see what this looks like for end users! 😄

    (I would not be a good code reviewer here, because I don't know this codebase at all.)

  • 🇺🇸United States phenaproxima Massachusetts

    Thanks for that screenshot, @srishtiiee, it looks good to me! It's a bit sad (and concerning) that none of the recipes in it are compatible -- are we sure that compatibility check is working? At the very least, I've been able to get kanopi/gin-admin-experience to work on Drupal 10.2.x.

  • Status changed to RTBC 9 months ago
  • 🇺🇸United States phenaproxima Massachusetts

    Well, I think that looks pretty good, at least from a code perspective!

    I need to give it a manual test, but since this is a proof of concept, I think we could call this RTBC. It probably doesn't need to be committed right now, but it could be applied in conjunction with the recipe system (which is a core patch itself). I'll add this to my recipe-test repo, which sets up the recipe system with Project Browser.

  • Status changed to Needs work 8 months ago
  • 🇺🇸United States chrisfromredfin Portland, Maine

    After talking to Tim about this, I like the idea that the plugin itself (and test) be provided externally to this project. However, it sounds like we still need/want to move the pagination into the base class.

    Can we break out the DrupalCore.php and ProjectBrowserSourceBase.php changes into a separate issue, and I'll merge that one?

  • 🇨🇦Canada mandclu

    There are recipes listed on the Recipes Cookbook that are posted to drupal.org as module projects, with newer releases reconfigured to be recipes. They don't seem to show up in the packagist.org list. Is that because of the packagist.org integration for drupal.org?

  • 🇺🇸United States thejimbirch Cape Cod, Massachusetts

    @mandclu. I believe that is because the function is pulling from https://packagist.org/, not https://packages.drupal.org/8

      private function fetchRecipeData() : array {
        $recipes = [];
        try {
          $url = 'https://packagist.org/search.json?type=drupal-recipe';
    

    from: https://git.drupalcode.org/project/project_browser/-/merge_requests/425/...

  • 🇺🇸United States chrisfromredfin Portland, Maine

    Also, yes. Only Drupal "general" projects go to Packagist, not module projects.

Production build 0.71.5 2024