Use Composer commands instead of parsing installed.json in TemplateProjectTestBase

Created on 10 March 2023, almost 2 years ago
Updated 14 March 2023, almost 2 years ago

Problem/Motivation

Discovered in #3316368-44: Remove our runtime dependency on composer/composer: remove ComposerUtility .

See \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::getInstalledPackages(), which is called only by \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::createVendorRepository().

These rely on parsing composer/vendor/installed.json, which is not public Composer API, but something internal.

Proposed resolution

That should be able to use ComposerInspector now — so we can stop relying on composer/vendor/installed.json 😊

Remaining tasks

User interface changes

None.

API changes

None.

Data model changes

None.

📌 Task
Status

Fixed

Version

3.0

Component

Code

Created by

🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @wim leers
  • Assigned to omkar.podey
  • Assigned to wim leers
  • Status changed to Needs work almost 2 years ago
  • 🇮🇳India omkar.podey

    As it is now , we can't use ComposerInspector in TemplateProjectTestBase ,as we can neither create a new object for ComposerInspector nor use it as a service, assigning to @Wim.leers for further clarification.

  • Assigned to omkar.podey
  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    Ah, d'oh, right!

    Because that's a service … and this is in a build test. My bad!

    Here you can use direct composer commands instead, and look at ComposerInspector for inspiration.

    So something like:

        $process = new Process(['composer', 'show', '--format=json'], $some_path);
        $process->run();
        if ($error = $process->getErrorOutput()) {
          $this->fail('Process error: ' . $error);
        }
        $output = json_decode($process->getOutput(), TRUE);
    
  • @omkarpodey opened merge request.
  • 🇮🇳India omkar.podey

    Attached diff from what's being returned from \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::createVendorRepository which uses \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::getInstalledPackages the new implementation provides less info , the missing info includes require , require-dev , installation-source. I haven't pinpointed why we hit the error of file not found yet but i think it's related to this.

  • 🇮🇳India omkar.podey

    So, I can't see the core folder at all with the new implementation, then probably this

        $this->runComposer('composer require --no-update --dev drupal/core-dev:*', $template_dir);
    

    is failing silently ?

  • 🇮🇳India omkar.podey

    The weird directory eg - /private/tmp/build_workspace_98a89bde521e2347f08a77b4271819e2f6KEcU/private/tmp/build_workspace_98a89bde521e2347f08a77b4271819e2f6KEcU is created after \Drupal\BuildTests\Framework\BuildTestBase::executeCommand is called .

  • 🇮🇳India omkar.podey

    The command which behave differently

     $command = sprintf(
          "COMPOSER_MIRROR_PATH_REPOS=1 composer create-project %s project --stability dev --repository '%s'",
          $this->runComposer('composer config name', $template_dir),
          json_encode($repository, JSON_UNESCAPED_SLASHES)
        );

    from 3.0.x probably because of the json it's working on as

      file_put_contents($vendor, json_encode($this->createVendorRepository(), JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
        $this->runComposer("composer config repo.vendor composer file://$vendor", $template_dir);

    these set the vendor.json using createVendorRepository() which uses the new implementation.

  • Assigned to phenaproxima
  • 🇺🇸United States phenaproxima Massachusetts

    I wonder if we could use a similar approach here as what core is doing in https://git.drupalcode.org/project/drupal/-/blob/10.1.x/core/tests/Drupa...

  • @phenaproxima opened merge request.
  • Assigned to tedbow
  • Status changed to Needs review almost 2 years ago
  • 🇺🇸United States phenaproxima Massachusetts

    This was some specialist stuff.

    From looking at the similar method in core, I deduced that we need to ensure Composer plugins (i.e., composer/installers, drupal/core-composer-scaffold) run correctly as well for the test project to be built properly. As core does, that requires us to import some information from the plugin's composer.json directly. (It was previously not a problem because all of that stuff is repeated in installed.json, and we weren't removing it.)

    So we still have to directly read some Composer files, but it's just composer.json, which has a defined public schema we can rely on.

    Over to Ted for review.

  • 🇺🇸United States phenaproxima Massachusetts
  • Assigned to phenaproxima
  • Status changed to Needs work almost 2 years ago
  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    Wonderful to see this working! 🚀

    A few requests for clarifying comments on the issue, to make this easier for future maintainers. 🙏

  • Assigned to wim leers
  • Status changed to Needs review almost 2 years ago
  • 🇺🇸United States phenaproxima Massachusetts

    OK, rewrote the comments in the hope of addressing your feedback.

  • Assigned to phenaproxima
  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    I think this is 99% ready. I'd still like to see us try to use options.symlink = FALSE while we're touching this anyway. But that doesn't need to block a commit. I'm fine with committing as-is if you feel strongly about not trying it here and now.

  • Status changed to RTBC almost 2 years ago
  • 🇺🇸United States phenaproxima Massachusetts

    Given Wim's approval in #20, and the fact that I've now detailed the explicit reason why we can't do options.symlink, I think this is good to go.

  • Status changed to Fixed almost 2 years ago
  • 🇺🇸United States phenaproxima Massachusetts
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024