Gitlab PHPunit jobs at D11 need compatible test dependencies

jonathan1055 → credited vishalkhode → .

This will be tested and committed first, before the other D11 issues.
Credit to @vishalkhode for providing the alternative repo information from #3434325-28: Automated Drupal 11 compatibility fixes for scheduler → . Not all of it was used, I picked the minimum necessary.

The PHPunit tests now run, which is the intention of this commit. None of the jobs pass yet, but that will be done in the other issues.

The additional composer_next_major.json will remain in the repo for testing, but may need review and adjustment in due course. The third-party modules will eventually have their own Drupal 11 release, and then that happens the additional vsc repos can be removed.

This was working, and the Next Major composer job was passing green. But one of the dependencies must have changed because now we get

Your requirements could not be resolved to an installable set of packages.
  Problem 1
    - zendframework/zend-feed[dev-master, dev-develop, 2.11.0, ..., 2.13.x-dev] require php ^5.6 || ^7.0 -> your php version (8.3.9) does not satisfy that requirement.
    - laminas/laminas-feed[2.12.0, ..., 2.12.x-dev] require php ^5.6 || ^7.0 -> your php version (8.3.9) does not satisfy that requirement.
    - symfony/psr-http-message-bridge[v1.1.0, ..., 1.1.x-dev] require php ^5.3.3 || ^7.0 -> your php version (8.3.9) does not satisfy that requirement.
    - drupal/core[8.9.11, ..., 8.9.x-dev] require php ^7.0.8 -> your php version (8.3.9) does not satisfy that requirement.
    - symfony/psr-http-message-bridge[v1.2.0, ..., v1.3.0] require php ^7.1 -> your php version (8.3.9) does not satisfy that requirement.
    - laminas/laminas-feed[2.13.0, ..., 2.14.x-dev] require php ^7.3 || ~8.0.0 -> your php version (8.3.9) does not satisfy that requirement.
    - drupal/core[9.0.10, ..., 9.0.x-dev] require php ^7.3 -> your php version (8.3.9) does not satisfy that requirement.
    - longwave/laminas-diactoros 2.14.1 requires php ^7.3 || ~8.0.0 || ~8.1.0 -> your php version (8.3.9) does not satisfy that requirement.
    - longwave/laminas-diactoros[2.14.2, ..., 2.14.x-dev] require php ^7.3 || ~8.0.0 || ~8.1.0 || ~8.2.0 -> your php version (8.3.9) does not satisfy that requirement.
    - Root composer.json requires drupal/core-recommended 11.x-dev -> satisfiable by drupal/core-recommended[11.x-dev].
    - drupal/commerce dev-3462426-drupal-11-compatibility requires drupal/inline_entity_form ^1.0@RC || ^3.0@RC -> satisfiable by drupal/inline_entity_form[3.0.0-rc15, ..., 3.x-dev].
    - drupal/inline_entity_form[3.0.0-rc15, ..., 3.x-dev] require drupal/core ^8.8 || ^9 || ^10 -> satisfiable by drupal/core[8.8.0-alpha1, ..., 8.9.x-dev, 9.0.0-alpha1, ..., 9.5.x-dev, 10.0.0-alpha1, ..., 10.4.x-dev].
    - You can only install one version of a package, so only one of these can be installed: drupal/core[8.0.0-beta6, ..., 8.9.x-dev, 9.0.0-alpha1, ..., 9.5.x-dev, 10.0.0-alpha1, ..., 10.4.x-dev, 11.0.0-alpha1, ..., 11.x-dev].
    - You can only install one version of a package, so only one of these can be installed: drupal/core[8.7.0-alpha1, ..., 8.9.x-dev, 9.0.0-alpha1, ..., 9.5.x-dev, 10.0.0-alpha1, ..., 10.4.x-dev, 11.0.0-alpha1, ..., 11.x-dev].
    - drupal/core[9.0.0-alpha1, ..., 9.0.0-alpha2] require symfony/psr-http-message-bridge ^1.2.0 -> satisfiable by symfony/psr-http-message-bridge[v1.2.0, v1.3.0].
    - drupal/core[8.9.0-beta1, ..., 8.9.10] require symfony/psr-http-message-bridge ^1.1.2 -> satisfiable by symfony/psr-http-message-bridge[v1.1.2, 1.1.x-dev, v1.2.0, v1.3.0].
    - drupal/core[8.8.0-alpha1, ..., 8.8.x-dev] require zendframework/zend-feed ^2.12 -> satisfiable by laminas/laminas-feed[2.12.0, ..., 2.23.x-dev], zendframework/zend-feed[2.12.0, 2.12.x-dev, 2.13.x-dev].
    - drupal/core[9.4.15, ..., 9.5.x-dev] require longwave/laminas-diactoros ^2.14 -> satisfiable by longwave/laminas-diactoros[2.14.1, 2.14.2, 2.14.x-dev].
    - drupal/core-recommended 11.x-dev requires drupal/core 11.x-dev -> satisfiable by drupal/core[11.x-dev].
    - Root composer.json requires drupal/commerce dev-3462426-drupal-11-compatibility -> satisfiable by drupal/commerce[dev-3462426-drupal-11-compatibility].

Now that we are using this Next Major job properly, and relying on it in MRs to allow PHPstan and PHPunit to run, we need to make the job fail, so that the others do not run if it the dependencies can't be loaded. Otherwise we get misleading messages such as

Executing curl -L --output web/modules/custom/scheduler/phpstan.neon https://git.drupalcode.org/project/gitlab_templates/-/raw/default-ref/assets/phpstan.neon
Warning: Failed to open the file web/modules/custom/scheduler/phpstan.neon: No such file or directory
curl: (23) Failure writing output to destination

in the PHPstan job, and

PHP Warning:  include(): Failed opening '/builds/project/scheduler/vendor/autoload.php' for inclusion (include_path='.:/usr/local/lib/php') in Command line code on line 1
PHP Fatal error:  Uncaught Error: Class "Drupal" not found

in PHPunit

This avoids the problem of the pipeline status remaining as 'blocked' and never appearing to finish.

Summarizing and removing the excess messages, it seems to come down to:

drupal/commerce dev-3462426-drupal-11-compatibility requires drupal/inline_entity_form ^1.0@RC || ^3.0@RC
-> satisfiable by drupal/inline_entity_form[3.0.0-rc15, ..., 3.x-dev].

drupal/inline_entity_form[3.0.0-rc15, ..., 3.x-dev] require drupal/core ^8.8 || ^9 || ^10

Has this changed from before?

Yes, now the temp commit has been removed: https://git.drupalcode.org/project/commerce/-/merge_requests/279/diffs?c.... Hopefully,  we will get the D11 fix for IFE soon.

Meanwhile, the VCS path can be used to unblock the D11 failure.

jonathan1055 → changed the visibility of the branch 3463494-phpunit-next-major to hidden.

jonathan1055 → changed the visibility of the branch 3463494-next-major to hidden.

jonathan1055 → changed the visibility of the branch 2.x to hidden.

Thanks for linking to that commit. Yes I saw there was activity on Commerce and IEF yesterday.

I have created a new MR158 trying to add the IEF vcs path like you suggested, but we now get the slightly different error:

Your requirements could not be resolved to an installable set of packages.
  Problem 1
    - Root composer.json requires drupal/commerce dev-3462426-drupal-11-compatibility -> satisfiable by drupal/commerce[dev-3462426-drupal-11-compatibility].
    - drupal/commerce dev-3462426-drupal-11-compatibility requires drupal/inline_entity_form ^1.0@RC || ^3.0@RC -> found drupal/inline_entity_form[3.0.0-rc15, ..., 3.x-dev] but it conflicts with your root composer.json require (dev-3438428-automated-drupal-11).

How can we override that latest Commerce commit? Maybe we can specify/target a precise commit in that branch? The one before the link you gave above?

We need to override that latest Commerce commit that reverts back to "drupal/inline_entity_form": "^1.0@RC || ^3.0@RC"

I have now added a specifc commit, the one before the link you gave above. We need to use 279/diffs?commit_id=0f7a58fe15f7c810f683c01f2808c9e958f6c377

That's right, but you also have to provide repository with following changes:

"inline_entity_form": {
            "type": "vcs",
            "url": "https://git.drupalcode.org/issue/inline_entity_form-3438428.git"
        },

Yes, that is already in, from before:
https://git.drupalcode.org/project/scheduler/-/merge_requests/158/diffs#...
The message we now get is

  Problem 1
    - Root composer.json requires drupal/commerce dev-3462426-drupal-11-compatibility#0f7a58fe15f7c810f683c01f2808c9e958f6c377 -> satisfiable by drupal/commerce[dev-3462426-drupal-11-compatibility].
    - drupal/commerce dev-3462426-drupal-11-compatibility requires drupal/inline_entity_form ^1.0@RC || ^3.0@RC -> found drupal/inline_entity_form[3.0.0-rc15, ..., 3.x-dev] but it conflicts with your root composer.json require (dev-3438428-automated-drupal-11).

The first line shows that the previous commit is being targetted, as required. But then the second message just has drupal/commerce dev-3462426-drupal-11-compatibility which does not have the commit #, so it still thinks ^1 || ^3 and we get the error.

We can try removing this line https://git.drupalcode.org/project/scheduler/-/merge_requests/158/diffs#...

Maybe, but I don't think that would help. I have found this in the Composer docs
https://getcomposer.org/doc/04-schema.md#package-links
The large warning box includes "the composer.json metadata will still be read from the branch name you specify before the hash". So whilst we would be getting the correct source for Commerce at that commit, the metadata still comes from HEAD of that branch (thus it ignores the commit that changed it). So in this case it is not going to be possible to do it.

I think we will have to wait for those issues to be progressed. Do you know why that temporary commit was removed, to go back to ^1 ||| ^3. That has really blocked our testing, and I think it will block other projects too.

@jonathan1055 Maybe we should fake composer to treat our branch as the stable release. Let me try that, and see if it works.

Yes, I think @jsacksick → has merge the commerce changes and waiting to IEF to be merge and release.

I am also following up on IEF D11 issue 📌 Automated Drupal 11 compatibility fixes for inline_entity_form Needs review to get it merge and release so that it can unblock other dependent modules to run their D11 CI.

If it takes time then we can ask @jsacksick to put that comment back for the time being.

It worked and CI is passing now. Hence, requesting review.

Thanks Vishal, now D11 tests are failing:https://git.drupalcode.org/issue/scheduler-3463494/-/jobs/2223977

Thank you @vishalkhode, I did not know about faking a repo to be named as another one, using 'as'. That's a great tip, and exactly what we need here.

I've created issues to fix the deprecation warnings in getSubscribedEvents() in two 3rd-party modules that Scheduler uses
📌 Add 'array' return type in Drupal\entity\EventSubscriber\EntityDuplicateSubscriber Active
📌 Add 'array' return type in Drupal\rules\EventSubscriber\GenericEventSubscriber Fixed

The MR repos can be added to composer_next_major.json to improve the testing at Next Major.

Rules MR 📌 Add 'array' return type in Drupal\rules\EventSubscriber\GenericEventSubscriber Fixed has been committed, so that change can be reverted to just use "^4@dev"

This is now going to used for any D11 testing, not just 'next major', so amending the title.

Back-ported to 1.x branch just to see if that can also be compatible with Drupal 11. Need this info before making a 2.x release for D11.

Automatically closed - issue fixed for 2 weeks with no activity.