Fix php_version in Drupal 7 pipelines

Created on 12 October 2024, 2 months ago

Problem/Motivation

The change to make Drupal 11 "current", which was implemented on 8 October #3463894-80: Update templates so 11.0 is the default/current branch β†’ , has caused Drupal 7 pipelines to fail - the log shows:

Composer version 1.10.27 2023-09-29 10:50:23
$ composer config repositories.0 composer https://packages.drupal.org/7
$ composer require --dev drupal/coder:^8.2@stable phpunit/phpunit symfony/filesystem drush/drush:^8.3.3
Warning from https://packages.drupal.org/7: Drupal.org packages ARE NO LONGER AVAILABLE FOR COMPOSER 1!!! Composer 1 is NOT supported. You should upgrade to Composer 2 now! See https://www.drupal.org/drupalorg/blog/ending-packagesdrupalorg-support-for-composer-1
PHP Fatal error:  Uncaught TypeError: Cannot access offset of type string on string in phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:622

I think this is caused by the _TARGET_PHP variable being updated from 8.1 to 8.3 in the above issue, which is also used as the source for PHP_VERSION in the D7 pipeline.

Steps to reproduce

See this weekly pipeline on 5th October which passed,
and this one on 12th October which failed.

Proposed resolution

There are several wasy to resolve this

  1. In includes/include.drupalci.main-d7.yml change global variable php_version to 8.1 instead of $_TARGET_PHP
  2. Create a new variable CORE_LEG_PHP: 8.1 to work alongside CORE_LEG_PHP_MIN and CORE_LEG_PHP_MAX, and use that as the default for PHP_VERSION
  3. Deprecate $_TARGET_PHP (like we have done with _TARGET_CORE, and was planned as a task anyway) and set the PHP_VERSION variable directly. This still allows for overriding, but is safer, and more immune to changes this like fault

Remaining tasks

πŸ› Bug report
Status

Active

Component

gitlab-ci

Created by

πŸ‡¬πŸ‡§United Kingdom jonathan1055

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @jonathan1055
  • πŸ‡ΊπŸ‡ΈUnited States drumm NY, US

    The cause is actually unrelated to any GitLab templates changes, it is what the error message says - Composer 1 support was completely dropped at packages.drupal.org https://www.drupal.org/drupalorg/blog/ending-packagesdrupalorg-support-f... β†’

    Any containers containing Composer 1, or D7 overrides reverting back to Composer 1 should be updated to Composer 2.

    I was able to fix this in one project by updating the PHP Version from 5.6 to 8.1. I believe 5.6 must still have Composer 1 in the container, since it is ancient.

    In this case, it looks like _D7_DRUPAL_COMPOSER_NEEDED: 1 is reverting to Composer 1 at https://git.drupalcode.org/project/scheduler/-/blob/7.x-1.x/.gitlab-ci.y...

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    Ah Thank you. Yes I just compared the logs in detail and can see before we had

    Warning from https://packages.drupal.org/7: New Drupal.org packages & releases ARE NOT AVAILABLE FOR COMPOSER 1!!
    Composer 1 support will be dropped after October 1, 2024. You should upgrade to Composer 2 now. 
    

    and now it says

    Warning from https://packages.drupal.org/7: Drupal.org packages ARE NO LONGER AVAILABLE FOR COMPOSER 1!!!
    Composer 1 is NOT supported. You should upgrade to Composer 2 now! 
    

    I saw the change we made to _TARGET_PHP and thought it could be related. I guess the packages change wasn't made exactly on 1 October, but sometime between then and today.

    _D7_DRUPAL_COMPOSER_NEEDED was added for Drupal 7 contrib projects that have their own composer.jon, I can't recal the details right now, but clearly that's going to have to be revisited.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Let’s go with a D7-variable separate from the other ones if we need to have a specific version of php.

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    @fjgarlin I think you have probably replied from e-mail or on phone, and have not seen the update here by drumm. It's not a php version problem, but is caused by dropping Composer 1 support entirely.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Phone πŸ™‚ yup i missed two comments (too slow to type)

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    It can wait until Monday :-)

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    For my scenario, the solution is really simple. I only set _D7_DRUPAL_COMPOSER_NEEDED=1 because doing this means that the modules required for testing are moved from $CI_PROJECT_DIR/vendor/drupal (where they are initially placed) to $CI_PROJECT_DIR/$_WEB_ROOT/sites/all/modules which is where Drupal 7 core expects them to be. But in switching to Composer 2 the modules are still downloaded just the same. So I think we need to do the move from vendor/drupal to sites/all/modules regardless of which composer version is being used.

    I don't know specifically what other needs there was to use Composer version 1. The issue that added _D7_DRUPAL_COMPOSER_NEEDED was πŸ› D7 contrib pipeline tests fail when requiring a 3rd-party contrib module Fixed so we can check back to find out our reasoning and decision process from there.

  • @jonathan1055 opened merge request.
  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    First commit tested on Scheduler here
    Since finding the probelms I have already committed a custom composer after_script to move these modules, so that is removed in the test.

    We also need to give a message in the log and I think there is no point in reverting Composer to version 1 as that can't be used now. The message can refer users to this issue and ask for problems to be reported here.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Downstream pipeline: https://git.drupalcode.org/project/api/-/jobs/3046591
    The D7 api project uses this scenario.

    We have in the code the following

          if [ "$COMPOSER_VERSION" == "2" ]; then
            if [ "$_D7_DRUPAL_COMPOSER_NEEDED" == "1" ]; then
              # Revert to Composer version 1, see https://www.drupal.org/project/composer/issues/3178950
              echo "Composer 2 is installed, reverting to Composer 1 because _D7_DRUPAL_COMPOSER_NEEDED=$_D7_DRUPAL_COMPOSER_NEEDED"
              composer self-update --1
            else
              echo "Composer 2 is installed, setting allow-plugins"
              composer config --no-plugins allow-plugins.dealerdirect/phpcodesniffer-composer-installer true
            fi
          fi
    

    We will need to change it to NOT force composer 1.

    I think we should also try to detect if we are on composer 1 (ie: very old image), and then try to force composer 2 coposer self-update --2 in that case.

  • πŸ‡ΊπŸ‡ΈUnited States drumm NY, US

    For API module, we should go ahead and mark the D7 version unsupported and can forget about testing it.

    I think it would be okay to not support very old images with composer 1 only. If someone is in that situation, hopefully they can use a new image, like I did for project_composer, or can do the self-update themselves by adding the command to their own .gitlab-ci.yml

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    I went ahead and marked 7.x-2.x unsupported. As long as we don't have other alternative for downstream project, we'll keep it there as it's useful to discover bugs in D7.

    As for the composer 1 images, I guess these are linked to PHP 5.6 and I agree that we shouldn't need to worry about supporting that. We just need to clean up the code from the templates here that _forces_ composer 1. It will keep composer 2 and if it fails it fails.

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    Thanks for the feedback in #11 - 13, I will work on that.

    Regarding the test failures in #10, those were unrelated to this MR, and caused me to raise πŸ“Œ Review D7 pipelines with D11 shift changes Active which is now fixed.

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    I've made the changes discussed above, (a) to remove the reverting to composer 1, and (b) update to composer 2 if one 1.

    Basic test witht _D7_DRUPAL_COMPOSER_NEEDED=0
    https://git.drupalcode.org/project/scheduler/-/jobs/3081121

    As above but with _D7_DRUPAL_COMPOSER_NEEDED=1
    https://git.drupalcode.org/project/scheduler/-/jobs/3081530

    With _D7_DRUPAL_COMPOSER_NEEDED=1 and _TARGET_D7_PHP=5.6
    https://git.drupalcode.org/project/scheduler/-/jobs/3085137

    All works OK for Scheduler, but this module did not need drupal/composer, so will be good to see the downstream pipelines. Maybe I will try to search for other modules which had _D7_DRUPAL_COMPOSER_NEEDED: 1 and see if we can test on them too.

    This is ready for review and feedback.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    The code looks good and the tests above too. Downstream pipeline on the D7 project: https://git.drupalcode.org/issue/gitlab_templates-3480296/-/pipelines/31...

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    I needed to update the _TARGET_PHP and _TARGET_DB_VERSION on the D7 file. Running again: https://git.drupalcode.org/project/gitlab_templates/-/pipelines/313405

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Merged. Thanks!

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    The API phpunit failed with

    PHP Fatal error:  Cannot declare class Composer\Autoload\ClassLoader, because the name is already in use in /builds/project/api/vendor/drush/drush/commands/composer/vendor/composer/ClassLoader.php on line 43
    Drush command terminated abnormally due to an unrecoverable error.   [error]
    Error: Cannot declare class Composer\Autoload\ClassLoader, because
    the name is already in use in
    /builds/project/api/vendor/drush/drush/commands/composer/vendor/composer/ClassLoader.php,
    

    I left in the move of vendor/drupal/composer to vendor/drush/drush/commands/composer
    https://git.drupalcode.org/issue/gitlab_templates-3480296/-/blob/3480296...
    But I guess that needs looking at too?

    [edit - I see you have made another change since I wrote this comment, but leaving it here for reference]

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    That error is more of a warning and it has been there for a while. See this from August for example: https://git.drupalcode.org/project/api/-/jobs/2355569

    So nothing that this change might have brought. As it is D7 API and things are running and we are no longer supporting that branch I've left it as it is.

  • πŸ‡¬πŸ‡§United Kingdom jonathan1055

    OK, thanks for the info. So long as it was not caused by the changes I made then that's great.

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024