Show pipeline info and gitlab_templates version in composer log

Created on 7 December 2024, 15 days ago

Problem/Motivation

When providing support to admins and developers to help with failing pipelines it would be useful if certain information was always available in the log, so we can answer some questions without requesting a re-run etc.

Proposed resolution

  • Display some key $CI_ variables in the composer log, regardless of the setting of _SHOW_ENVIRONMENT_VARIABLES
  • Display the version of gitlab_templates that is being used

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Component

gitlab-ci

Created by

🇬🇧United Kingdom jonathan1055

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @jonathan1055
  • 🇬🇧United Kingdom jonathan1055

    The version can be displayed using

    cat CHANGELOG.md | grep '##' | head -1 | sed 's/##/Gitlab Templates version/' 
    

    This will print the first (highest) version found in the changelog. Obviously, if the pipeline is being run using ref: main then there could be a few more commits after this latest tag. But this at least gives the newest version used.

  • Merge request !302#3492411 CI variables and templates version → (Merged) created by jonathan1055
  • Pipeline finished with Success
    15 days ago
    Total: 113s
    #361999
  • 🇬🇧United Kingdom jonathan1055

    The variable $CI_COMMIT_AUTHOR is currently being included in the output of .show-environment-variables but this can contain the users personal e-mail address. We already remove all variables starting with GITLAB_USER so I suggest we also add CI_COMMIT_AUTHOR to the pattern. I have pushed this change too.

  • Pipeline finished with Success
    15 days ago
    Total: 54s
    #362005
  • Pipeline finished with Success
    14 days ago
    Total: 112s
    #362403
  • Pipeline finished with Failed
    14 days ago
    Total: 112s
    #362494
  • Pipeline finished with Success
    14 days ago
    Total: 112s
    #362497
  • Pipeline finished with Failed
    14 days ago
    Total: 51s
    #362499
  • Pipeline finished with Success
    14 days ago
    Total: 52s
    #362500
  • Pipeline finished with Success
    14 days ago
    Total: 51s
    #362502
  • 🇬🇧United Kingdom jonathan1055

    I have added the variable $GITLAB_TEMPLATES_VERSION to the hidden-variables.yml file and added a test in our "Check Code" job to make sure we match the variable value with the changelog.md. I intentionally set the variable value to '1.6.8' to demonstrate that the pipeline fails - see https://git.drupalcode.org/issue/gitlab_templates-3492411/-/jobs/3627227

    Fixed the version and the job passes
    https://git.drupalcode.org/issue/gitlab_templates-3492411/-/pipelines/36...

  • 🇬🇧United Kingdom jonathan1055

    I have also added the same check to "run-local-checks.sh" so that we can easily detect this locally. This also triggered some Shellcheck suggestions, which I have repeated in the pipeline job.

    Tesated here https://git.drupalcode.org/issue/scheduler-3445052/-/jobs/3627284

    This is ready for feedback.

  • 🇪🇸Spain fjgarlin

    I left a question in the MR about the GITLAB_TEMPLATES_VERSION variable. The rest of the code looks good, but I wonder if we can detect this differently rather than having another variable that we'd need to update.

  • Pipeline finished with Failed
    11 days ago
    Total: 1168s
    #365405
  • Pipeline finished with Success
    11 days ago
    Total: 52s
    #365419
  • 🇬🇧United Kingdom jonathan1055

    Using the new script you created (thank you), we can get the following

    I think we also need to use the value of _CURL_TEMPLATES_REF if it exists, ahead of _GITLAB_TEMPLATES_REF, because when testing a gitlab templates MR via customised .gitlab-ci.yml the _GITLAB_TEMPLATES_REF may be unchanged from the contrib repo permanent value (as is the case with Scheduler).

  • 🇪🇸Spain fjgarlin

    Good point, let's use that other variable instead when calling the PHP script (we can leave the default value, when empty, inside the PHP script).

  • 🇬🇧United Kingdom jonathan1055

    we can leave the default value, when empty, inside the PHP script

    In fact _CURL_TEMPLATES_REF will never be empty, because this will be done after we call *calculate-gitlab-ref and so we'll already have it even if it was empty. But yes, the script should also get some default value just in case.

  • Pipeline finished with Success
    11 days ago
    Total: 51s
    #365553
  • Pipeline finished with Success
    11 days ago
    Total: 55s
    #365696
  • 🇬🇧United Kingdom jonathan1055

    First test of the new script - pipeline submitted via UI, so _GITLAB_TEMPLATES_REF is default-ref (from the form) but the actual ref and the value of _CURL_TEMPLATES_REF is 3492411-ci-variables-and-templates-version as committed in the branch.
    https://git.drupalcode.org/issue/scheduler-3445052/-/jobs/3664387#L26
    The log shows GitLab Templates version: 3492411-ci-variables-and-templates-version which is good, its the value of the input when there are no tag matches.

    I tried to "fake" running with 'default-ref' by setting this is the UI, but of course the curl then fails.
    https://git.drupalcode.org/issue/scheduler-3445052/-/jobs/3664756

    So I think I'll mock a test by calling the script multiple times, in a custom after_sctipt, just to check what happens.

  • Pipeline finished with Failed
    11 days ago
    Total: 51s
    #365819
  • Pipeline finished with Success
    11 days ago
    Total: 51s
    #365884
  • 🇬🇧United Kingdom jonathan1055

    Tested - at the top is the regular output. In the after_script there are some additional calls.
    https://git.drupalcode.org/issue/scheduler-3445052/-/jobs/3666569

    Thanks for starting off the script. Still todo - match it in D7. But this is ready for review.

  • 🇪🇸Spain fjgarlin

    I'd say the main version is good to go, so we need to apply the same for D7. I left some comments in the MR in case we want to make the PHP script compatible with old PHP versions. NW for the D7 part. Great work so far.

  • Pipeline finished with Success
    9 days ago
    Total: 52s
    #367480
  • Pipeline finished with Success
    9 days ago
    Total: 51s
    #367739
  • 🇬🇧United Kingdom jonathan1055

    I've made the changes for Drupal 7, and allowed for PHP5.6 as requested.

    Tested here on API and PHP5.6 https://git.drupalcode.org/issue/api-3391946/-/pipelines/368535 - phpunit fails (but this is unrelated, the same failure happens in the default 7.x-2.x branch

    PHP Fatal error:  Cannot redeclare class Composer\Autoload\ClassLoader in /builds/project/api/vendor/drush/drush/commands/composer/vendor/composer/ClassLoader.php on line 43
    Drush command terminated abnormally due to an unrecoverable error.   [error]

    The important bit is that the Composer job ran the new script OK and correctly displayed the gitlab templates version

    Tested in Scheduler 7.x-1.x on PHP8.1, all OK
    https://git.drupalcode.org/issue/scheduler-3480501/-/pipelines/367774

    Re-tested in D11
    https://git.drupalcode.org/issue/scheduler-3445052/-/pipelines/368543

    Ready for review.

  • 🇪🇸Spain fjgarlin

    Thanks for porting the changes to PHP5.6 and for the additional tests.

    I've re-checked the code and I think this is ready to be merged.

  • Pipeline finished with Skipped
    6 days ago
    #369803
Production build 0.71.5 2024