Pipeline task fails with composer.json file

I don't see how the related issue could cause this to be honest. This problem seems to be different.

You can easily test that if you change the version of the templates to use (https://project.pages.drupalcode.org/gitlab_templates/info/templates-ver...) in your .gitlab-ci.yml file.

Right now, we are on 1.7.1. You could change it to 1.6.14 and see if the issue happens.

So you'd need to change from ref: $_GITLAB_TEMPLATES_REF to ref: 1.6.14. You might need to set this too:

variables:
  _CURL_TEMPLATES_REF: 1.6.14

The related task only affected projects without composer.json file.

Noting that this is a distribution, not a module.

PHPStan seems to be complaining about symlinks:

  RecursiveDirectoryIterator::__construct(/builds/project/droopler/web/module  
  s/custom/droopler/web/modules/custom/droopler/web/modules/custom/droopler/w  
  eb/modules/custom/droopler/web/modules/custom/droopler/web/modules/custom/d  
  roopler/web/modules/custom/droopler/web/modules/custom/droopler/web/modules  
  /custom/droopler/web/modules/custom/droopler/web/modules/custom/droopler/we  
  b/modules/custom/droopler/web/modules/custom/droopler/web/modules/custom/dr  
  oopler/web/modules/custom/droopler/web/modules/custom/droopler/web/modules/  
  custom/droopler/web/modules/custom/droopler/web/modules/custom/droopler/web  
  /modules/custom/droopler/web/modules/custom/droopler/web/modules/custom/dro  
  opler/web/modules/custom/droopler/web/modules/custom/droopler/web/modules/c  
  ustom/droopler/web/modules/custom/droopler/web/modules/custom/droopler/web/  
  modules/custom/droopler/web/modules/custom/droopler/web/modules/custom/droo  
  pler/web/modules/custom/droopler/web/modules/custom/droopler/web/modules/cu  
  stom/droopler/web/modules/custom/droopler/web/modules/custom/droopler/web/m  
  odules/custom/droopler/web/modules/custom/droopler/web/modules/custom/droop  
  ler/web/modules/custom/droopler/web/modules/custom/droopler/web/modules/con  
  trib/search_api/tests/modules/search_api_test_excerpt_field/config/install)  
  : Failed to open directory: Too many levels of symbolic links    

And PHPCS seems to be analysing core code, which is somehow placed inside the "module" custom folder: https://git.drupalcode.org/project/droopler/-/jobs/4201053 (see how "comment" module is being looked at)

Not sure how yet, but I think 🐛 PHPCS error in contributed module caused by core recipe.README.txt Active might be a better candidate for the issue that introduced the issue.

It works now – I changed _CURL_TEMPLATES_REF to 1.6.14, thanks.

Thanks for confirming it works on this version.

We will investigate further on this, for which we might open an issue in your project to investigate and be able to trigger pipelines.

I've created a task for this: https://www.drupal.org/project/droopler/issues/3503626 🐛 [Gitlab] Pipeline task fails Active

Thanks for opening 🐛 [Gitlab] Pipeline task fails Active

Did you know that you can refer to issues using the syntax [ #issuenumber] and this will be rendered with the full title, and coloured according to status, making it easier to see which issue is being linked.

I also spotted in the composer log:

ls: cannot access '*.info.yml': No such file or directory
PROJECT_NAME=droopler_theme, PROJECT_TYPE=theme

So even if this is a distribution, we are deciphering it as a theme. That might may or may not be relevent.

Also in the composer log we get confirmation of which symlinks are being created
https://git.drupalcode.org/project/droopler/-/jobs/4201051#L834 shows

php symlink_project.php "$CI_PROJECT_NAME" 'modules/custom'
Creating symlinks in /builds/project/droopler/web/modules/custom/droopler pointing back to files in /builds/project/droopler
.gitignore
.gitlab-ci.yml
README.md
composer.json
ddev_commands
launch-droopler-cms.sh
patches
recipes
starter-theme
web

So, yes this problem was introduced by 🐛 PHPCS error in contributed module caused by core recipe.README.txt Active because it appeared that these folders were part of the project, because they all existed before Composer and Drupal were installed. So either these files considred part of the project and we need to exclude them from phpcs and phpstan. Or they are not considered part of the project and more nuanced logic is required in symlink_project.php

A very simple solution to this could be to always ignore 'web' when symlinking, even if it did appear to be a project file and existed right at the start.

Actually, I think the correct solution here, if a distribution wants to use phpcs and phpstan, is for that project to have it's own phpcs.xml with the appropriate ignore paths specified. Likewise for phpstan. I will folllow up on this tomorrow ...

I've done some work on how a distribution (or any project) can exclude paths from their own project folder structure when running phpcs. The details are on the Droopler distribution → issue #3503626-5: [Gitlab] Pipeline task fails →

This requires running PHPCS in the project's proper directory, not the overall CI build top-level folder as we currently do. We made a similar change for PHPStan in #3397162: Tweak PHPStan config so paths are always correct and baseline is more usable → and it's been on our wish list to make the same change for PHPCS - we can use the existing issue ✨ Add basepath parameter to PHP Code Sniffer Active which has some useful information.

Thanks for investigating in the other issue. Once you get to the bottom of things we may or may not need to alter things in our scripts, but regardless we can include a section or page in the documentation to talk about "distributions", or even "themes", as the templates are optimised for "projects". In this new section we can do some recommendations like "include their own phpcs.xml file with the right ignore paths", etc.

After doing more investigation I found a really simple solution to both the phpcs and phpstan problems reported in the issue summary. It just requires a different value for _web_root which is 'web', for example _WEB_ROOT: 'alternative-name'. This sorts out the iteration recursion problem in phpstan it allows proper phpcs scanning of the projects own files in /web and does not require the enhancements I'm working on in ✨ Add basepath parameter to PHP Code Sniffer Active

Droopler MR18 has the details, and some debug is still left in, so you can see the results, and we can run it again. But then I will tidy up that MR and suggest they merge it.

That's a great find!!

Something that we could do in the templates is to check early in the "composer" step if _WEB_ROOT folder already exists in the codebase (before we create it) and then throw a warning either:
- Asking them to change that value and exit with an error.
- Informing them (via job output) that the templates will use another value, then change it and persist it via build.env

I like those ideas. (a) Failing with an error and explanation means that the maintainer(s) will have to do something, so they know and have understanding of what the problem was. But (b) internally logging and then changing the value right at the start means we do the work for them and the pipeline runs OK. _WEB_ROOT is a gitlab templates variable, so you could say it is up to the pipeline mechanics to get it right. But I think I prefer (a) because it is simple to do, reliable, and guaranteed no impact on any other pipelines.

I have altered the title so that we can used this issue to to the work, as it contains all the history, investigation and links etc.

I'm happy with (a) as it'd clearly show there is a conflict with the codebase and the setup and that it needs to be fixed one way or another. Having a "web" folder means something for a project, but we can't know what that is, so we can just inform about it and offer a workaround.

There is also the argument for (b), but I think that doing so could just hide potential issues, which (a) would force to fix (either change the value of the variable for CI or rename that folder in the code).