Composer Lint needs to check original composer.json in addition to the modified version

Good one. The reason why is because we are creating the composer.json for the Drupal project inside the pipeline and that's the file that we are checking after, but we should check the original composer.json file instead.

Cool, thanks for confirming. Thought I had lost my marbles for a spell there, spent a good while debugging this :)

Interesting. This implies that the parsing and loading of the project's composer.json here in scripts/expand_composer_json.php works just fine and no error was detected.

The un-modified composer.json is restored into $_WEB_ROOT/modules/custom/$CI_PROJECT_NAME so either we run composer validate in that folder, not the top level /builds/project/flexible_permissions, OR we copy the un-modified composer.json back to the top-level at the start of the Composer Lint job (we did this for CSpell)

Yeah, probably json_decode is not too picky about trailing commas.

More than happy with that approach (copy the file back to the root for the job).

I've put together an MR that should fix this situation. Please test/review (instructions here)

Wouldn't it make sense to validate both composer.json files? Although the generated one is not distributed with the module, it is still used to install the test environment.

Good point. It won't hurt to do both for sure. The last commit addresses that.

Downstream pipelines: https://git.drupalcode.org/project/decoupled_pages/-/jobs/1714414 and https://git.drupalcode.org/project/keycdn/-/jobs/1714430

We can see how the created composer.json file is good but the module's composer.json files report some issues. I'll remove the lock file as starters as we don't need it for this job and then see.

KeyCDN is happy now but Decoupled Pages is not...
https://git.drupalcode.org/project/decoupled_pages/-/jobs/1714483

$ test -f $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME/composer.json && cp $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME/composer.json composer.json && rm composer.lock && composer validate
In PluginManager.php line 752:
                                                                               
  composer/installers contains a Composer plugin which is blocked by your all  
  ow-plugins config. You may add it to the list if you consider it safe.       
  You can run "composer config --no-plugins allow-plugins.composer/installers  
   [true|false]" to enable it (true) or disable it explicitly and suppress th  
  is exception (false)                                                         
  See https://getcomposer.org/allow-plugins                                    
                                                                               
validate [--no-check-all] [--check-lock] [--no-check-lock] [--no-check-publish] [--no-check-version] [-A|--with-dependencies] [--strict] [--] [<file>]

I've no idea why this happens. It works locally as the file is actually very small and simple.

It seems happier with this approach.
1. We validate the global composer.json file, as before
2. If there is a composer.json file in the module, we "cd" into the module folder, and we run "composer validate" there, and then we go back to the project root folder

Downstream pipeline: https://git.drupalcode.org/project/gitlab_templates/-/pipelines/184800

We now have one single command that changes directory into $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME, runs validate and switches back to to the top level $CI_PROJECT_DIR. Because we are validating two composer.json files which can give different results, I think we need to be clearer in the log regarding what is being done. It needs an echo or something to show the second directory that is being checked, or some print to state 'this is your projects composer.json' and 'this is the generated composer.json'. If it's not clear we will get some confused maintainers asking for support. Maybe break the command into a couple of separate statements?

Also, is the --no-check-lock option working? The log still shows Lock file warnings:

$ test -f $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME/composer.json && cd $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME && composer validate --no-check-lock && cd $CI_PROJECT_DIR
./composer.json is valid but your composer.lock has some warnings
# Lock file warnings
- The lock file is not up to date with the latest changes in composer.json, it is recommended that you run `composer update` or `composer update <package name>`

Added a couple of echos to let people know what's happening.

Also confused by the --no-check-lock option. Maybe it just makes it a warning instead of an error. No idea. I changed the approached by renaming the lock file into something else and then renaming it back to what it was.
- keycdn: https://git.drupalcode.org/project/keycdn/-/jobs/1716252
- decoupled_pages: https://git.drupalcode.org/project/decoupled_pages/-/jobs/1716236

I feel like I'm not qualified to RTBC this, but from what I can see and judging by the comments it seems to make sense.

Thanks for those changes. I have made a few minor suggestions in the MR.
I will also test with Scheduler, and check it works with a project that does not have a composer.json. Is this change going to be ported to main-d7? I think it should be, even though composer.json is not mandatory in D7.

I applied all the feedback, thanks!

Is this change going to be ported to main-d7
There is no "composer-lint" job in D7, so I'd say no unless we can do something really easy to check this.

We'd need to test:
- Project without composer.json file
- Project with composer.json file with an error in it (like the one shown when creating this issue)

There is no "composer-lint" job in D7

of course! sorry, dumb question :-)

I will check some projects that I'm involved with, but I think they all have a composer.json file. Could fake it, by deleting the file in a prior commit. I will try that.

New messages and pwd look good
https://git.drupalcode.org/project/scheduler/-/jobs/1716603

Testing when project has no composer.json file - also runs OK
https://git.drupalcode.org/project/scheduler/-/jobs/1716727

Tested with a bad composer.json (added trailing comma) - fails as expected
https://git.drupalcode.org/project/scheduler/-/pipelines/184999

Retested after last commit (different backup name) and the job ran as before. Given the comments in #15 by OP @kristiaanvandeneynde and the input from @simonbaese I am marking this RTBC.

RTBC +1

Merged. Thanks so much for the comments/review/testing.

Thanks for fixing this!

Automatically closed - issue fixed for 2 weeks with no activity.