Preserve original composer.json in the modules directory

Suggestion (not yet tested).
It could be tested setting the following in your gitlab file:

  - project: issue/gitlab_templates-3397699
    ref: 3397699-preserve-original-composer-file

I don't think we require modules to have a composer.json file yet.

Checked that the file exists. I haven't tested on a project without composer.json file so it'd be great if somebody reviews the code and tests this on a project without it.

MR conflict.

Needs to be tested. This would be useful to unblock #3422720: [D7] Test dependencies are not automatically brought →

Uploading a patch file to test a theory. Please ignore me :-)

I'm testing MR72 on a 7.x branch (which does have a composer.json) and I'm getting an error

cp: cannot stat
 '/builds/project/scheduler/web/modules/custom/scheduler/module.composer.json':
 No such file or directory

I'm going to add some mor debug to the MR to see whats going on.

The problem above was my own fault, as I has not added the extra two variables into my .gitlab-ci.yml

variables:
  _CURL_TEMPLATES_REPO: issue/gitlab_templates-3397699
  _CURL_TEMPLATES_REF: 3397699-preserve-original-composer-file

so that the modifed symlink_project.php was not being retrieved, and the job now works
See https://git.drupalcode.org/project/scheduler/-/jobs/938083

But I want to ask what is the expected behavior? I assumed that the composer.json file should be preserved in locations, the original project root /builds/project/scheduler and the active modules directory /builds/project/scheduler/web/modules/custom/scheduler. But as coded here, the symlink_project script is executed first, then the preserved composer.json.backup is restored only in the active web/modules/custom/{project} folder. The changed version remains in the root project folder. Is that what is wanted? It means we have broken the symlink and now have one actual file in the active modules directory when all the others are symlinked. It may be fine, but just wanted to check that this is what is intended. It should still work OK with the changes on #3423236: Symlink vs Copy →

But I want to ask what is the expected behavior?

For now and for this issue, I think the (only) goal is to have the original module's composer.json under the /builds/project/scheduler/web/modules/custom/MODULE-NAME folder, as the one in /builds/project/MODULE-NAME in changed by the scripts that build the composer step.

Having said that, this was something that was done from the very beginning and perhaps can be revamped from scratch. We could maybe create and target /builds/project/MODULE-NAME/docroot as the folder to build the project on, and then we can do whatever we need on that folder without modifying the "current" folder. Then, when configuring apache within the jobs, we just need to point it to that folder. Also, as a consequence, the symlinking (or copying) might even become easier. If this might be a good idea, it should definitely be done in a separate issue. I think that issues like this one or #3423236: Symlink vs Copy → are a good indication that we should.

Left some really minor feedback in the MR. I think it's almost ready. I'm triggering the MR pipelines against the other projects to run it on them and see the results.

I have added some debug to the D10 main.yml file now, if you want to re-run.

Also the logic for D10 is not quite the same as D7. In D10 we do not assume that the project has a composer.json but do not create one if there is none. But the restore assumes that there is a composer.json. If we are allowing a project not to have a composer.json then maybe we need to add a check before trying to delete it at the end before restoring the backup?

I ran a test on D10 when the contrib module does not have a composer.json

https://git.drupalcode.org/project/scheduler/-/jobs/965421

php expand_composer_json.php gives the following error, because it is trying to read the non-existent file

PHP Warning:  file_get_contents(composer.json): Failed to open stream: No such file or directory in /builds/project/scheduler/expand_composer_json.php on line 16
PHP Warning:  foreach() argument must be of type array|object, null given in /builds/project/scheduler/expand_composer_json.php on line 177

It would be clearer and cleaner to create an empty composer.json if none exists, just like we do in D7. A composer.json file is created anyway, in expand_composer_json.php and is deleted afterwards. Having the conditional test not only removes the error but shows to anyone reading the code that we do not assume every project has a composer.json, which is the correct thing to do, as it is not mandatory even in D10.

I have made the changes and tested on a copy of the job, will test again when pushed to the MR.

Here's a test with the latest change for no composer.json in D10.
https://git.drupalcode.org/project/scheduler/-/jobs/966174

The error is gone and we now get composer.json does not exist, creating empty file
and later

deleting web/modules/custom/scheduler/composer.json
checking for composer.json.backup

This MR is ready for review and others to test. There are several echos to show what's being done, and I will remove these when all testing is complete.

Oddly I see that only BramDriesen is being given credit, maybe because he uploaded a patch file? Commits to a merge request do not seem to cause automatic credit. Maybe you could add me with credit too, before merging and marking fixed.

Added test instructions to issue summary, in particular because it needs the two CURL variables.

Oddly I see that only BramDriesen is being given credit, maybe because he uploaded a patch file?

Don't worry, that was a test on an unrelated drupal.org issue where we were testing a theory. I always review the credits before marking an issue as fixed.

This looks good to me pending the removal of the echo statements.

I have removed the debug echos, merged in 'main' and fixed the conflicts.

Also adjusted the path for the restore of composer.json.backup in D7, now that #3423236: Unify symlink approach, allow folder argument → creates the correct folder.

Re-tested 7.x without a composer.json
https://git.drupalcode.org/project/scheduler/-/pipelines/110484

Re-tested 7.x with a composer.json
https://git.drupalcode.org/project/scheduler/-/pipelines/110493

Well this has now become more important because cspell for contrib has been enabled and this adds a spelling mistake that projects can't do anything about...

./composer.json:36:14     - Unknown word (drupalspoons) -- "drupalspoons/composer-plugin": true

I'm running the latest tests and doing the final checks and will merge shortly.

Merged and marked this as fixed. Will create tag 1.2.2 and roll it out to contrib.

Also:
- Issue description:

There is no known problem with the current expanded composer.json, but there could be in the future.

- Comment #22:

Well this has now become more important because cspell for contrib has been enabled and this adds a spelling mistake that projects can't do anything about...

So perfect timing to merge.

Automatically closed - issue fixed for 2 weeks with no activity.