Add gitignore(s) to Composer-ready project templates

Created on 22 September 2019, about 5 years ago
Updated 15 June 2024, 5 months ago

Problem/Motivation

The composer-ready project templates (drupal/recommended-project) nearly shipped with a .gitignore that would have caused all the composer-managed files to be excluded from any git repository the owner of the new project might set up. Discussion late in #2982680: Add composer-ready project templates to Drupal core revealed that there are established and opposing schools of thought about whether vendored files should be a part of one's git repository. That .gitignore was deemed too opinionated for core, and it was just removed entirely with the creation of this follow-on issue to find a more helpful solution.

Excluding any .gitignore from the official composer-built drupal/recommended-project project template is very likely to create an immediate chore for users of it, to figure out what is an appropriate .gitignore for Drupal and get it added to their project.

When this issue originated (in 2019), the goal was to provide a .gitignore starting point in our project templates for these two personas:
a) People who deploy site updates using only git, and so require all code to exist in git.
b) People who do not want large quantities of vendored code in their git repository, and develop advanced CI/build artifact systems to deploy site updates.

As several years have passed, it doesn't seem necessary to provide a starting point for folks who want to have all code in Git (persona "a" above), as best practices have evolved.

Proposed resolution

Have one well-documented .gitignore per project template that meets persona b)'s needs.

Remaining tasks

  • Document how to test Composer templates locally under Tricks for using Composer in local development
  • Check locally with Composer if the scaffolding: 1. Creates a .gitignore, 2. Which is not overwritten/updated
  • Determine the best way to include a .gitignore file when using the drupal/recommended-project Composer template that won't overwrite any existing .gitignore file when the drupal/core-composer-scaffold runs.
  • Determine how to write a test for this.

Release notes snippet

TBD

Feature request
Status

Needs work

Version

11.0 🔥

Component
Composer 

Last updated 1 day ago

No maintainer
Created by

🇺🇸United States mbaynton Minneapolis, US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇩🇰Denmark ressa Copenhagen

    It's so sad that this issue has stalled.

    I just started a new Drupal 10 project, and using the .gitignore file from the MR here worked beautifully.

    I really appreciate all the hard work which went into getting it. Not only are the rules very precise, so that only relevant files are committed, the comments are also extremely helpful and enlightening. For example in helping to choose whether to include custom modules and themes as separate repositories, or simply commit them directly to the repository of the site where they are used.

    This is the kind of information which is invaluable for a new Drupal user, helping them get started well, on their Drupal journey.

    I hope this issue can be restarted, so new Drupal users can have a .gitignore file included in freshly created projects before too long. It will save them much time and trouble down the road.

  • Status changed to Needs review over 1 year ago
  • 🇺🇸United States crasx Denver, CO

    Bringing this back to life @ DC Pittsburgh. Learned way too much about the Drupal composer project templates. Some brilliant stuff for sure.
    First, I wanted to share this nifty command I came across to make testing this easier (ie: create-project from a local directory)
    composer create-project --prefer-dist drupal/recommended-project="@dev" 3082958-test-`date +%s` --repository '{"type": "path", "url": "drupal-3082958/composer/Template/RecommendedProject", "options": {"symlink": false}}'
    source

    I tested MR1627 using the above method with both legacy and recommended project and they worked well. I also think the approach of a .gitignore file with commented lines makes the most sense.
    My initial hesitation with this was in the multiple gitignore files. But looking back at #3082958 it makes sense that we would have different gitignore files for each project template. The rules for those are pretty tightly coupled to the template they come from.
    We have the example.gitignore file that gets scaffolded in from drupal/core, but that only has rules for sites/default/… I played with the idea of putting everything in example.gitignore and scaffolding it to project-root instead of web-root, but I think that will make the file overwhelming as noted in 3082958.
    Now that we have a few years of composer deployments under our belts I think we can make these changes with a little more confidence. I +1 the approach in MR 1627. Using this approach has the added benefit of being compatible with projects like acquia/drupal-recommended-project who also provide a gitignore file via composer template (and not via scaffold)
    In terms of remaining tasks I think we might want to consider opening another ticket to deal with the non-composer gitignore files. If we keep this ticket scoped to the composer templates we can probably move this along

  • 🇩🇰Denmark ressa Copenhagen

    Thanks for the thorough review @crasx. I recently started a Drupal 10 project, and the example .gitignore worked really well (comments in #96 Add gitignore(s) to Composer-ready project templates Needs work ).

    Getting this committed will help users with limited Drupal and Git experience start out on the right foot, and be a step in the right direction of attracting more Drupal site builders and increase Drupal's potential user base, as proposed by Dries in his blog post Drupal is for ambitious site builders.

    Another relevant ambitious site builder issue would be getting a README included in the Recommended project repository: 📌 Add README.md to composer project templates Needs work . Maybe that issue can get some attention at Pittsburgh2023 as well? :)

  • Status changed to Needs work over 1 year ago
  • 🇺🇸United States smustgrave

    Can the MR be updated for 11.x please.

  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MySQL 5.7
    last update over 1 year ago
    30,126 pass
  • Status changed to Needs review over 1 year ago
  • last update over 1 year ago
    29,436 pass
  • 🇩🇰Denmark ressa Copenhagen

    I tried re-basing via the Gitlab GUI ( as someone else also recently did Make it possible for contrib modules to reuse core's commit-code-check.sh Needs work ) which resulted in disaster, and I am getting Failed to create branch '3082958-add-gitignores-11x': invalid reference name '11.x' from Gitlab when I try to create a new branch ...

    So I am attaching a good old patch, what is essentially #70 again.

  • Status changed to RTBC over 1 year ago
  • 🇺🇸United States smustgrave

    Thanks!

  • 🇩🇰Denmark ressa Copenhagen

    Fantastic, thanks @smustgrave and @crasx for moving this issue closer to completion!

  • 🇺🇸United States brad.bulger

    Is the project root .gitignore file meant to replace the webroot example.gitignore? The project root file looks to me like it does everything the webroot file does except ignore /vendor/, which doesn't initially exist in the webroot anyway.

  • 🇺🇸United States crasx Denver, CO

    @brad.bulger it is not. The example.gitignore in Drupal root is used when a user downloads Drupal as a zip, the others are used during composer create-project. Do we maybe want to add a post composer create-project hook to remove example.gitignore?

    Also linking #3294241 which removes the legacy project.

  • 🇺🇸United States brad.bulger

    I see. I'd think comments in the files saying something like what you just said would suffice.

  • last update over 1 year ago
    29,437 pass
  • 18:32
    14:55
    Running
  • last update over 1 year ago
    29,450 pass
  • last update over 1 year ago
    29,486 pass
  • last update over 1 year ago
    29,499 pass
  • last update over 1 year ago
    29,505 pass
  • last update over 1 year ago
    29,551 pass
  • Status changed to Needs work over 1 year ago
  • 🇺🇸United States crasx Denver, CO

    Updating status, planning on circling back to this soon

  • 🇺🇸United States crasx Denver, CO
  • Status changed to Needs review over 1 year ago
  • last update over 1 year ago
    29,511 pass, 1 fail
  • 🇺🇸United States crasx Denver, CO
  • last update over 1 year ago
    29,804 pass
  • 🇩🇰Denmark ressa Copenhagen

    Thanks @crasx, it's more precise now.

    It looks like core/assets/scaffold/files/example.gitignore also needs to be updated. I also see now that Composer is not capitalized a few places, and there was a +80 character line, so here is a re-roll.

  • Status changed to RTBC over 1 year ago
  • 🇺🇸United States smustgrave

    Tests are green and change looks good still.

  • last update over 1 year ago
    29,811 pass
  • last update over 1 year ago
    29,814 pass
  • last update over 1 year ago
    29,815 pass
  • Open on Drupal.org →
    Environment: PHP 8.2 & MySQL 8
    last update over 1 year ago
    Waiting for branch to pass
  • last update over 1 year ago
    29,826 pass
  • last update over 1 year ago
    29,879 pass
  • last update over 1 year ago
    29,877 pass
  • last update over 1 year ago
    29,882 pass
  • last update over 1 year ago
    29,886 pass
  • last update over 1 year ago
    29,908 pass
  • last update over 1 year ago
    29,946 pass
  • last update over 1 year ago
    29,946 pass
  • last update over 1 year ago
    29,953 pass
  • 48:32
    44:02
    Running
  • last update over 1 year ago
    29,958 pass
  • last update over 1 year ago
    29,958 pass
  • last update over 1 year ago
    29,958 pass
  • Status changed to Needs work over 1 year ago
  • 🇳🇿New Zealand quietone

    I am doing triage on the core RTBC queue .

    1) The Issue Summary has a proposed resolution but there are 4 remaining tasks. In reading the comments I see that in #89 @ressa, pointed out that they need to be decided on. From #89 onwards I don't see discussion of those 4 points. Have those items been resolved and I missed it? Was it done in a Slack discussion?

    2) There is a change record here which has not been touched since Jan 2020. That should be reviewed for accuracy and to update the branch/version numbers. Adding tag and setting to NW for this.

    3) I applied the patch and read the comments. I thought the new paragraph in the 2 example.gitignore files could be easy to read if it were in plain English. I have made a patch as a suggestion. I am not running tests, that can be done if there is agreement to keep the changes.

  • 🇳🇿New Zealand quietone

    I just remembered that because this is changing a file managed by the site, it should have a release note snippet. Correct me if I am wrong.

    I've added that heading to the issue summary.

  • Status changed to Needs review about 1 year ago
  • 🇩🇰Denmark ressa Copenhagen

    Thanks @quietone.

    1) The Issue Summary has a proposed resolution but there are 4 remaining tasks. In reading the comments I see that in #89 @ressa, pointed out that they need to be decided on. From #89 onwards I don't see discussion of those 4 points. Have those items been resolved and I missed it? Was it done in a Slack discussion?

    About deciding the structure (the first three items) we probably need @mbaynton, @greg.1.anderson, @Mile23, and @Mixologic to chime in, and make a final decision.

    About committing /vendor and related files by default or not (the fourth item), since #60 Add gitignore(s) to Composer-ready project templates Needs work by @geerlingguy December 2019, the consensus in the comments is to ignore /vendor, and these users have stated that they prefer to not include /vendor:

    • @geerlingguy
    • @nicxvan
    • @ressa
    • @tyeth
    • @lpalgarvio
    • @Leksat (I think ?)
    • @brad.bulger

    Currently, the patch includes /vendor, so as an example, I am attaching a patch which reverses this, and excludes /vendor, and also directories generated by Composer and scaffold files. What do you all think abut that?

  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MariaDB 10.3.22
    last update about 1 year ago
    30,483 pass
  • 🇩🇰Denmark ressa Copenhagen

    Ambitious Site Builders use Git, and it would be nice if they had a great .gitignore file in their new Drupal 10 projects.

  • 🇺🇸United States smustgrave

    Not sure who could make the call about those remaining tasks. May be worth posting in #contribute on slack?

  • 🇩🇰Denmark ressa Copenhagen

    I don't use Slack, but If you could nudge it along by mentioning it in that channel, that would be fantastic.

  • 🇬🇧United Kingdom joachim
    +++ b/composer/Template/RecommendedProject/.gitignore
    @@ -0,0 +1,113 @@
    +# When a development team creates one or more custom modules that
    +# are intended for use on more than one site, the typical strategy
    +# is to register them in Packagist and give them the type
    +# `drupal-custom-module` instead of `drupal-module`. This will cause
    +# Composer to install them to the directory `modules/custom`.
    +#
    

    That seems rather specialized to go in uncommented.

    On all project I've worked on, web/modules/custom is committed to the repo.

    If we have our own modules in a Composer Packagist, they might go in web/modules/COMPANY_NAME for example.

  • 🇩🇰Denmark ressa Copenhagen

    That makes sense @joachim, do you have a suggestion on how to reword it?

    In the meantime, here is a re-roll, to match changes in file locations, and extension.

  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MariaDB 10.3.22
    last update about 1 year ago
    30,511 pass
  • Status changed to Needs work about 1 year ago
  • The Needs Review Queue Bot tested this issue.

    While you are making the above changes, we recommend that you convert this patch to a merge request . Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)

  • Status changed to Needs review about 1 year ago
  • 🇩🇰Denmark ressa Copenhagen

    I tried this and failed (see #112 Add gitignore(s) to Composer-ready project templates Needs work ). Should we stick to an old school patch?

  • 🇺🇸United States smustgrave

    Tagging for framework manager to hopefully get this moving.

  • 🇺🇸United States nicxvan

    nicxvan changed the visibility of the branch 3082958-add-gitignores-to to active.

  • 🇺🇸United States nicxvan

    I'm moving the patch to an MR

  • Merge request !5622Add example gitignores → (Open) created by nicxvan
  • 🇺🇸United States nicxvan

    @ressa I was able to convert your patch to an MR.

  • 🇺🇸United States nicxvan

    Hiding outdated patches.

  • Pipeline finished with Success
    12 months ago
    Total: 489s
    #57401
  • 🇩🇰Denmark ressa Copenhagen

    Perfect, thanks @nicxvan :)

  • 🇦🇺Australia imclean Tasmania
    +++ b/composer/Template/LegacyProject/.gitignore
    @@ -0,0 +1,114 @@
    +# Ignore directories generated by Composer
    +#
    +# See the "installer-paths" section in the top-level composer.json
    +# file.
    +# ------------------------------------------------------------------
    +/drush/Commands/contrib/
    +/composer/
    +/core/
    +/modules/contrib/
    +/themes/contrib/
    +/profiles/contrib/
    +/libraries/
    +
    +# Generally you should only ignore the root vendor directory. It's important
    +# that core/assets/vendor and any other vendor directories within contrib or
    +# custom module, theme, etc., are not ignored unless you purposely do so.
    +/vendor/
    

    I'm a little unclear on this. Wouldn't core/assets/vendor always be ignored if /core/ is ignored, as suggested in the previous section? Same with contrib directories.

  • 🇩🇰Denmark ressa Copenhagen

    It was added in comment #50 Add gitignore(s) to Composer-ready project templates Needs work by @greg.1.anderson, and taken from the example.gitignore file added in 2018 , so perhaps it can be handled in a follow up issue?

    Maybe a framework manager can give it a look over? It would be great if this improvement got committed in 2024.

  • Status changed to Needs work 11 months ago
  • 🇺🇸United States smustgrave

    Learned something new.

    So for the framework manager to take a look can the pros/cons on each remaining task be listed in the summary vs needing to read 130+

  • 🇺🇸United States mile23 Seattle, WA

    +1 on #120. We should not be instructing people to make new packagist entries for their custom modules, and we should not ignore the custom modules directory.

  • 🇺🇸United States mile23 Seattle, WA

    We also need to summarize decisions here in the IS, so a maintainer can review it easily.

  • Status changed to Needs review 11 months ago
  • 🇩🇰Denmark ressa Copenhagen

    I agree, the default .gitignore file should be aimed at the broader audience, and managing modules via Packagist is rare.

  • Pipeline finished with Success
    11 months ago
    Total: 614s
    #70732
  • Status changed to Needs work 11 months ago
  • 🇺🇸United States smustgrave

    Issue summary is still needed though.

  • 🇺🇸United States ultimike Florida, USA

    The fact that there is no example.gitignore in the project root when using the drupal/recommended-project Composer template has always struck me as very odd, especially considering that one exists in the Drupal docroot.

    I've been complaining about this for a few years now (mainly while muttering to myself) and unfortunately for me (heh), with a little help I was able to find this issue. Seeing how close it is to the finish line, I'd love to be a part of getting it done.

    I just reviewed the existing issue summary and the last 70-ish comments. FWIW, my feelings are as follows:

    1. I don't see any need for a project-level example.gitignore to be part of the Drupal core tarball. I really think it is important for one to be included as part of drupal/recommended-project. The fact that we've named it "recommended-project" makes it a bit opinionated, no? So, there's no reason why we shouldn't have an example.gitignore that recommends that Composer dependencies are ignored by Git.

    2. I think a more aggressive approach (as @nickvan detailed in comment 66) is the way to go.

    3. As I just alluded in my first thought above, I think it should be an example.gitignore and not a .gitignore file. I'm a bit confused though because the existing MR [!5622] has a title of "Add example gitignores", but it actually adding two .gitignore (not example.gitignore) files. Is this just a case where the title is old/incorrect? (related to the first "remaining task" listed in the issue summary)

    Regarding the second "remaining task" (.gitignore in tarballs) in the issue summary, is this really something we still need to consider four years later? I think the whole point to this issue is to help folks newer to Drupal who are using drupal/recommended-project (or similar) to set up their code base properly. I think that if you're cloning Drupal core (or downloading the tarball - {{shudder}}), you probably don't need any help with setting up a .gitignore file.

    Regarding the third remaining task (strategy to exclude .gitignore in downloads), I don't see any discussion about this in previous comments. Is this still an issue? If we decide to use example.gitignore instead of .gitignore, how does that impact this task?

    Regarding the fourth remaining task (vendor committed by default), I think this is the easiest to decide - we should ignore the vendor directory. The fact that the current MR does this makes me think that this task is obsolete.

    Finally, my only comment on the existing MR is that if we're including a rule to ignore PhpStorm files, then we should probably do the same for Visual Studio Code files, as the most recent Drupal Local Development Tools survey shows 55.4% of respondents using PhpStorm and 36.9% using Visual Studio Code - not an insignificant amount.

    Let's figure out the next steps we need to complete in order to get this ready for review. It seems that updating the issue summary's remaining tasks is a priority, so any guidance that anyone can provide on that front seems like it would be most helpful.

    thanks,
    -mike

  • 🇩🇰Denmark ressa Copenhagen

    Thanks for nudging this issue along @ultimike, let's get it moving :) About your comments

    I think a more aggressive approach (as @nickvan detailed in comment 66) is the way to go.

    I agree, and that's how it is now, after #116, where I summarized the general opinion (aggressively ignore) and created a patch accordingly.

    As I just alluded in my first thought above, I think it should be an example.gitignore and not a .gitignore file.

    The Issue Summary has this sentence, which I agree with:

    Have one well-documented .gitignore per project template that meets persona a)'s needs without modification, and has the additional rules desired by persona b) commented out. [...]

    I would prefer to not clutter the folder with an example.gitignore file, but prefer a hidden .gitignore file, as it is in the current patch.

    About PHPStorm and VSCode, I agree -- let's add ignore rule for Visual Studio Code files as well.

    About the two example.gitignore files in the current patch, I have no opinion. Perhaps @mbaynton, @Mixologic, or @greg.1.anderson can weigh in, if they are necessary, or can be removed?

    My main goal is to get an aggressively ignoring .gitignore in the project root of drupal/recommended-project, out of the box.

  • 🇬🇧United Kingdom AaronMcHale Edinburgh, Scotland

    I decided to try out the Gorgie for recommended-project in a recent project and am pleased to see it works well.

    Something though I found odd, the .env line is commented out, I don’t think we should comment this out. I can’t think of a single case where you would ever want to commit the .env file, so if you have one you’re pretty much always going to be uncommenting that line. Or worse, forget to uncomment it and accidentally commit your .env file, which could be a real problem!

    Let’s do ourselves a favour and have the .env always ignored by default. I think that’s in the spirit of this being a recommended setup.

  • 🇬🇧United Kingdom james.williams

    tl;dr - I vote for an example.gitiginore file instead of .gitignore !

    I reckon the ignore file would be customised on many (perhaps most?) projects, which would require tricks to get around the composer scaffolding , which can be tricky and frustrating if & when updates are made to the original version of the file in future. It would become an extra thing for core release managers to keep track of changes to, just as they currently have to explicitly state when files like .htaccess are changed. (I suspect putting more responsibility on them is unhelpful & makes the Drupal project less sustainable.)
    I do not see the .gitignore file as a 'functional' file, it's much more of a development / project-management file - so more like settings.php, which we have an example.settings.php version for too.

    But regardless of how any one of us sees this file, the very fact that there are opposing schools of thought - as evidenced by the original report in this issue's summary, and the issue this one came out of - makes me think that we really shouldn't be imposing an approach too much, but instead providing an easy place to start from. For me, that means an aggressive example.gitignore, rather than confusing users with a .gitignore file that will easily lead users to have partially-built codebases that would not be functional. An incomplete codebase would also encourage running composer on production, which I don't believe is a good idea either.

    ---

    Meanwhile, let's be careful about getting confused about what drupal/recommended-project actually is. It is not supposed to be a opinionated recommendation of a default codebase. That's what distributions are/were. Instead, it is simply about setting up the codebase with the recommended directory structure, with a relocated document root, and with versions of dependencies that are 'recommended' in the sense that they have been tested with Drupal core. I suggest that we carefully restrict this issue's goals to the scope of the two personas in the issue summary rather than getting too excited about each of our own opinions ;) Even if ideas like excluding .env files etc might be good! I just mean let's ensure the motivations that drive decisions in this issue align with what the packages exist for.

  • Status changed to Needs review 9 months ago
  • 🇩🇰Denmark ressa Copenhagen

    Thanks @AaronMcHale, great suggestion adding .env by default, I have updated the patch.

    @james.williams: I think I missed the argument before, that using example.gitignore eases the burden on the core release managers, or didn't understand it. But it's a strong argument, so I have renamed the two .gitignore files files to example.gitignore.

  • last update 9 months ago
    29,722 pass
  • 🇺🇸United States nicxvan

    I still think that having an actual .gitignore is more important than an example.

    If there is a .gitignore and you don't want one then the resolution is a lot simpler than if there is no .gitignore and you expected one.
    I also think the use case where a .gitignore is expected is a lot higher in the Drupal community, but I may be wrong.

    As a developer using composer I expect vendor, core, and contrib to be ignored. When starting a new project and those get committed it is something I have to resolve. If I am using a host with no build process and need to commit those files the resolution is a lot simpler, I just delete or comment out the default gitignore.

    For my use case having an example.gitignore does not solve the problem that brought me to this issue in the first place.

  • Pipeline finished with Failed
    9 months ago
    Total: 167s
    #98850
  • 🇩🇰Denmark ressa Copenhagen

    Sorry @nicxvan, I forgot that you managed to create an MR in Gitlab, which I have updated.

    I agree, .gitignore would be easier out-of-the box, so I'll leave it at that again, and not convert to example.gitignore. So this commit only adds ignore rule by default for PhpStorm and VSCode folders, and .env file

    Core maintainers should be allowed to veto .gitignore-files, if it makes maintenance very hard. But it would be great with some feedback from them, to clarify if this assumption is true or not.

  • 🇩🇰Denmark ressa Copenhagen

    @james.williams:

    I reckon the ignore file would be customised on many (perhaps most?) projects, which would require tricks to get around the composer scaffolding, which can be tricky and frustrating if & when updates are made to the original version of the file in future.

    As I understand it, when composer create-project drupal/recommended-project is run, the composer/Template/RecommendedProject/.gitignore file will be copied to the root of the project. After that, it's left to its own devices, and Composer will not update it again.

  • 🇺🇸United States ultimike Florida, USA

    Heh - seems like we're playing ping-pong with example.gitignore vs. .gitignore...

    I really appreciate everyone's speedy comments in the past few days - while that's a great sign of a healthy debate about a feature we all want to make happen, it also could lead us to the dark side (bikeshedding).

    From @ressa's (thanks for keeping the MR updated, by the way) previous comment:

    As I understand it, when composer create-project drupal/recommended-project is run, the composer/Template/RecommendedProject/.gitignore file will be copied to the root of the project. After that, it's left to its own devices, and Composer will not update it again.

    Actually, I think that if we go with .gitignore, and then it is customized, the next time drupal-core-scaffold runs, it will overwrite the changes - just like what happens if robots.txt or .htaccess is customized. This, IMHO, is a solid reason why I think using example.gitignore is the way to go.

    If we name it .gitignore, why don't we just use sites/default/settings.local.php instead of sites/example.settings.local.php (amongst other examples - pun intended)?

    Back to the 4 remaining tasks in the issues summary:

    1. Nothing to be added to the tarball.
    2. It seems like we have consensus on a more aggressive approach.
    3. To .gitignore or example.gitignore - undecided.
    4. Vendor directory should be ignored.

    Am I missing anything?

    -mike

  • Pipeline finished with Failed
    9 months ago
    Total: 170s
    #100275
  • 🇩🇰Denmark ressa Copenhagen

    Thanks for the fast reply @ultimike, I do think that the current debate is healthy and we are making progress, and feel we pretty close now. I much prefer discussion to silence :)

    About "3. To .gitignore or example.gitignore - undecided.":

    Actually, I think that if we go with .gitignore, and then it is customized, the next time drupal-core-scaffold runs, it will overwrite the changes - just like what happens if robots.txt or .htaccess is customized.

    That does seem to be the case ... But it looks like .gitignore can be excluded from any further updates during scaffolding, by setting the file-mapping value to false. See Drupal's Composer Scaffold > Excluding scaffold files .

    I have tried to add this to the MR.

    To experiment with excluding a file from getting updated in a standard drupal/recommended-project installation with an existing scaffolded file, try adding the below snippet, edit the /web/.htaccess file, and update with or without it being excluded. The file is overwritten with default set up, but excluded from updates when file-mapping is set to false.

    "file-mapping": {
        "[web-root]/.htaccess": false
    }
    

    It would be great to be able to actually test the patch with Composer commands locally, and have it do the scaffolding. Tricks for using Composer in local development exists, but is only for contrib modules, so far.

  • 🇺🇸United States nicxvan

    I agree that not updating gitignore after the initial scaffold is ideal as well.

  • 🇬🇧United Kingdom AaronMcHale Edinburgh, Scotland

    I'm in favor of .igtignore and excluding it from scaffold updates.

  • 🇺🇸United States ultimike Florida, USA

    @ressa - thanks for keeping the MR up-to-date.

    It makes sense that with

    "file-mapping": {
        "[web-root]/.htaccess": false
    }

    The .gitignore file that is being added to the drupal/recommended-project template will not get overridden each time the drupal/core-composer-scaffold plugin runs. This will ensure that if/when the .gitignore is modified, those changes are not overwritten the next time drupal/core-composer-scaffold runs.

    While going with example.gitignore is my preferred solution, it is not a hill I am willing to die on and can (quite comfortably) live with this MR providing a .gitignore file by default.

    With this being said, I think we can remove all of the remaining tasks, no?

    • Decide if we want both .gitignore and example.gitignore in the project template files.
    • Decide if we want to add .gitignore to the download tarballs (perhaps not?)
    • Decide on a strategy on how to exclude .gitignore in downloads but not in the project templates, if that's what we decide.
    • Decide if we want to stick with the proposed resolution (vendor committed by default), or reverse to match existing example gitignore file (which by default ignores vendor). The consensus is to ignore vendor

    If so, we can do that and then update the issue summary with more up-to-date info regarding the contents of the MR. Am I missing anything?

    -mike

  • 🇩🇰Denmark ressa Copenhagen

    You're welcome @ultimike, and I agree that the four remaining tasks have been resolved, so the Issue Summary can be updated.

  • 🇺🇸United States joestewart

    Would overwrite be a better option for the .gitignore file?
    https://www.drupal.org/docs/develop/using-composer/using-drupals-compose...

  • Status changed to Needs work 9 months ago
  • 🇩🇰Denmark ressa Copenhagen

    That could work better, thanks for the suggestion @joestewart. Feel free to update the MR using that method instead.

    I am not even sure the current solution works, since testing Composer scaffolding locally is not simple, so I haven't actually had opportunity to run it ... (see last paragraph in #148 Add gitignore(s) to Composer-ready project templates Needs work , "It would be great [...]")

  • Status changed to Needs review 9 months ago
  • 🇩🇰Denmark ressa Copenhagen

    I updated the patch to this, perhaps it could work?

    ".gitignore": {
        "path": "composer/Template/RecommendedProject/.gitignore",
        "mode": "replace",
        "overwrite": false
    }
    
  • Pipeline finished with Failed
    9 months ago
    Total: 169s
    #108333
  • 🇺🇸United States ultimike Florida, USA

    I've updated the issue summary and have been talking to some folks in the Drupal Slack workspace about this issue in hopes of getting more voices in this thread.

    I've also hidden previous patch files - let's stick with the existing MR for this issue.

    Of particular note, I'm really curious to figure out if/how we need to write a test for this change.

    From @crasx's comment 98 above, I'd love to manually test this using the method he described, but I can't decipher exactly what I need to do to make that happen. Can someone post step-by-step instructions?

    thanks,
    -mike

  • Status changed to Needs work 8 months ago
  • 🇺🇸United States smustgrave

    Removing that tag

    But MR appears to have a spell error.

    And also would love to see this land (hopefully by D11)

  • 🇺🇸United States smustgrave

    May just need a rebase

  • Assigned to nicxvan
  • 🇺🇸United States nicxvan

    I'll do it.

  • Issue was unassigned.
  • Status changed to Needs review 8 months ago
  • 🇺🇸United States nicxvan

    That seems to have worked!

  • Pipeline finished with Success
    8 months ago
    Total: 489s
    #114801
  • Status changed to RTBC 8 months ago
  • 🇺🇸United States smustgrave

    lot of comments to digest but this does appear to be working when used. Maybe any additional additions can be follow ups

  • 🇺🇸United States ultimike Florida, USA

    Thanks, @nicxvan and @smustgrave - do either of you feel that a test of some sort will be necessary for this to get merged?

    -mike

  • 🇩🇰Denmark ressa Copenhagen

    Also, can someone test with Composer commands locally and have it do the scaffolding, to check if the .gitignore is overwritten, or not? Or perhaps that has already been confirmed? I added it in the Issue Summary as a remaining task.

  • 🇬🇧United Kingdom longwave UK

    I don't understand what composer/Template/LegacyProject/.gitignore is for or where it would be used.

    Also, I don't see how the path setting works in the recommended template, because the composer directory isn't present in scaffolded installs.

  • Status changed to Active 8 months ago
  • 🇩🇰Denmark ressa Copenhagen

    Great questions @longwave, perhaps the LegacyProject file can be removed from the patch?

    It's a blocker for this issue, that we can't test this Composer template patch locally, to check if the scaffolding works.

    @mbaynton kindly shared a method in #19 Add gitignore(s) to Composer-ready project templates Needs work , but things have changed since 2019, so perhaps someone can share an updated method? I have added two tasks in the Issue Summary:

    • Document how to test Composer templates locally (and add to Composer docs )
    • Check locally with Composer if the scaffolding: 1. Creates a .gitignore file, 2. Which is not overwritten/updated
      [...]
  • 🇺🇸United States benjifisher Boston area

    I will see if I can come up with testing instructions. But before I forget ...

    There is some discussion of whether to call the file .gitignore or example.gitignore., and the current MR uses .gitignore. The problem with that is it can have unexpected effects because Git will respect what it says in any subdirectory of composer/Template/LegacyProject/ or composer/Template/RecommendedProject/.

    TL;DR: I think we should go with example.gitignore.

  • Status changed to Needs work 8 months ago
  • 🇺🇸United States benjifisher Boston area

    I think my previous comment duplicates the start of the issue summary:

    The composer-ready project templates (drupal/recommended-project) nearly shipped with a .gitignore that would have caused all the composer-managed files to be excluded from any git repository the owner of the new project might set up.

    @longwave asked on the MR:

    How does this path work?

    In fact, the current approach does not work.

    It is reasonable to guess that the drupal/recommended-project metapackage is generated from composer/Template/RecommendedProject/ by cloning that directory, since currently that means just the one file composer.json. If that were the case, then adding .gitignore in that directory would add it to the metapackage.

    In fact, the metapackage is generated using the code in composer/Generator/. Looking at the code there, I think it creates a metapackage with just the one file composer.json, just as it says in composer/Metapackage/README.txt:

    A metapackage is a Composer package that contains only a composer.json, and
    has no other content. In other words, the purpose of a metapackage is to
    provide dependencies, not to provide code or data.

    I think that what will work is

    1. Add a file, say recommended-project.gitignore, to core/assets/scaffold/files/.
    2. Reference that file in composer/Template/RecommendedProject/composer.json.

    Back to NW.

    From Comment #162:

    ... this does appear to be working when used.

    @smustgrave, can you describe your testing steps?

  • 🇺🇸United States benjifisher Boston area

    On second thought, referencing the gitignore file from composer/Template/RecommendedProject/composer.json does not work. The code in composer/Generator/Builder/DrupalCoreRecommendedBuilder.php builds composer/Metapackage/CoreRecommended/composer.json from composer.lock.

    What does work is adding the extra/drupal-scaffold section to the array in DrupalCoreRecommendedBuilder::initialPackageMetadata().

    Testing instructions

    For manual testing, execute

    composer run-script post-update-cmd
    

    and confirm that composer/Metapackage/CoreRecommended/composer.json is updated as intended.

    If someone can confirm this, then we should add these instructions to the issue summary.

  • 🇺🇸United States nicxvan

    What if we called it template.gitignore?
    Is there a way to have the scaffold change the name from template.gitignore to .gitignore?
    The objection to using example.gitignore is that it will not by default ignore the files that a CI approach would want ignored whereas having a gitignore you don't want is much easier to resolve.

    Example implies suggestion where template implies that something is meant as a starter I think.

  • 🇺🇸United States benjifisher Boston area

    BTW, the other drupal-scaffold configuration in composer/Metapackage/CoreRecommended/composer.json comes from core/composer.json.

  • 🇺🇸United States nicxvan

    Just to clarify what I think the requirements have become.

    After setting the initial .gitignore it should not be overwritten if the user updates the gitignore.
    The template .gitignore should not add gitignore rules to where the template is stored i.e. it should not be called .gitignore

    An actual gitignore file was settled on because it is less destructive to have a gitignore and not want one than to not have a gitignore and want one.

  • 🇬🇧United Kingdom AaronMcHale Edinburgh, Scotland

    I agree that the end result should be that a .gitignore file (of that name) is placed at the root of the project after running the composer create-project command; And that existing project should, ideally, not be impacted.

    Whether we call it template.gitignore or something different in our source control, I don't feel strongly, assuming we have a mechanism to place it at the project root with a different name.

  • 🇺🇸United States ultimike Florida, USA

    @benjifisher - thank so much for your work on this issue. I know it has been a few weeks since we discussed this at MidCamp - I appreciate your patience with me finally having a moment to take a look at this.

    @nicxvan, @AaronMcHale - appreciate your input in keeping this discussion going.

    To summarize what I believe @benjifisher is suggesting:

    1. From comment 168 above, add a new recommended-project.gitignore (it can't be example.gitignore because that file already exists for web/example.gitignore for drupal/recommended-project), to core/assets/scaffold/files/.
    2. From comment 169 above, add a new extras section to DrupalCoreRecommendedBuilder::initialPackageMetadata():
    protected function initialPackageMetadata() {
        return [
          "name" => "drupal/core-recommended",
          "type" => "metapackage",
          "description" => "Core and its dependencies with known-compatible minor versions. Require this project INSTEAD OF drupal/core.",
          "license" => "GPL-2.0-or-later",
          "conflict" => [
            "webflo/drupal-core-strict" => "*",
          ],
          "require" => [
            "drupal/core" => Composer::drupalVersionBranch(),
          ],
          "extra": {
            "drupal-scaffold": {
              "file-mapping": {
                "[project-root]/.gitignore": "assets/scaffold/files/recommended-project.gitignore"
              }
            }
          }
        ];
      }

    If I understand all this correctly, we are no longer involving LegacyProject at all, nor do we need to, correct?

    Am I understanding this correctly?

    thanks,
    -mike

  • 🇬🇧United Kingdom AaronMcHale Edinburgh, Scotland

    Thanks for summarizing that @ultimike, based on what you're saying there I think that all makes sense, and it looks like the end result will be a .gitignore file in the project root, so I am happy :)

  • 🇩🇰Denmark ressa Copenhagen

    To prevent this issue from stalling, it would be amazing if someone with the right skills could check locally with Composer if the scaffolding in the patch:

    1. Creates a .gitignore
    2. Which is not overwritten/updated
  • 🇩🇰Denmark ressa Copenhagen

    Drupal 11 is planned to be released the week of July 29, 2024. It would be great to include this improvement.

Production build 0.71.5 2024