Match the version of drupal/core-* metapackages

> The reason that we don't want to require drupal/core with self.version is that it makes upgrading harder.

In what way?

We now recommend that upgrading core be done with "composer update drupal/core*" which updates both packages simultaneously. Won't the dependency work ok with that?

> The reason that we don't want to require drupal/core with self.version is that it makes upgrading harder.

In what way?

I was thinking about why Greg said this and what I saw in https://www.drupal.org/files/issues/2020-08-03/3163114-6.patch → ... so do we really expect and reality also confirms that within the same Drupal minor versions, these meta packages could be on different versions at any time and everything should work? Because if not, the current solution with self.version in the MR is the only way to move forward, if not, then we get back to patch 6 and only lock on major.minor.

It occured to me that Greg's comment is 4 years old and maybe it is from those days when core-recommded did not allow patch version update of dependencies. In those days the concerns he raised regarding self.version was valid.

The Needs Review Queue Bot → tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

Still mergable according to Gitlab.

In what way?

We now recommend that upgrading core be done with "composer update drupal/core*"

I believe I answered this question elsewhere recently, but I am not sure where, so I will re-post here for context.

During the development of Drupal 8.8.x, composer update drupal/core* failed to update, because Composer became confused by the chained strong constraints. One might think that strong constraints would make the dependency graph easier to resolve, because it branches to fewer variations, but for whatever reason, it was harder for Composer in the past. If modern Composer deals with it well, then this change is an improvement.

If I recall correctly, the conflict statement that I recommended worked better (e.g. #9 can be rewritten to allow only one patch version). I don't recall if there was any reason we didn't add it; maybe just never got to it. We certainly did not want to encourage folks to mix different core versions.

This was the Slack thread: https://drupal.slack.com/archives/C392CHBEW/p1720016380610809

I also dumped it via Slackdump before it gets lost, see attachment.

Thanks for looping back to this thread, I'll also do that early next week.

Thanks for saving the conversation before it was lost @mxr576!

As we talked about in #2952616-17: Adopt CommonMark spec for Markdown files → , an automated Slack dumper service would be awesome, pulling down and saving all the great Drupal conversations. A daily drupal.org/project/drupal issue could work as a data repository, consolidating the text for inclusion when searching in Drupal core issues :-)

Drupal Slack Dump for 2024-07-03

Besides a CR any next steps needed here?

Well, Greg expressed his opinion in the referenced Drupal Slack about why he would not rely on core-dev for running tests on downstream projects, but he did not mention that in #27.

I think we are on the an agreement regarding:

We certainly did not want to encourage folks to mix different core versions.

I do not think that enforcing this with conflict or require would make a difference in Composer 2.7+... if there are strong concerns we can try figure that out somehow... without that we can merge the current approach and switch to other when it proved to be problematic.

Well, Greg expressed his opinion in the referenced Drupal Slack about why he would not rely on core-dev for running tests on downstream projects

Don't worry about that old comment, it was unfounded. My point is that you shouldn't use core-dev unless you're using test classes packaged by Drupal to run tests on downstream projects, which is exactly what folks are doing.

My latest comments on moving forward, copied from slack:

It needs to be tested. Testing wouldn't be super easy; someone needs to clone the various Drupal projects with all / most of the versions tagged, and then make some test Drupal sites that are out of date, and try to run "composer update" on them. The proposed dependencies were too much for Composer some years ago; the hypothesis is that modern Composer will handle it just fine. This is likely the case, but it would be unfortunate to deploy the changes to everyone & then find out in the field that it doesn't work well.

To do a good test, you'd probably want to rewrite the tags so that the old versions in the repo have the new dependency relationships. If you only made the proposed changes for the newest tag (as would happen when the MR is merged), then it's possible that Composer would do okay if there was only one version with the new dependency relationships, but might get confused in the future when they started to grow in number.

From the summary:

1. composer create-project 'drupal/recommended-project:^8.9' d8_9_composer
2. cd d8_9_composer
3. composer require drupal/core-dev

You would expect to get drupal/core-dev for 8.9.x

First: Why would you expect that? It's not how Composer works unless the other packages have declared tighter constraints. If you get the error message, you'll figure out that you need to specify a version. Speaking of tighter constraints....

Second: If it's a reasonable (or even just desired) expectation, then having drupal/core-dev declare that it conflicts with future (and maybe even past) versions of drupal/core is the most reasonable solution to deliver that.

So, we leave everything else alone, and then only change drupal/core-dev (and -pinned) to conflict with future versions of drupal/core. We can do this because we always generate a new drupal/core-dev with each release, even if all the dependencies are exactly the same.

So drupal/core-dev release 11.0.0 would look like this:

    "conflict": {
        "drupal/core": "<11.0.0 || >11.0.0",
        "webflo/drupal-core-require-dev": "*"
    },

In this circumstance, Composer gets the version of drupal/core, and then finds the version of drupal/core-dev that doesn't conflict, and then we're done. We can also tell people to composer require drupal/core-dev:@stable.

First: Why would you expect that? It's not how Composer works unless the other packages have declared tighter constraints. If you get the error message, you'll figure out that you need to specify a version. Speaking of tighter constraints....

At the very least, these packages should be tied together at the minor version level. Based on past experiences, it might even be beneficial to tie them at the patch level as well. For more details, see this Slack thread about how the 10.2 -> 10.3 upgrade broke tests because Composer installed incompatible versions of Drupal core packages: https://drupal.slack.com/archives/C223PR743/p1719331712444049

Drupal core is a monorepo that is only partially managed as such. If we could guarantee that split packages in the subpackage repositories are always compatible with each other, this problem wouldn't exist—but we can't.

The drupal/core-dev package defines the dependencies needed to run tests, while drupal/core contains the tests that might need adjustments as core-dev dependencies evolve... which is not a problem in a monorepo because update and adjustments could happen in the same MR.

However, the butterfly effect is more visible for downstream projects that require drupal/core-dev, and they often only realize changes are needed when a minor (or, in the worst-case scenario, a patch) update to drupal/core-dev breaks their test suite.

First: Why would you expect that? It's not how Composer works unless the other packages have declared tighter constraints.

I want to second the statements by @mxr576. I would expect drupal/core-dev to be tied to drupal/core. They are designed to work together and are linked where changes in one can require explicit changes in the other, especially since they are developed in the same repository. The same applies true to drupal/core-recommended as we can very easily end up using releases of dependencies that have not been tested with a version of Core defeating the purpose of the package

I do not really care how this is implemented (require or conflict, in drupal/core-dev or on drupal/core) however I would add this question that might help justify which makes more sense if done in drupal/core-dev: Is there ever a case where you would want drupal/core-dev that you would not also want drupal/core? If the answer is no, than require makes the most sense as drupal/core-dev should also bring with it drupal/core, if the answer is yes than conflict would be more logical.

Either case testing should not be too hard, download https://repo.packagist.org/p2/drupal/core-dev.json (or whatever other package to modify), loop over the require/conflict section and add the desired constraint, place the resulting file into a local composer repo available on localhost and can easily see if any noticeable impact occurs as you use composer to require specific versions and constraints of packages.

Updated the IS with a link to testing instructions and tagged.

@cmlara: "I want to second the statements by @mxr576. I would expect drupal/core-dev to be tied to drupal/core."

Well, drupal/core-dev is tied to core. They have the same version number. You can currently say composer require --dev drupal/core-dev:[your core version constraints here] and it works.

However, if you want to be able to not know your current core constraints and just say composer require --dev drupal/core-dev, then we need to change drupal/core-dev as described in #35. We make drupal/core-dev (and -pinned) conflict with versions of drupal/core which aren't the same, and then Composer will do the sorting for us.

@mxr576: "At the very least, these packages should be tied together at the minor version level. Based on past experiences, it might even be beneficial to tie them at the patch level as well."

That's exactly what I proposed in the message you're responding to.

The Needs Review Queue Bot → tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.