[policy, no patch] Consider whether to keep Package Manager and Automatic Updates in a separate repo/package than core in order to facilitate releasing updates to the updater

Since the IS is filled out and it's a policy question that doesn't need a patch in this issue (but will affect how to create the MR in ✨ Add Alpha level Experimental Package Manager module Needs review ), setting this issue's status to Needs review.

If we commit this improvement to 11.2 only (not backport to 11.1 or 11.0), then no one can test it (without applying some version remapping shenanigans) until after 11.3 is released, since you'd first need to update to 11.2 using 11.1's updater and then updating to 11.3 would be the first core minor update that you can perform with 11.2's updater.

I think we can work around this with early alpha releases for minor versions, or hidden development module that allows you to update to dev releases.

Now suppose that after 11.1 is released, and work is happening towards 11.2, that we come up with some improvement to the way that Automatic Updates can update minor core versions. Let's suppose we would classify this as either a non-critical bugfix or as a feature improvement, rather than as a critical bugfix.

We backport non-critical bugfixes (and tasks) to the current stable branch quite regularly - as long as they're safe to backport that's fine. I can't really think of a change that's an 'improvement' to a minor update that wouldn't also in some way be a bug fix, unless it was purely a UX change to attended updates, although even that could be an accessibility issue.

So for me I'd rather be more aggressive about backports for automatic updates, than add the overhead of a separate repository with its own issue queues and releases to do.

If we want anyone to actually be able to use it right away, we'd need to backport it all the way to 11.0, which is something we typically would only do for security fixes.

This is much more of a problem usually, however we have occasionally backported upgrade path fixes to security-only branches though, so if it was something that was actually going to prevent people from updating from that branch, it would still fit within what we currently do already, just an extra subset of what we might have to backport.

Disadvantages of a separate repository:

1. Any changes to core APIs that affect package_manager or automatic_updates won't run tests against those modules, making it more likely they get broken by core deprecations or out of sync with phpcs/phpstan changes.

2. To resolve #1, we could ensure that they're tested as part of the core pipeline, but that introduces a further problem that any MR for example deprecating a method across core would need versions both for core and AU/PM, and then we'd need a way to test the core MRs with the MR versions of those modules.

3. Theoretical, but it opens up the possibility of someone pinning AU in their project composer.json to a specific branch or tag, preventing AU updates and it getting out of sync with core. This is unlikely, but it could happen if there was a paper bag release of AU and someone worked around it by forcing a downgrade via composer cli, then forgot to remove their workaround.

These are all problems that are specific to modules shipped with core that are in a separate repository. They don't affect contrib modules which have their own independent release schedules, they don't affect PHP libraries that don't call core APIs, and they don't affect modules that are included with core.

So, I think that we shouldn't do this because it would introduce a lot of complexity that we currently don't have to deal with.

That leaves a couple of options:

1. We spend effort to identify any PM/AU issues that should be backported to patch releases and try to make sure that happens fairly promptly after commit to a minor release. And also do earlier minor alpha releases if necessary to support additional testing.

2. We could completely change course and leave Automatic Updates in contrib so that it has its own independent release schedule. However automatic updates in contrib only works if Project Browser is in core, which requires Package Manager in core - so it could be heavily promoted via that and installed without needing composer. If Package Manager is in core, then I'm not sure that would help resolve the problem in the issue summary anyway.

We backport non-critical bugfixes (and tasks) to the current stable branch quite regularly - as long as they're safe to backport that's fine. I can't really think of a change that's an 'improvement' to a minor update that wouldn't also in some way be a bug fix, unless it was purely a UX change to attended updates, although even that could be an accessibility issue.

So for me I'd rather be more aggressive about backports for automatic updates, than add the overhead of a separate repository with its own issue queues and releases to do.

I agree with both of these.

Not removing needs FM tag to give other FMs a chance to review

From an RM perspective I agree that the disadvantages seem to outweigh the advantages. Given that we have more control over the code that is running if we keep it all in the one repository, and we can always make exceptions to our release policy in order to fix a critical bug, I think that we should try to be aggressive with AU related backports and if there is a technical need to make an extra release of an otherwise unsupported minor version then we should just do that. If we needed to I think it's much easier to document that you must "upgrade to 10.x.y before upgrading to 11.x.y" rather than having to explain that someone must upgrade the AU Composer package separately before upgrading core.

Thanks for all the feedback. I discussed this with @tedbow and we agreed that as long as sufficiently important bug fixes are backported to the production branch(es), then that resolves the original concern here, so closing this as "works as designed" (even though "will work as designed" would be technically more accurate).

When I have a chance I'll open a followup to discuss some more details about what else might be needed to facilitate testing: both manual and automated, both pre-commit and post-commit.

For the reasons already stated I also support a more 'aggressive' approach to AU related backports.

Whatever the decision here it should be documented. For example, if we have a different approach to backports here then the backport policy should be updated.

I think we would still follow the existing backport policy, but at the moment we lean towards not backporting issues if they require any additional thought/effort to backport even if something technically could be backported. Early on in 8.x minor releases we tried to backport more things to patch releases, but in some cases by the time a backportable version was ready, it was too late anyway. Now we're more likely to mark the issue 'fixed' in the development branch, and then if someone actively wants to work on a backport, they can re-open it and do that.

So for AU if it's something we really want to backport, we might need to do things like create a dedicated backport MR that doesn't include deprecations or similar, and default to trying this instead of not. But the policy for what can actually be backported would remain the same. We can still add release notes for patch releases if there's no non-disruptive way to backport a critical bugfix too.

It was an accident to tag, Needs framework manager review.

FWIW, +1 to keeping it all in the core repo and trying to backport as much as we can. That's how we've handled update manager for the last ~15 years, and while far from perfect, we've all made it work.

Given #9, I think there's nothing to document. Sounds like we vetted that the status quo plan, both in terms of targeting Automatic Updates to be in the core repository, and working within existing backport policy guidelines but just being more likely to actually backport the things that are backport-eligible, is good enough.