[policy] Remove update hooks added until 10.3 from Drupal 11

Created on 13 November 2023, about 1 year ago
Updated 8 February 2024, 10 months ago

Problem/Motivation

See 🌱 [policy, no patch] Remove old update hooks prior to each major version (Drupal 10 and later) Needs review for background discussion and reasoning. See #3290810: Remove updates added prior to 9.4.0 (9.4.4 for ckeditor) and add 9.4.0 database dumps β†’ for the 9.4/9.5 example for how the actual process happens.

Very short tl;dr:

1. When updates exist in both 10 and 11 major branches, this means they can run going from 10.0 to 10.5, 10.0 to 11.0, 10.0 to 11.5 , 10.5 to 11.5 etc. The more potential release versions there are to upgrade to and from, the more potential interactions and dependencies there are between different upgrade paths - either core updates added years apart from each other, or core + contrib updates running at the same time. We do not have an upgrade test framework that can handle every possibility of core version updates (this would only be possibly with build tests), and even if we did, there is still contrib updates to add on top.

2. To reduce (but not eliminate) this risk, for every major release since 8.x (and also before), we've remove updates added to the previous major, so that you had to update to at least 8.8.x before 9.x, and at least 9.4.x before 10.x

3. Because 10.x is going to be maintained for two years or more into the 11.x cycle, we will need to either continue to backport updates from 11.x to 10.x, so that they run on either version, or only commit them to 11.x if they're not required for backwards/forwards compatibility. Either way this increases the likelihood of things going wrong.

Steps to reproduce

Proposed resolution

For 9.x/10.x, we originally removed updates to 9.3. We then went back and removed 9.3-9.4 updates in a later issue/commit. This was partly because by then we knew we'd be releasing Drupal 10.0 in December with 9.5.

The trickiest update currently in 10.x (that I can think of) is πŸ› Errors: The following table(s) do not have a primary key: forum_index Needs review and would be included in 10.2 update removals. However we also have πŸ› Timestamp field items are affected by 2038 bug Needs work which is even more complex of an upgrade path, and is almost certainly not going to land in 10.2.x.

I think we should consider deciding now that we're going to remove 10.3.x updates from 11.x, and then do the update removals, recreate the database dumps etc. once when 10.3 approaches alpha. This will reduce doing work twice, and we can set the expectation that all sites will need to get onto 10.3 before moving to 11.x, this would be whether or not we release in the June, September, or December windows.

By getting all sites onto 10.3 before they do the major upgrade, it also helps to get sites updated to the latest contrib versions as well (indirectly), and there will be much better 11.x compatibility amongst contrib modules when 10.3 is out.

This is stricter/more pre-emptive than previous major releases, but given we have a likely, tricky, update path to add and also the longer 10.x cycle, should save everyone lots of pain later hopefully.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

πŸ“Œ Task
Status

Fixed

Version

11.0 πŸ”₯

Component
Database updateΒ  β†’

Last updated 9 days ago

No maintainer
Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @catch
  • πŸ‡³πŸ‡ΏNew Zealand quietone
  • Status changed to Needs review about 1 year ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Moving to needs review.

    Assuming we go ahead with this plan, we should open a postponed issue and remove the old database updates as close as possible to the 10.3 beta, to minimize the chances of having to regenerate database dumps multiple times since that's still time consuming and error prone. On the other hand if we're able to get πŸ“Œ Rework database update tests so we don't have to ship database dumps in git Active done it might then be easier to remove updates on a rolling basis.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    By getting all sites onto 10.3 before they do the major upgrade, it also helps to get sites updated to the latest contrib versions as well (indirectly), and there will be much better 11.x compatibility amongst contrib modules when 10.3 is out.

    I like this part.

  • Status changed to RTBC 12 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Been about a week but no one has openly objected. Want me to open the followup mentioned in #3?

  • πŸ‡¬πŸ‡§United Kingdom catch

    This could use a +1 from a release manager who's not me still I think, tagging for that.

  • Status changed to Needs review 11 months ago
  • πŸ‡­πŸ‡ΊHungary GΓ‘bor Hojtsy Hungary

    Now that Drupal 10 will be supported until the Drupal 12 is released, would it not be in line with that intent to keep the update functions instead? I understand that removing the upgrade functions is standard practice, but https://www.drupal.org/blog/drupal-10-will-be-supported-until-the-releas... β†’ would be counter to that?

  • πŸ‡¬πŸ‡§United Kingdom catch

    There are three reasons to remove older update functions:

    1. To encourage sites to update contributed modules to 11.x-compatible versions before updating core to 11.x. i.e. sites on 10.3.x with updated contrib module are much more likely to update to 11.0.x successfully than sites on 10.0.0 trying to jump all the way to 11.x and updating their contrib modules at the same time. They probably won't be able to update contrib modules running 10.0.0 because they may rely on new APIs added in 10.1 to 10.3.

    2. To minimize the number of (core and contrib) update functions that can potentially have to run in the same update, for example if a site tried to update all the way from 10.0.0 to 11.6.0 (which would represent 4-5 years of core development) there could be tens of updates that need to run, config that gets updated multiple times for different changes etc. The more updates can potentially run, the more likely it is that we end up with dependency issues where update X has to run before update Y but after update Z and similar.

    3. To reduce the matrix of update paths that we don't have test coverage for. e.g. ideally we'd have test coverage for 10.3-11.0, 10.4-11.0, 10.5-11.0, 10.3-11.1, 10.4-11.1, 10.5-11.1, 11.0-11.1 etc. but we don't have anything like that and can't add it with the current update test infrastructure.

    I think that #1 is about the same with 10-11 as it was for 9-10. But #2 and #3 will actually be worse with two active branches, since we'll have more total minor releases to juggle, more updates paths to potentially backport between major versions (or not), and more potential paths to for people to take between different combinations of versions.

    Or tl;dr, we've said that 10.x will be supported until Drupal 12.0, but we didn't say that Drupal 10.0 will be supported until 12.0.

  • Status changed to RTBC 11 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Not a release manager but believe this can be back in RTBC.

  • πŸ‡¬πŸ‡§United Kingdom catch

    fwiw I discussed #7 briefly with GΓ‘bor in slack and we realised that removing updates from 10.0-10.3 from 11.x doesn't preclude updating from Drupal >= 10.3 to 12.0.

    That will depend on the functions added from 10.4/11.0 onwards and when those get removed. e.g. there could be a situation where you can update from 10.6 and 11.4 to 12.0, but not from 10.5 and 11.3. Or it could be that you can update only from 11.4+ to 12.0 - it all depends what gets backported to Drupal 10 minor releases, and what the cut-off point is. This doesn't automatically mean we'll support a Drupal 10-12 update path, but it adds an extra thing to think about in 🌱 [policy, no patch] Remove old update hooks prior to each major version (Drupal 10 and later) Needs review and is unaffected by this issue.

    What this issue will do is prevent direct updates from Drupal 9.5-10.2 to Drupal 11.

  • πŸ‡¦πŸ‡ΊAustralia larowlan πŸ‡¦πŸ‡ΊπŸ.au GMT+10

    +1 from me. We're telling people they can jump from LTS to LTS so we'd expect people to be on 10.4 and stay there until 11.4 (assuming that's the LTS) comes out. This is like how Ubuntu users go from 20.4 to 22.4 and skip the non LTS versions in the middle

  • πŸ‡³πŸ‡ΏNew Zealand quietone

    I agree with this as well. I must admit I am fond of the reduced workload part regarding the dumps.

  • Status changed to Fixed 11 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    I've updated 🌱 [policy, no patch] Remove old update hooks prior to each major version (Drupal 10 and later) Needs review quoting/linking some of the discussion here. Now that this has had an additional +1 from another release manager, going to mark it fixed. Have also opened πŸ“Œ [11.x] [PP-x] Remove updates added up until 10.3.0 from Drupal 11 Active for the implementation issue.

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024