Update templates so 11.0 is the default/current branch

Makes sense to me to postpone this for a few weeks. We could pick a half-way point between 11.0 and 11.1 like October 1st maybe?

I agree with this. I kept thinking about it when we first talked about it and I agree that it'd be more disruptive than helpful to do the jump straight away, so waiting a few weeks seems easy enough and sensible at the same time.

There is an issue about LTS here #3444714: Support LTS version of Drupal → . I don't know if it will be needed or if, once we jump, we keep the previous major to the LTS version.

Yes there is really no benefit at all in pushing quickly for D11 to become the default test version. We have a good infrastructure for testing against D11 now, which is opt-in and non-disruptive. There are many of the D11 Automatic Bot issues and MRs which are being actively worked on. Also those issue forks are used for testing Next Major when that module is a test dependency. See 📌 Gitlab PHPunit job at Next Major needs D11-compatible test dependencies Fixed where we have a replacement temporary composer.json for Next Major which adds the repos and forks of the D11 issues. It is an interim solution as things are in flux with other modules, but that is OK because its not the 'current' branch test.

At some point we could ease in to encourage testing at D11 by simply switching the default for OPT_IN_TEST_NEXT_MAJOR to 1 as a temporary measure during the transition. Then we set it back to 0 when D11 becomes current and we have the 'real' next major.

At some point we could ease in to encourage testing at D11 by simply switching the default for OPT_IN_TEST_NEXT_MAJOR to 1 as a temporary measure during the transition. Then we set it back to 0 when D11 becomes current and we have the 'real' next major.

This also seems like a great idea - maybe a couple of weeks after 11.0.0 for that one?

To confirm:
The plan would be to update all the variables when 11.x is released and separately (in this issue) change DRUPAL_CORE for all the Composer jobs correct?

While the suggestion could be less disruptive to the community it also feels like a break in our job API promise to me. Next Major is promised to be a next major, current is promised to be current major and previous major is previous major.

This proposal feels like it is trying to fix a fault in our core release timing (too short of a time sync between API lock in Beta1 and Stable release) or an issue with inactive maintainers by delaying updating testing.

IIRC we did have at least one incident (might have been D7 related) where Core threw errors because it was not latest that did break tests when it was not expected.

If a contrib module did a similar check to make sure that the current major release is actually the current major (maybe the module feared that we would updating and added their own defense) this could force maintainers to make a manual change to their CI files.

I will also note that until #3458238: CI installs wrong Drupal version → lands we do not always respect the DRUPAL_CORE target value (although in this case it may not pose a significant issue, we may just end up testing D10 and D11 with the “current major” job depending upon the project).

The following suggestion is not a fix today unless we open up a 2.x branch of templates however maybe instead our “current major” job should be replaced with a “max dependencies” job where we let composer find the latest compatible versions, including core (be it D9/D10/D11) and just run from there.

Yes, I bet that we will need to update some of the variables in one batch, when new versions are available (the check_versions job might even start failing if we don't) and then use this issue to "promote" or "push" all the testing so that "current" is D11 and "previous major" is D10.

We are getting the failure now as Drupal 11.0.0 is out there: https://git.drupalcode.org/project/gitlab_templates/-/jobs/2311261

We will ignore these for now until we change the values. I'm not sure that we should do something for the "check_versions.php" script in the interim.

I've put together the changes for when we do the switch. It's in https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/242

Most notably current versions will be 11-based, previous versions 10-based. Anything next is still 11.x-based as that is the equivalent of "main".
That also means that we are no longer testing anything D9 unless maintainers change the variables themselves (which is totally possible).

Still to fix:
- PHP versions (minimum 8.2 for 11.0.x) 🟢
- SQLlite version ( #3447792: See if we need to raise sqlite version for Drupal 11 → ) 🔴
- Downstream projects should be green:
-- API: 🟢
-- Decoupled pages: 🔴 (only one nightwatch test)
-- Key CDN: 🟢

Updated title and description slightly.

MR needs review in any case.

We will ignore these for now until we change the values. I'm not sure that we should do something for the "check_versions.php" script in the interim

Per comments #6/#7 I though the plan was to update the values today and use this issue to “roll back the impact” to tests by overriding core target per job.

Updated summary with todo tasks.

We need to have all the dowstream projects sorted before rolling out any changes. What I meant in #7 was that we might need to update some variables if something was really broken, but it's "just" our script to check_versions, nothing else. So that leaves this issue to take care of all the rest.

I've updated the issue description with things to do in this issue before it is merged.

Updated TODO.

Updated completed tasks.

That also means that we are no longer testing anything D9 unless maintainers change the variables themselves (which is totally possible).

This alone is a reason to wait -- most modules should still be supporting Drupal 9.5 (~33% of D8+ sites are still on D9 as of Aug 1st 2024), and dropping test coverage means that seemingly innocuous changes (looking at you, phpcs issues) can break these sites and there would be no warning to the maintainer.

I could write a whole blog post about this -- but generally, IMHO modules (and thus the testing suite) should support three major versions of Drupal. Its better for the modules and testing suites to be -behind- the adoption curve to ensure stability and upgradability for users. Especially since those on older versions are likely larger, more complex sites. As a community we should strive to make the upgrade process easier, which means supporting a wider set of versions. That said, for Drupal 11 readiness, dropping Drupal 8.9 support made sense as very few sites were still on it, and there comes a point where old support should just end.

Brainstorming out loud here: The question then becomes naming and timing. For the next year we'll have core support like this:

Aug '24: 10.2, 10.3, 10.4, 11.0, 11.1
Q4 '24: 10.3, 10.4, 10.5, 11.0, 11.1, 11.2
Q2 '25: 10.4, 10.5, 11.1, 11.2, 11.3
Q4 '25: 10.5, 11.2, 11.3, 11.4, 12.0

Looking at this schedule, I think it makes sense that 9.5 remains in the testing mix until Q4 '25, which would be when D12 is starting work.

However this also introduces a lot of testing scenarios, and thus once Drupal 10.x goes into LTS phase, I don't think we need to test all of these versions. Once 10.4 comes out as the LTS release, we can probably reduce our D10 window from 10.2, 10.3, 10.4 to simply 10.4. THIS is when Drupal 11 should become the current version of Drupal.

Therefore, I propose there are 3 times where a job pipeline change should occur in the test scenarios:

• n.1 release:
  This is kickoff for the lifecycle of the current major version of Drupal. This corresponds with the previous release entering LTS phase.
  --> NEXT_MAJOR_MINOR (11.1) to CURRENT
  --> NEXT_MAJOR (11.0) to PREVIOUS_MINOR,
  --> NEXT_MINOR (10.4) to PREVIOUS_MAJOR,
  --> PREVIOUS_MAJOR (9.5) to PREVIOUS_LTS.
  The NEXT_MAJOR and NEXT_MAJOR_MINOR pipelines get disabled.
• n+1 alpha release:
  This phase is when we're running readiness for the next version. At that time, PREVIOUS_LTS tag gets disabled and NEXT_MAJOR tag gets enabled with the next version of Drupal.
• n+1 .0 release
  The only change needed when a new major release occurs is the addition of NEXT_MAJOR_MINOR. This allows projects to test against 11.1, but still allowed to fail to support contrib module catch-up.

Updated issue description.

--

Re #18. We are waiting, and we are trying to get the best strategy and to get all the dependencies in place first. Once that's done, we will offer information so maintainers can decide what to do.

We have been running during all these months D9 + D10 (current) + D11. The shift, when it happens, would still cover Drupal 10, until Drupal 13 is released. For example, a lot of maintainers were running "current" + "next major", and when the shift happens, "current" + "previous major" would be equivalent.

Drupal 9 is EOL since December 2023. There will be options to continue testing "old" versions if desired, as we are not limiting anything. We are providing recommended defaults. You can pin the version of the templates that you are using to a static version that won't change, you can override any part of the template, you can create variants easily, so I think that there will be options suitable for the different cases.

📌 Automated Drupal 11 compatibility fixes for decoupled_pages RTBC needs to be merged. I RTBCed the issue and pinged the maintainers via slack.

I've tested this MR with Scheduler and it basically runs, but I have some questions. For ease of viewing, the version info can be seen at the end of the composer jobs.

1. For current, max php and next minor we get 11.0.x-dev a17b3d3 but for Next Major we have 11.x-dev c03b98e. Is think this difference is expected, is that correct?
2. If #3426289: Cater for empty next/previous minor/major when it's not possible → were done it would have a beneficial effect, because 'next minor' testing is a waste of resource right now
3. Previous Major is 10.2.7 but Previous Minor is 10.2.2 (older than previous major). I guess this is also a variant which is not applicable immediately after the switch to D11

I have separated out the TODO tasks into technical work and actions. These can be expanded as and when, and a timetable worked out. But for now we are concentrating on the technical work. I will test the deprecated reference next.

Fix list syntax. The "preview" does not work properly and does not show this type of formatting error.

Re #21.

1. Yes, those values seem correct.
2. Agree that doing this as well would be ideal to avoid the cases in 1, where three variants run the same thing.
3. Right now we are using the last (previous major) security release as opposed to the last (previous major) stable release. Happy to change this as we also have the LTS issue where the last (previous major) security release might make more sense.

Thanks for tidying up the issue description and for the additional testing.

As discussed I have added a check for $CORE_PREVIOUS_STABLE into check-versions.php. I also reverted the value back to the wrong one we found, so the pipeline job should fail.

As expected, the job failed https://git.drupalcode.org/issue/gitlab_templates-3463894/-/jobs/2369415 showing that $CORE_PREVIOUS_STABLE 10.2.7 should be 10.3.1

Setting to NR as, in addition to the code changes, we now have a draft-announcement.md" class="!text-blue-400 !no-underline" target="_blank" rel="nofollow" >draft announcement →

I have cerated MR 246 for step 1 of the process, to change the default of OPT_IN_TEST_NEXT_MAJOR to 1 temporarily.

📌 Automated Drupal 11 compatibility fixes for decoupled_pages RTBC deployed.

#3447792: Add _TARGET_PHP_IMAGE_VARIANT to cater for newer sqlite versions for Drupal 11 → (and ideally #3458238: CI installs wrong Drupal version → ) to go.

#3447792: Add _TARGET_PHP_IMAGE_VARIANT to cater for newer sqlite versions for Drupal 11 → merged. This was the last technical blocker.

#3458238: CI installs wrong Drupal version → is still under discussion, but it's not a blocker.

We will review the announcement blog post this week and proceed with https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/246

Updated the issue description with the current situation.

I commented on https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/242... but not sure if you see it as the thread is closed. The change to allow 'Composer Next Major' to fail should be in MR 246 and then reverted in MR 242. I can make this change if you confirm that I've understood it correctly.

You are right @jonathan1055, the change was meant to go in the other MR, and I just added it. I don't think it'll need to be reverted, as we were already allowing failure in PHPUnit and nightwatch jobs for the "next major", so this way it'll be more consistent. The error will show, but the overall pipeline result won't fail if any of the future jobs ("next major") fail.

I don't think it'll need to be reverted, as we were already allowing failure in PHPUnit and nightwatch jobs for the "next major", so this way it'll be more consistent.

You are right that it is "more consistent" but it also means that if there is a failiure in the Composer Next Major then the PHPUnit and Nightwatch Next Major jobs will still be attempted (even though they will almost certainly fail because the Composer Next Major failed). I think we left 'Composer Next Major' without being allowed to fail so that resources are not wasted in running the later jobs. Also, if the Composer job is allowed to fail then attention is taken away from it, and devs try to work out why the phpunit job is failing (I did this and wasted some time on it, only to realise that the problem was the Composer job instead).

So it's not clear cut which scenario is better. Happy to go with your preference, but just pointing out the other side of the argument.

Can we test in a pipeline what happens if the "composer (next major)" job fails, whether the "phpunit (next major)" and/or "nightwatch (next major)" run at all? I'd assume that they do not run (thus not wasted resources) but we'll clear any doubt with a test for this case.

I tested with this:

test a:
  stage: test
  allow_failure: true
  script:
    - exit 1

test b:
  stage: test
  allow_failure: true
  needs:
    - "test a"
  script:
    - exit 0

See the result here: https://git.drupalcode.org/project/gitlab_templates/-/pipelines/260356

It does seem to run. So yeah, let's add a note to revert this change.

Nice test :-) That's what happened to me when I started testing at Next Major. The phpunit job ran, failed, and I spent some time trying to debug it, not realising that the fault was in Composer Next Major. Even though it is Amber, your attention is drawn to the red phpunit jobs. If there was a way to tell a job that its needs: pre-requisite has to end with 0 before it can run, that would be good. Then we could allow the Composer job to fail.

I think that's the difference between needs and dependencies but I'm not sure. Maybe something to play with in another issue.

https://www.drupal.org/drupalorg/blog/gitlab-ci-templates-will-make-drup... → is now published.

I added extra logic to allow for empty variables on certain variants as a result of #3426289: Cater for empty next/previous minor/major when it's not possible → , which will be committed soon.

There are four variables that can be blank, you catered for three of them, but missed CORE_PREVIOUS_STABLE, so I added that. I removed the exception that allowed CORE_NEXT_MINOR to be the same as CORE_SUPPORTED. This is not needed now that CORE_NEXT_MINOR can be blank instead. Also did some more improvements to the script and the test values.

Thanks for the additions to the script @jonathan1055.

--

As per https://www.drupal.org/drupalorg/blog/gitlab-ci-templates-will-make-drup... →
- Merged MR246 (Temporarily turn OPT_IN_TEST_NEXT_MAJOR on for all contrib)
- I will release tag 1.5.6 later today.

#3458238: CI installs wrong Drupal version → could be merged after it is RTBC.

#3462681: Add W3C compliant JS testing → should ideally be sorted before we do the final switch.

Great news that we have completed step 1. I've had some more thoughts on the timing and plan for steps 2 and 3, and even though I know expected dates have been published in the blog, I thought I would at least share my ideas in case we want to revise them.

We currently have flexibility in when to commit step 2, but once that is done the clock is ticking on the gap between that and step 3 because step 3 only involves moving the default-ref tag. In practice this means we cannot make any other important bug fix and release it to all Contrib during this time without step 3 also being bundled in with that change. It would therefore be beneficial to minimise the chances of major bugs being found after step 2 is completed, which suggests that we allow more time in the step 1 phase (where we are now) before commiting step 2. We may see an upturn in support requests in the next few days now that all Contrib is testing against Drupal 11. This may involve making some fixes to the templates, or at minimum it may take time and effort away from preparing for step 2.

The current planned date is Thursday 5th September (7 working days from today). Maybe we should put this back a bit more, say another week? This would give more time for Contrib to find any problems in the templates at Next Major, give us time to resolve them and release 1.5.7 say. It would also allow us to finish #3458238: CI installs wrong Drupal version → and #3462681: Add W3C compliant JS testing → before step 2, both of which would be beneficial.

Agree. That's why we used the word "expected".

It'll depend on whether new issues/bugs are reported in the next week or two. But yeah, I think that these two are now requirements before Step 2 happens:
- #3458238: CI installs wrong Drupal version → (nearly there)
- #3462681: Add W3C compliant JS testing → (not there at all, and this one is more crucial)

I'd say let's focus on the above two first, watch the queue, and play it by ear. We have some flexibility in the dates and we don't need to force things, but also, if everything goes well, we can also stick to the plan.

I updated the issue description with the above issues.

Yes that sounds like a good plan.

Regarding the blog post, does anything have to be done/requested for it to be inclued in the weekly e-mailing of the Drupal Newsletter? I don't know if they are sent out all at the same time, but mine last was Thursday 22 August, (one day after the blog post was published) and it did not contain anything about it. The next would be Thurday 29th, and it would be good to have this included.

I made a request to our marketing team to see what they can do about it. We don't control "The Weekly Drop" but we sometimes send them blog posts for consideration. "The Drop Times" already posted about it.

Your request worked, it was included in "The Weekly Drop" mailed today
https://mailchi.mp/drupal/20240829?e=2de42d4404

One of the blockers was committed now. Updating description.

The other blocker is ready for review.