Disable DrupalCI testing

Created on 4 January 2024, 12 months ago
Updated 15 July 2024, 5 months ago

Problem/Motivation

With GitLab CI working well, DrupalCI will stop running new tests this summer. Drupal Core already primarily uses GitLab CI testing. DrupalCI is in maintenance mode and any infrastructure issues may not be fixed.

The DrupalCI data retention policy is at https://www.drupal.org/drupalorg/docs/drupal-ci/drupalorg-testing-policy...

The overall goal is to remove the project_issue_file_test module from Drupal.org and decommission dispatcher.drupalci.org

Planned schedule

  • Done
    • Projects without DrupalCI testing cannot add this kind of testing anymore.
    • Projects with DrupalCi testing cannot add new DrupalCI testing configurations anymore.
    • Direct access to log output and artifacts on DrupalCI is no longer available. Results are still summarized on the Automated Testing tab of project pages.
  • Late April 2024: A week or so before DrupalCon, no existing DrupalCI schedules will be able to be updated.
  • July 1, 2024: Running of DrupalCI tests will be shut down. No more tests run after this.
  • Jan 1, 2025: End of DrupalCI test result data retention.

Possible tasks

  • No new testing environments - this is already effectively policy. We will do any updates to ensure accuracy, like updating the PHP 8.3-rc1 label to PHP 8.3 when PHP 8.3.0 was released. PHP 8.4 will not be made available.
  • Remove QA.Drupal.org results - 📌 Remove QA.Drupal.org results Fixed . Issues like #70722: Search results should respect the content type's "Display author and date information." option actually still have results from qa.drupal.org included. As part of removing project_issue_file_test module, these can be removed as they are well past current data retention policies.
  • Projects without DrupalCI can not set it up - 📌 Remove “Automated testing” tab when a project has no DrupalCI testing Active
  • Disable old testing environments - needs evaluation.
  • Discourage patch file uploads - to be planned. When starting to upload a patch to Drupal.org, we should highlight that it is deprecated.
  • No new testing schedules - to be scheduled [30 April 2024] Improve removal of jobs and prevent adding new ones Needs review project maintainers will no longer be able set up new testing schedules to run for issues, on commit, daily, etc.
  • Core label updates - needs evaluation. We may discontinue some updates to core labels for testing.
  • Disable new testing - to be scheduled. Will likely be done by disabling all testing environments.
  • Remove pift module and dispatcher.drupalci.org - 180 days after last test starts according to the data retention policy.
    • There are some UI alterations in pift.extended_file_field.inc and/or pift.nodechanges.inc that should be moved to drupalorg module.
🌱 Plan
Status

Fixed

Component

Policy

Created by

🇺🇸United States drumm NY, US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @drumm
  • 🇺🇸United States drumm NY, US
  • 🇺🇸United States drumm NY, US

    CI errors will no longer automatically requeue with 📌 Reduce the number of attempts to requeue a test on CI Error (retry tests) Fixed deployed.

  • 🇺🇸United States drumm NY, US
  • 🇺🇸United States drumm NY, US

    Adding note about removing pift module:

    There are some UI alterations in pift.extended_file_field.inc and/or pift.nodechanges.inc that should be moved to drupalorg module.

  • 🇺🇸United States xjm

    drumm credited xjm .

  • 🇺🇸United States drumm NY, US

    Another possibility is to disable tests for environments that are not supported. For example, 30% of testing schedules use php5.3_mysql5.5

    We could get rid of some of the D9-only environments. Just keep around PHP 7.4/MySQL 5.7, maybe one PHP 7.4/postgres and 7.4/SQLite, and then delete all the other combos that are not compatible with D10 now.

    D7 only supports PHP 5.6+ officially after the PSA from last year.

    Same for PHP 8.0. So maybe we only need:

    • Environments compatible with D10
    • PHP 7.4/MySQL 5.7
    • PHP 8.0/MySQL 5.7
    • PHP 5.6
    • A few other PHP versions in between

    This would help us narrow down the active usage, cleaning up dormant testing schedules.

    Overall, we should pick a straightforward deprecation schedule so it is easy to communicate. Not all of the possibilities in the issue summary will be used.

  • 🇭🇺Hungary Gábor Hojtsy Hungary

    I think the way we made Drupal 9-only releases unsupported, we can remove testing environments that are not yet supported by current supported Drupal versions. Do you think that would need additional announcement and/or a waiting period before doing it? (I can help draft it but I am not sure it is needed since its likely mostly dormant projects/branches with probably not very attentive maintainers to update testing setups).

  • 🇺🇸United States drumm NY, US

    Adding 📌 Remove “Automated testing” tab when a project has no DrupalCI testing Active , which I see no reason not to do in the next few days.

  • 🇺🇸United States drumm NY, US

    I think we should keep this simple, and only schedule:

    • No new testing schedules
    • Disable DrupalCI testing

    The most official communication about disabling DrupalCI testing is:

    While we don't have a specific date for that yet, it will be retired within a year (possibly by July 2024).

    Let’s go ahead and say July 1st for Disable DrupalCI testing.

    For now new testing schedules, DrupalCon Portland is May 6-9. We might as well pick a date around then.

    We should send emails to people with DrupalCI testing set up. Maybe 2 weeks before each date, so we’re sending at most 2 emails.

    There are currently 2,817 projects with 5,285 testing schedules set up. This has been steadily dropping already.

  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    There are currently 2,817 projects with 5,285 testing schedules set up. This has been steadily dropping already.

    Those are the numbers for DrupalCI, right? How do these numbers compare to the ones for GitLab CI?

  • 🇺🇸United States hestenet Portland, OR 🇺🇸

    Let’s go ahead and say July 1st for Disable DrupalCI testing.

    For no new testing schedules, DrupalCon Portland is May 6-9. We might as well pick a date around then.

    We should send emails to people with DrupalCI testing set up. Maybe 2 weeks before each date, so we’re sending at most 2 emails.

    I agree with this plan.

    For no new testing schedules, DrupalCon Portland is May 6-9. We might as well pick a date around then.

    I would say disable around a week before - that will again encourage folks to keep moving over to GitLabCI at contribution day.

  • 🇺🇸United States drumm NY, US

    I first started looking at DrupalCI abandonment on October 24, 2023 when 3,455 projects had 7,155 DrupalCI schedules. 534 were removed when testing with Drupal 9 was disabled, leaving 104 moving away from DrupalCI on their own in 3 months. I expect many of the remaining projects are minimally maintained and we can't expect their maintainers to put in extra work.

    1,713 projects have used GitLab CI.

  • 🇬🇧United Kingdom jonathan1055

    Regarding task "Discourage patch file uploads" this is likely to cause many more MR issues branches to be created. Contributors who are not familiar with the workflows are also likely to create more than one, unnecessarily. Is there any plan to give module maintainers the permission/ability to delete unmerged branches once an issue is closed? Then we can contribute to tidying up the wasted space used. Or is there no problem with 1000s of repositories and unmerged MR just being left there? I know there has been discussion about not allowing anything to be deleted. This is not an urgent problem, no need for a prompt reply, just asking what the current feeling is.

  • 🇳🇿New Zealand jweowu

    > ... delete unmerged branches once an issue is closed?

    "Issue is closed" isn't a guarantee that users aren't using the branch.

    That also relates to my question:

    Can someone confirm that either (a) a patch file upload workflow will continue to be available under the new system; or (b) the new system provides equivalent functionality: persistent URLs for specific revisions of a patch (regardless of the current state of the MR/branch, and immune to git garbage collection).

    It sounds like it's only the testing of patch files which will be disabled, so presumably we do still get option (a) sans testing; but that being the case, option (b) would be nicer -- otherwise users still have to deal with patch files, but now they have to deal with MRs as well, and there's no obvious connection between the patch URL and the testing that was performed for it.

    Last I knew, the nearest merge request option was a patch URL for "whatever the MR head happens to be at the moment" and users were being told to download local patch files as a workaround (which I've been presuming is a temporary workaround because it's a bit rubbish by comparison). https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... doesn't indicate any improvements in this area.

  • 🇭🇺Hungary Gábor Hojtsy Hungary

    Based on all the info available, I added this planned schedule to the issue summary. Please correct if I misunderstood.

    Planned schedule

    • Done: Projects without DrupalCI testing cannot add this testing anymore. Projects with DrupalCi testing cannot add new DrupalCI testing configurations anymore.
    • Late April 2024: A week or so before DrupalCon, no existing DrupalCI schedules will be able to be updated.
    • July 1, 2024: Running of DrupalCI tests will be shut down. No more tests run after this.
    • Jan 1, 2025: End of DrupalCI test result data retention.
  • 🇺🇸United States hestenet Portland, OR 🇺🇸

    @jweowu - users will still be able to attach accepted file types to issues (even when they move to GitLab) so that may include patch files. They won't have any automated features or testing associated, but they will be attachable.

    However we have *never* recommended hot-linking to patch files on a remote server. That is a really bad/dangerous development practice. We know people have been doing it for years, but it has always been against Drupal.org recommendations.

    It has always been much safer and more reliable to host copies of patches you are using locally.

    There is a feature request (actually multiple feature requests) open requesting GitLab add stable urls for diffs:

    We absolutely agree this is a feature GitLab needs to add. However it won't block the migration, and we still won't recommend hot linking to them in any deployment process.

  • 🇳🇿New Zealand jweowu

    The pairing of a stable patch URL and a checksum of its expected contents is also safe, and significantly more convenient -- developers can safely share reviewed pairings without having to pass around the actual patch files.

    For composer users, version 2 of cweagans/composer-patches supports this out of the box, aborting noisily if it fetches a patch which does not match the checksum specified in the patches.json file.

  • 🇭🇺Hungary Gábor Hojtsy Hungary

    Expanded the timeline's done section with "Direct access to log output and artifacts on DrupalCI is no longer available. Results are still summarized on the Automated Testing tab of project pages."

  • 🇺🇸United States drumm NY, US

    Thanks for the issue summary updates. I also added 📌 Replace “View results on dispatcher” link with deprecation message Fixed since those links now go nowhere.

  • Status changed to RTBC 8 months ago
  • 🇺🇸United States drumm NY, US

    [30 April 2024] Improve removal of jobs and prevent adding new ones Needs review is done, next step is deciding when to send reminders to maintainers still using DrupalCI ahead of July 1st.

  • Status changed to Fixed 6 months ago
  • 🇺🇸United States drumm NY, US

    DrupalCI is now disabled and no new tests will run. Existing test results will be available for 6 months. No DrupalCI test results will be migrated to GitLab, so a project with issues migrated to GitLab will have DrupalCI test results deleted at that time.

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024