Contrib.social

Prevent setup tasks running for tests that are always skipped

Open on Drupal.org →
Created on 2 April 2025, 25 days ago

Problem/Motivation

There are many tests in core that are skipped unconditionally due to random fails or other todo items. Quite a few of these tests still perform expensive setup tasks. The list below shows the worst offenders, ordered by how long they took to run locally for me. This adds up to around 4.5 minutes of setup only to skip the tests.

core/tests/Drupal/FunctionalJavascriptTests/Core/Field/TimestampFormatterWithTimeDiffTest.php # 47.699
core/modules/system/tests/src/Functional/UpdateSystem/UpdateScriptTest.php # 44.790
core/modules/ckeditor5/tests/src/FunctionalJavascript/MediaLibraryTest.php # 31.519
core/modules/layout_builder/tests/src/FunctionalJavascript/LayoutBuilderUiTest.php # 29.927
core/modules/layout_builder/tests/src/FunctionalJavascript/MoveBlockFormTest.php # 21.947
core/modules/settings_tray/tests/src/FunctionalJavascript/SettingsTrayBlockFormTest.php # 20.983
core/modules/system/tests/src/Functional/System/ThemeTest.php # 19.795
core/modules/system/tests/src/Functional/Module/DependencyTest.php # 13.397
core/modules/announcements_feed/tests/src/FunctionalJavascript/AlertsJsonFeedTest.php # 12.314
core/tests/Drupal/FunctionalJavascriptTests/Ajax/AjaxTest.php # 11.762
core/modules/announcements_feed/tests/src/Kernel/AnnounceFetcherTest.php # 10.616

Steps to reproduce

Grep for markTestSkipped and run each test where the skip is unconditional

Proposed resolution

Simple approach is to skip the test in the setUp function. A better approach might be to introduce an attribute that can be put on the test method. That way it would be reusable, and it would still be obvious when looking at only the method that it should be skipped. Note that phpunit already has some attributes for this, but they are not quite suitable for these cases.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

📌 Task
Status

Active

Version

11.0 🔥

Component

phpunit

Created by

🇦🇺Australia mstrelan

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mstrelan
  • Merge request !11706Skip tests earlier → (Open) created by mstrelan
  • Pipeline finished with Failed
    25 days ago
    Total: 182s
    #462963
  • Comment 25 days ago →
    🇺🇸United States nicxvan

    I think this is great, do we need something more generic long term?

  • Comment 25 days ago →
    🇦🇺Australia mstrelan

    I think this is great, do we need something more generic long term?

    Yeah the attribute I mentioned in the proposed resolution would be good to investigate. We can probably make use of the native phpunit attribtues for tests that are skipped on requirements like php extensions. Might be good to see if we can do another attribute for skipping on certain database driver too. But long term any permanently skipped tests should be fixed :joy:

  • Pipeline finished with Success
    25 days ago
    Total: 551s
    #462965
  • Comment 22 days ago →
    🇺🇸United States smustgrave

    This seems like a great idea, I opened 📌 Investigate adding phpunit attribute for skipping test Active for discussing the use of an attribute but don't think it should hold up this issue.

    Thanks!

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024