Contrib.social

Ensure run-tests.sh and PHPUnit CLI run with the same list of tests to be executed

Open on Drupal.org →
Created on 29 September 2024, 26 days ago

Problem/Motivation

When running tests with filtering (e.g. by test suite, or by group), run-tests.sh Drupal own runner and PHPUnit CLI runner should ideally discover the same list of test classes to be executed. This is currently not happening, because:

  1. Some core tests classes do not have a name that ends with *Test. These tests are found by run-tests.sh but not by PHPUnit.
  2. Some core tests classes are placed in directories that PHPUnit 10 cannot reach via the current phpunit.xml.dist configuration. These are tests placed in core submodules. Unfortunately, usage of the globstar pattern like <directory>modules/**/tests/src/Unit</directory>

    3. was only introduced in PHPUnit 11, https://github.com/sebastianbergmann/php-file-iterator/pull/83. Until then, the only possibility is to explicitly indicate the path in phpunit.xml.dist.

Steps to reproduce

Proposed resolution

Fix the issues and add a test that continuosly compares the result of PHPUnit and TestDiscovery test discovery lists and fails if they differ.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Active

Version

11.0 🔥

Component

phpunit

Created by

🇮🇹Italy mondrake 🇮🇹

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mondrake
  • Merge request !9684Closes #3477634 → (Open) created by mondrake
  • Pipeline finished with Success
    26 days ago
    Total: 1138s
    #296232
  • Comment 26 days ago →
    🇮🇹Italy mondrake 🇮🇹
  • Comment 15 days ago →
    needs-review-queue-bot

    The Needs Review Queue Bot → tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

  • Comment 15 days ago →
    🇮🇹Italy mondrake 🇮🇹

    rebased

  • Pipeline finished with Failed
    15 days ago
    Total: 141s
    #306608
  • Pipeline finished with Failed
    15 days ago
    Total: 136s
    #306623
  • Pipeline finished with Failed
    15 days ago
    Total: 135s
    #306631
  • Pipeline finished with Failed
    15 days ago
    Total: 628s
    #306677
  • Pipeline finished with Failed
    about 12 hours ago
    Total: 66s
    #320605
  • Pipeline finished with Failed
    about 11 hours ago
    Total: 675s
    #320618
  • Pipeline finished with Success
    about 10 hours ago
    Total: 563s
    #320692
  • Comment about 10 hours ago →
    🇮🇹Italy mondrake 🇮🇹

    Rebased and added <directory>modules/**/tests/src/Build</directory> to build test suite.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024