Contrib.social

Make run-tests.sh --directory testable

Open on Drupal.org →
Created on 20 April 2017, over 7 years ago
Updated 23 January 2023, almost 2 years ago

Problem/Motivation

#2869120: run-tests.sh ignores classes if they have whitespace before the declaration using --directory → was committed without a test. This means we can't discover whether future changes will be regressions.

TestDiscovery already has a scanDirectory() method which has very few dependencies, but which requires that we have already built a class loader namespace map, which we don't have at that level of run-tests.sh, and might be something we're trying to avoid during the --directory scan.

Proposed resolution

Move the code block dealing with --directory to a class named something like DirectoryTestFinder.

This code can then be unit tested using vfsStream or other mocking, according to the many test cases written in that code block's inline comments:

      // Extract test case class names from specified directory.
      // Find all tests in the PSR-X structure; Drupal\$extension\Tests\*.php
      // Since we do not want to hard-code too many structural file/directory
      // assumptions about PSR-0/4 files and directories, we check for the
      // minimal conditions only; i.e., a '*.php' file that has '/Tests/' in
      // its path.
      // Ignore anything from third party vendors.

        // '/Tests/' can be contained anywhere in the file's path (there can be
        // sub-directories below /Tests), but must be contained literally.
        // Case-insensitive to match all Simpletest and PHPUnit tests:
        // ./lib/Drupal/foo/Tests/Bar/Baz.php
        // ./foo/src/Tests/Bar/Baz.php
        // ./foo/tests/Drupal/foo/Tests/FooTest.php
        // ./foo/tests/src/FooTest.php

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

9.5

Component
Simpletest  →

Last updated 2 days ago

Created by

🇺🇸United States mile23 Seattle, WA

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment almost 2 years ago →
    🇺🇸United States mile23 Seattle, WA

    The patch in #11 no longer applies. The reason is that Drupal core now uses this line to do the directory discovery, after #3035312: Move file_scan_directory() to file_system service → :

    foreach (\Drupal::service('file_system')->scanDirectory($directory, '/\.php$/', $ignore) as $file) {

    That is, the system under test is discovering the tests.

    That means that while there's probably test coverage for the file_system service, it also means that the test coverage for the file_system service is potentially discovered by the file_system service.

    Therefore it's impossible to ascertain whether the goal of test coverage for run-test's --directory feature is met, though we could probably just go on assuming our quality is high without any data, which seems to be The Drupal Way.

    It's also unclear whether anyone really wants to go about gathering the data which would help us understand the quality of the product, so I'm going to set this issue as Active. If anyone wants to champion this cause they can pick it up.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024