drupal_phpunit_find_extension_directories() uses infinite recursion ⇒ more directories = slower tests

The Needs Review Queue Bot → tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

Re-rolled #37 for 9.5.x with the following changes:
1. Fixed $ignore_directories = Settings::get('file_scan_ignore_directories', []); not working as settings has not been initialised yet.
2. Updated deprecation messages to point to 9.6.x

When testing custom modules or themes using Drupal Core's PHPUnit configuration file, core/phpunit.xml.dist, note that the core/tests/bootstrap.php file scans all extension roots (core, modules, themes, etc.), and there is currently no way to opt out of this behavior. While we may want to use Drupal Core's bootstrap file to reuse PHPUnit behavior when testing custom modules or themes within real sites, we may not want to use the same targets.

In practice, the current implementation of core/tests/bootstrap.php can lead to longer test execution times that scale proportionally with the number of installed modules. For instance, if you have a site with 100 contributed modules and one custom module with 1 unit test, running that test will scan through core and 100s of contributed modules to then only run one test in your custom module.

And if you run these tests in a Docker container, the performance degradation can be even worse due to PHP's constant checking for directory and file existence while performing such a lookup.

We've been running the version from [#14292061] with our phpunit.xml referencing a custom bootstrap.php for ~4 years. You just need to update it from time to time as the major phpunit version changes.

@alex.skrypnyk thanks for finding the other issue where I was also involved, or made an attempt to make a related change...

+ $ignore_directories = Settings::get('file_scan_ignore_directories', []);

I did the some, but @Lendude might have a valid point on the other issue

Also, are we really sure we want to re-use an existing setting? Does 'file_scan_ignore_directories' always match the pattern we want to use here? The current use case is to ignore dirs where we look for modules, do we really want to add 'and where we look for tests' to that? Sure, there might be a lot of overlap, but is this always 100%?

(Source [#2943172#comment-13216764])

@larowlan just reminded me of this. Still would be very nice to land this.

P.S.: I forgot I even created this. @larowlan talked about #3050881: Use RecursiveExtensionFilterIterator in drupal_phpunit_find_extension_directories for performance → and then told me it's a duplicate of "mine" — this issue 🤣