CoreRecipesTest is slow

Uploading where I got to as a patch just for documentation of what didn't work, but pretty sure that's not the way to do this, however no idea what recipes shipping tests would look like either.

Can we do something more like GenericTest where we add a test for every recipe, but where that test is a minimal subclass of a recipe test base class?

We were hoping to eventually do something like this, but it wasn't a priority. But since core's test performance is being impacted, I guess this just became more important.

I would propose the following conventions and rules for recipe tests:

• They can be kernel, functional, or functional JavaScript tests. (I can't think of any way that recipes would benefit from unit tests.)
• They don't need a @group annotation.
• Core recipes are in the Drupal\(FunctionalTests|KernelTests|FunctionalJavaScripts)\Core\Recipe\recipe_name namespace.
• Non-core recipes are in the Drupal\Tests\Recipe\recipe_name\(Functional|Kernel|FunctionalJavascript) namespace (this is for another issue to decide; I'm not adding support for non-core recipes in this issue right now).

Looked at https://git.drupalcode.org/issue/drupal-3462383/-/pipelines/228183 (the last green test run)

First functional tests job took 3m41s - so that's 50 seconds less. Obviously the time is just distributed amongst the runners more evenly, but splitting tests like this should mean we use the runners more efficiently (i.e. minimises the time at the end of a job where just one test is still running) as well as letting the entire pipeline return quicker (the restriction on total pipeline runs is determined by the single slowest test, at least unless we significantly speed up installs).

https://git.drupalcode.org/issue/drupal-3462383/-/jobs/2162836

Left a couple of nits on the MR but it looks great to me.

I am not close enough to the recipes system to have a strong opinion on where and how to ship tests with recipes so will leave that bit for someone more involved - it seems like a good idea to support it to me but that is as far as I get.

The Needs Review Queue Bot → tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

It's with sad regret about test autoloading that I rtbc this. Nice work @phenaproxima within the confines of our system.

catch → credited longwave → .

Committed/pushed to 11.x and 11.0.x, thanks!

This doesn't cherry-pick to 10.4.x, but I'm not sure how much we're going to be changing recipes or their tests in 10.4.x anyway so maybe that's fine? Marking fixed but not opposed to a backport if people want to.

catch → closed merge request !8823

Automatically closed - issue fixed for 2 weeks with no activity.

This is registering functional tests as unit tests, opened 🐛 Generic recipe functional tests ar discovered as a unit test Active .

Tried a quick MR on 🐛 Generic recipe functional tests ar discovered as a unit test Active but it will need a bit more work. Re-opening this because I've reverted it from 11.0.x for now - we're going to need to change the namespace the classes are registered in so it's a test API change. Will need an 11.0.x release note although probably won't affect any contrib recipes because it's only been in 11.0.x for a week or so.

Follow-up MR up for review.

Seems to have gone through rounds of review

The change to core/lib/Drupal/Core/Test/TestDiscovery.php seems straight forward enough.

Going to mark it.

Committed ccca0ab and pushed to 11.x. Thanks!

Marking PTBP as this needs a new MR for 11.0.x.

I think 🐛 Generic recipe functional tests ar discovered as a unit test Active can be closed as duplicate.

Backport is up - just had to cherry-pick two commits from the 11.x MR without the additional revert in 11.0.x because that was already in HEAD.

Went ahead and committed/pushed the backport to 11.0.x since there were no changes from 11.x except the sequence of commits.

catch → closed merge request !9203

Automatically closed - issue fixed for 2 weeks with no activity.