Improve concurrency: replace experimental package with core script

This is ready for review.

I'm using it already on a contrib module overwriting the gitlab-ci template. See here: https://git.drupalcode.org/project/api/-/blob/2.x/.gitlab-ci.yml#L50

The MR with the changes is here: https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/41/...

Merged! Thanks for the work here.

When CONCURRENCY is used, are the tests running with sqlite regardless of the DB that the user specified. Or is just a results table thats using sqlite? If its the former, we should error out if the specified DB was mysql or postgres IMO.

Also, we are now not honoring _PHPUNIT_EXTRA, and making unavailable all the CLI options that are used with it (--fail-on-warning, --filter, --exclude-group, ...).

IMO we should start with core overriding in run-tests.sh. If we can make that generally useful for contrib projects, it belongs in gitlab_templates. My .02

I added another minor MR with really small changes, mostly text to document the variables and also allow _PHPUNIT_EXTRA even in concurrent mode. The MR is here: https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/42/...

You can ignore the hardcoded "sqlite" argument as that is specific to "run-tests.sh" and is related to the "keep-results" argument. The database for the tests is respected, whether it's mysql, pssql or sqlite.

_PHPUNIT_EXTRA will continue working with default "phpunit" options if _PHPUNIT_CONCURRENT is set to 0 (default value), so there is no change there at all.
If _PHPUNIT_CONCURRENT is set to 1, then the value for _PHPUNIT_EXTRA will be options for the "run-tests.sh" script.

I documented the above in the MR above.

@moshe-weitzman - I'm not sure if you have more feedback here. The goal of the issue was achieved and everything keeps on being under the "experimental" feature.

Are there any specifics that you want to address or discuss? I will mark it as fixed but feel free to re-open, ping, create a follow-up with any thoughts.

In #9 @moshe-weitzman wrote

Also, we are now not honoring _PHPUNIT_EXTRA, and making unavailable all the CLI options that are used with it (--fail-on-warning, --filter, --exclude-group, ...).

In #11 @fjgarlin replied

_PHPUNIT_EXTRA will continue working with default "phpunit" options if _PHPUNIT_CONCURRENT is set to 0 (default value), so there is no change there at all.
If _PHPUNIT_CONCURRENT is set to 1, then the value for _PHPUNIT_EXTRA will be options for the "run-tests.sh" script.
I documented the above in the MR

I've hit this problem, I was using --group to divide the phpunit test into parallel streams, and it was working fine but now fails with

PHP Warning:  Trying to access array offset on value of type null in /builds/project/scheduler/web/core/scripts/run-tests.sh on line 1270
ERROR: Unknown argument '--group'.

To get --group to work and split the job into parallel streams it looks like I will have to set the value of _PHPUNIT_EXTRA varying, depending on the value of _PHPUNIT_CONCURRENT

Yes, that is correct. Apologies.

We found out that leveraging apache and the run-tests.sh script from core we could get better results than using the experimental package "paratest" (which didn't support all options).

We are working with core maintainers to potentially increase even more the concurrency options when this is used with GitlabCI.

Thanks for the feedback.

I have found a difficulty with using the updated template implementation of run-tests.sh, so have raised #3388185: Tests cannot be filtered by @group when running with phpunit_concurrent=1 → to work on it.

Automatically closed - issue fixed for 2 weeks with no activity.