Prepare for PHPUnit 10

Unknown option "--fail-on-deprecation"

So we need to detect the PHPUnit version and then apply the options.

@fjgarlin hope you do not mind, I tried with --process-isolation only to see the effect.

... but it looks like I can't trigger the 'Test Key CDN' job.

I don't mind at all, you're very welcome to try. I just triggered the pipeline: https://git.drupalcode.org/issue/gitlab_templates-3444792/-/pipelines/16...

The other option seems to have worked. So, what are the side effects of not having --fail-on-deprecation, does it need to be added somehow too? or is it really not needed?

Well, in core that depends on the value of the SYMFONY_DEPRECATIONS_HELPER environment variable. If that enables the deprecation handler, then unexpected deprecations lead to test failure. If we want to keep in sync with core, we probably need to match that behavior here.

https://project.pages.drupalcode.org/gitlab_templates/jobs/phpunit/#depr...

We don’t do a one to one match with core as it changes between versions so maybe we can document it.

contrib doesn't fail in deprecations by default, so do we really need that?

I agree. We don't currently fail on deprecations, we just have extra documentation on how to match what core does in the page mentioned in #10.
I expanded that file mentioning the new PHPUnit10 option.

The MR is ready: https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/198...
- It adds --process-isolation
- It documents --fail-on-deprecation

I suggest to wait a sec... there are other changes in command line options that will pop-up: for example, the --verbose option is removed, and I think the --printer one too.

https://docs.phpunit.de/en/10.5/textui.html

Happy to wait.

Change record for phpunit version change: https://www.drupal.org/node/3365413 →

Do we need to remove the --no-interaction flag from includes/include.drupalci.main.yml?

I'm getting this output in testing:

PHPUnit 10.5.20 by Sebastian Bergmann and contributors.
Unknown option "--no-interaction"

https://git.drupalcode.org/project/bootstrap_horizontal_tabs/-/jobs/1556...

I didn't find documentation about --no-interaction being removed, but executing vendor/bin/phpunit does not list --no-interaction as an option.

Yeah, it seems that some options are only available in one version and are not supported in the other. We might need to do some conditions to see which options to pass.

The --no-interaction option seems to be tied to the TestDox options, which we don't use by default, so I think that can be removed.

I'd say we should no longer wait as there are tests runs that break because of this.

The MR documents how to get the same as core and also adds options depending on whether we have PHPUnit 9 or 10.

MR: https://git.drupalcode.org/project/gitlab_templates/-/merge_requests/198...

I think the --process-isolation argument can be removed, too, since the final solution in core was to enable process isolation on the base test classes. Maybe this could go in docs as the --fail-on-deprecation argument, but should not be necessary unless in edge cases, truly.

Thanks for letting me know about that. I think that if we don't use it, we don't need to document it. I just removed it.

Based on @fgm's comment on the parent issue, I think this is critical.

#3447105: DB requirements for next major were changed → is also affecting the "next major" tests. I'm planning to ship them one after the other.

After the last change, the job from the downstream pipeline that proves that this is working is here: https://git.drupalcode.org/project/keycdn/-/jobs/1592460

It detects PHPUnit 9 and adds the printer option.

--no-interaction was removed from versions 10 and 11. There is no option documented anymore.
Otherwise looks good.

Merged.

Thanks for these changes, fjgarlin! I can confirm that with this change, contrib module tests on NEXT_MAJOR with PHPUnit 10 are no longer failing due to the --no-interaction parameter.

However, I believe the root path from which PHPUnit expects a path to a file has changed between version 9 and version 10. In my own testing, the following syntax works with PHPUnit 9 but fails with PHPUnit 10:

vendor/bin/phpunit --bootstrap /builds/project/updated/web/core/tests/bootstrap.php web/modules/custom/updated

The test output reports:

Test file "web/modules/custom/updated" not found

Changing the syntax to the following works with PHPUnit 10 (i.e., omit web/):

<code>vendor/bin/phpunit --bootstrap /builds/project/updated/web/core/tests/bootstrap.php modules/custom/updated

Here's a failed run for reference: https://git.drupalcode.org/project/updated/-/jobs/1609612

Maybe either $_WEB_ROOT should be omitted in https://git.drupalcode.org/project/gitlab_templates/-/blob/main/includes... or population of the $_WEB_ROOT environment variable needs to be conditioned?

Thanks for reporting this. We'd normally run multiple tests for all the cases to verify that the changes work in both, but in this case, I wanted to get this issue and the related one over, so we could actually unblock things a bit.

Having a different error now is fine, and I'll continue working on that. I might open a new issue in the "updated" project so I can run pipelines as needed.

Thanks for reporting back and for the details. It helps a lot.

However, I believe the root path from which PHPUnit expects a path to a file has changed between version 9 and version 10.

Actually, this is captured in 🐛 PHPUnit 10 behaves differently when invoked outside web root Needs review , so maybe the relevant change will be in /core/tests/bootstrap.php, not the GitLab execution.

That issue looks really promising and seems like it's where the fix should be. It seems like it doesn't like to be run in a subdirectory (ie: web).

I wonder if, instead of removing $_WEBROOT, we could add the full path instead, like this $PWD/$_WEB_ROOT/....

Following the syntax you posted, can you try the following?
vendor/bin/phpunit --bootstrap /builds/project/updated/web/core/tests/bootstrap.php /builds/project/updated/web/modules/custom/updated

That's a change that we could try here.

fjgarlin → closed merge request !205

fjgarlin → changed the visibility of the branch main to hidden.

maybe the relevant change will be in /core/tests/bootstrap.php, not the GitLab execution.

1. I can confirm that the MR in 🐛 PHPUnit 10 behaves differently when invoked outside web root Needs review does resolve the webroot issue!

Following the syntax you posted, can you try the following?vendor/bin/phpunit --bootstrap /builds/project/updated/web/core/tests/bootstrap.php /builds/project/updated/web/modules/custom/updated

Yes! I can confirm that using an absolute path/full path as indicated above works, and perhaps most importantly, that change works with or without the proposed code change in 🐛 PHPUnit 10 behaves differently when invoked outside web root Needs review ! So, that seems like there isn't any reason not to change the GitLab execution to use the full path...

Great! Thanks for testing and confirming. I'm testing the change on PHPUnit 9 right now https://git.drupalcode.org/project/keycdn/-/pipelines/174401.

If that goes through green, I'll merge so we don't need to wait for the core fix.

Using full path is ok in PHPUnit 9: https://git.drupalcode.org/project/keycdn/-/jobs/1610163
#36 confirms that it also works for PHPUnit 10.

I will deploy the fix shortly.

The new fixes were deployed and the tags were also updated.

Confirming that with the latest commit, tests run via GitLab with NEXT_MAJOR (Drupal 11, PHPUnit 10) are passing! Thanks!

https://git.drupalcode.org/project/updated/-/pipelines/174426

Do I have to update something in my yml file to take advantage of the update? https://git.drupalcode.org/project/forum/-/jobs/1611292

That’s weird. It should really take the new changes as I also updated the defaul-ref tag.

Do you by any chance set the _GITLAB_TEMPLATES_REF variable at project level for that project or MR?

#43 I found re-running a test didn't work but starting the pipeline from scratch did.

That would make sense.

Automatically closed - issue fixed for 2 weeks with no activity.