Allow PHPUnit build and Nightwatch jobs to retry once

Linking a related, and still open, issue that also discusses retrying known flakey tests. No decision is in this issue, and the issue predates GitLabCI.

I think this makes sense for now. It’s a small change, if it does not work it can be easily reverted.

In PHPUnit upstream, this is being proposed in #[Retry] attribute to support retrying flaky test #6182. That will allow working on a test class instead of the entire CI job, but surely will only happen in PHPUnit 12 or 13.

Linking related issues. I think a single retry is worth doing for these two jobs since we know they're very flaky. Adding related issues that are potentially why they're flaky. We're most likely to fix nightwatch but removing it entirely, and in that case the job will go. For build tests we should try to make the tests less flaky and remove the retry option again.

I support this change. It will simply do what I do manually to get those tests to pass.

Yep let's do it - we can always revert if we think we've made them generally more reliable, or use #[Retry] when it's available.

Now that this issue is closed, please review the contribution record.

As a contributor, attribute any organization helped you, or if you volunteered your own time.

Maintainers, please credit people who helped resolve this issue.

catch → closed merge request !12922