[PP-1][CI] Spin off Drupal Components tests in a job of their own

Unit tests are falling under linting stage now though?

#5 yes, but not introduced here. See 📌 [CI] Introduce a separate stage for unit tests Active .

Going to go on a limb and say let’s try it

Should we have a report on coverage with the artifacts generated, aside of the output?

#8 I tried to configure coverage reporting, but it does not seem to produce any effect?

Looks like this is still in progress.

With regards to #8, we already have had coverage visualization configured in the MR.
This is producing inline display of the covered line in the merge request diffs. It's not visible in this MR as it does include changes to components code, but an example is here: 📌 Introduce a @return-type-will-be-added annotation to indicate return type addition in next major; enforce with a PHPStan rule Active : in the MR visualization of changes https://git.drupalcode.org/project/drupal/-/merge_requests/10111/diffs#7... you can see that the covered code has a green sideline close to the line numbers.

When it comes to summary metrics, IMHO the standard GitLab solution with adding a coverage key to the job yaml is insufficient, for the following reasons:

1. trivial but annoying: it is not compatible with color output by PHPUnit's coverage text reporting. If you set --colors=always to PHPUnit CLI command, the coverage text report includes ANSI color codes that the regex in coverage cannot cope with.
2. more importantly: the standard approach is limited to reporting line coverage only and averages results from different jobs - this would lead to meaningless figures if we were later to, say, have code coverage for core too as it will make simple average without any weighting etc.

So I will try to take cue from the work in 📌 [CI] Report PHPStan baseline statistics in job Active , and see if with some postprocessing of the coverage text report, we can get a metrics report that can be more tuned to Drupal.

Added a small script that processes PHPUnit's coverage text report into an OpenMetrics file with the relevant metrics included.

See file attached with a screenshot of the resulting report in the MR.

Added a small script that processes PHPUnit's coverage text report into an OpenMetrics file with the relevant metrics included.

See file attached with a screenshot of the resulting report in the MR.

Are the metrics suppose to be clickable?

#15: see https://docs.gitlab.com/ci/testing/metrics_reports/

Apparently, not.

Guess it's still neat to have that info though it would be cool to know what classes aren't being covered. Maybe would start a new initiative :)

But having the info is a net improvement.

#17 that’s just the summary. All the details are visible in the raw logs https://git.drupalcode.org/issue/drupal-3485069/-/jobs/4674359 and as artifacts.

Obviously a code coverage analisys service (e.g. Codecov or the likes of it) would provide full interactive reporting, trends etc but I suppose it’s well beyond the scope of this issue.

The code coverage report is really nice to have here!

It looks like even though there's no concurrency, the number of tests here is low enough that the jobs still finish pretty fast.

However, can you confirm why each job needs 8 CPU when they're only running with single concurrency? I would have thought 1-2 CPU each would be enough.

Huh, did not even look at that, was just a copypasta exercise. Let's see with 2.

No sensible difference, I think the change is small enough to let me self-RTBC this back.

Yep let's do it. Committed/pushed to 11.x, thanks!

Would be good to open an issue to look into exporting/importing these into a visual code coverage report that we can link from somewhere on the project page, or something like that.

catch → closed merge request !10027