[CI] Report PHPStan baseline statistics in job

Nice! lets do it. https://git.drupalcode.org/issue/drupal-3499245/-/jobs/3994661

No wait, shouldn't we expose this as a metrics report?

https://docs.gitlab.com/ee/ci/testing/metrics_reports.html

This would make this visible in the gitlab interface.

And an example:

https://gitlab.com/gitlab-org/ci-sample-projects/metrics-reports

File format is extremely easy

Did a. test here: https://git.drupalcode.org/project/deprecated_code/-/merge_requests with a mr to main and 1.x with different metrics.

Few things to note:
Metric changes are based on the target branch of the merge request, which means this will work pretty quite well to show the correct metrics based on the main branches (11.x, 11.1.x etc) without the need of any weirdness or logic.

Seems also to be provable with a MR locally, but since the current job only submits artifacts when the job fails it might need a little more massaging. So lets follow up on that after this issue.

I've checked the reporting thingie, it seems reports are always uploaded so my assumption (and therefor the use of a follow up) is invalid. I'm going to update the mr here to include the metrics for easy viewing.

Nice! If we do this, we need however to have the stats calculated right also (and especially!) for the baseline resulting from changes in the MR. We need to get it right first, though, ✨ [CI] Generate full baseline upon PHPStan failure Active .

@bbrala how will the delta be calculated though? i.e. suppose my MR reduces the baseline errors from x to y: how will we get the info that there was a change from x to y?

I think i mentioned that earlier. It uses the stored metrics from the target branch of the mr as a base and shows the diff compared to that.

Ah thanks, but where are they stored in the target branch? Not as a file in the git repo, right?

As an artifact on gitlab :)

Gotcha. Thanks!