- Issue created by @catch
- Status changed to Needs review
3 months ago 12:43am 23 August 2024 - 🇬🇧United Kingdom catch
The pipeline fails due to expected versions which I think might be HEAD and I'm not able to trigger a child pipeline (probably due to that, or maybe permissions?).
- 🇬🇧United Kingdom catch
First phpunit run with no real changes (just a comment) was 5 min 26 sec. https://git.drupalcode.org/issue/paragraphs-3469830/-/jobs/2527933
Realised 10.3/10.4 doesn't have test timings in run-tests.sh, so adding opt in next major to see the longest running tests.
- Status changed to RTBC
3 months ago 8:43am 23 August 2024 - 🇪🇸Spain fjgarlin
Triggered downstream pipelines https://git.drupalcode.org/issue/gitlab_templates-3469828/-/pipelines/26...
Based on the mentioned tickets where there is plenty of investigation I think this change makes sense.
- 🇬🇧United Kingdom jonathan1055
I don't know if this is useful, but here is a test on Scheduler using MR250. The PHPunit job is using _concurrent=1 and there are five branches, run in a parallel matrix. The js and kernel tests are separated out already so the benefit might not be seen.
https://git.drupalcode.org/project/scheduler/-/pipelines/264154Here's the same branch, but using ref 'main'
https://git.drupalcode.org/project/scheduler/-/pipelines/264160 - Status changed to Needs work
3 months ago 12:00am 26 August 2024 - 🇬🇧United Kingdom catch
hmm that is useful, it actually makes it slower.
e.g .https://git.drupalcode.org/project/scheduler/-/jobs/2546041 vs https://git.drupalcode.org/project/scheduler/-/jobs/2546071
There is every possibility that the target -> main job was able to use spare CPUs rather than just 2, which means the 32 concurrency might be fine in those scenarios, but ideally we want the results of this to be neutral when runs aren't CPU constrained and faster when they aren't. If it fails to take advantage of extra CPUs when they're available, it's not really an improvement.
Currently when --directory is passed we order by test suite which results in
functional js -> functional -> kernel -> unit. Because scheduler already divides by test suite as you point out, there is no 'running quick tests at the end' effect.
Something like the most optimal ordering would need to expand on the logic in 📌 Order tests by number of public methods to optimize gitlab job times Fixed , so that we first order by test suite, then order by number of methods.
However for example https://git.drupalcode.org/project/scheduler/-/blob/2.x/tests/src/Functi... uses a dataProvider which isn't taken into account by the logic there.
We can't expect all contrib authors to add @group #slow manually to their tests, and don't want to significantly regress existing performance, so should try to automate this a bit more I think.
I just opened 📌 Include a check for data providers when ordering by method count Active against core which would try to account for data providers and run them earlier.
The next step after that would be implementing the test ordering by type and method logic when --directory is used.