Thought we already had an meta issue for this, but can't find one. Gitlab has allowed us to get core pipeline runtimes down to ~8 minutes wall time, but we still have more to do:
1. A single long running test, like a functional test using a data provider with 15+ items or so, can take 5-6 minutes just by itself, singlehandedly taking pipeline times closer to 10-15 minutes and tying up a runner. This is very easy to do - it's just using the phpunit API to organise a test logically, but our functional and functional javascript tests are not normal phpunit tests.
2. The fast wall times rely on a lot of parallelisation, core pipelines must use approx 20 pods or more. So compute time is still significant.
3. iirc, the gitlab cluster still relies on very beefy aws instances, this likely increases costs on top of the compute time. The more we can do to make things efficient, the more flexibility there is to use smaller/cheaper test runners.
4. Similar to #1, we have a lot of individual tests that take 3-4 minutes to run each, this results in a floor where while we might be able to tweak pipelines down to about 7m (I think the fastest run so far was about 7m20 but things have regressed a bit recently), getting things significantly under 7 minutes will be nearly impossible without significant breakthroughs.
🌱 [META] Convert some tests into Kernel or Unit tests Active is maybe the meta-issue you were thinking of?
Ah yeah that might have been it, added as a child issue of this one since this includes gitlab tweaks and other things too.