- Issue created by @catch
- Status changed to Postponed
28 days ago 12:24pm 22 August 2024 - 🇬🇧United Kingdom catch
What this does:
1. Lowers the CPU request from 24 to 16 for most jobs. The theory behind this is that the total-CPUs-per-machine is more likely to be a multiple of 16 than 24 so theoretically we can fit more jobs on a lower number of machines (or on 16 CPU machines if such a machine exists). I don't fully (or even much) understand the relationship between CPU requests, kubernetes and AWS instances, so this might be flawed, but also in general lower and simpler numbers seems better.
2. Lowers the concurrency of a couple of jobs quite a lot, especially functional tests where I am pretty sure the concurrency in HEAD is leading to CPU contention and hence slower rather than faster test runs. This is made possible by 📌 [PP-1] Order tests by number of public methods to optimize gitlab job times Postponed which removes @group #slow from the vast majority of tests, relying on a better ordering algorithm instead.
3. Increases the parallelism for functional js and functional tests by 1 each. This is because in theory most test runs (in the sandbox branch with various changes applied) can finish within about 2m30s, but we still have a lot of individual tests over 2 minutes each. With lower concurrency, those long running jobs are spread out enough we don't run two slow tests end to end. I'm pretty sure there is potential to bring this lower by continuing to optimise some of these slower tests, but it also gives us a bit of headroom when we add new coverage.
If we look at the jobs, we can see that the overall CPU requirement is reduced dramatically:
Functional JS:
Before: 2 * 24 = 48
After: 3 * 16 = 48Functional:
Before: 7 *24 = 168After: 8*16 = 128
W3 legacy:
Before: 1 * 24 = 24After: 1 * 16 = 16
So an overall reduction of 48 CPUs, with potential scope to reduce further.
- 🇬🇧United Kingdom catch
Found an extra 16 CPU requests to drop on 📌 [PP-1] Order tests by number of public methods to optimize gitlab job times Postponed which brings the total to 64 here.