Enable distributed caching in GitLab Runner

I've put together an internal MR to enable this. It's based on the setup suggested here https://docs.gitlab.com/runner/configuration/advanced-configuration.html... and using "iam" authentication.

Ran into this too: https://drupal.slack.com/archives/CGKLP028K/p1695219548515259

This can deliver a nice boost in some scenarios! 😊

Bump — this has the potential to help with

11.5 GiB Project Storage

— https://git.drupalcode.org/project/big_pipe_sessionless
🤯11.5 GB!!! 👆 for a trivial module, that is being tested against 9.5, 10.0, 10.1 ,10.2 and 11.0.
This is because of a nightly scheduled build that has to do multiple composer builds, each of which takes ~100 MB: https://git.drupalcode.org/project/big_pipe_sessionless/-/artifacts.

If we extrapolate this to 100s or even 1000s of contrib modules doing multi-version testing (which they should!), then this is going to quickly become untenably expensive AFAICT? (/cc @hestenet (he/him)). Can we automatically clear out all artifacts from all previous scheduled runs, at least the ones that were successful? That’d make a huge difference I think?

— https://drupal.slack.com/archives/CGKLP028K/p1698398728821119

I bet even start using local runner's cache can speed-up pipelines

Simplest approach is just add fallback cache https://docs.gitlab.com/ee/ci/caching/#per-cache-fallback-keys
Then tune-up push/pull policy https://docs.gitlab.com/ee/ci/caching/#use-a-variable-to-control-a-jobs-...

Alternatively, cspell for example (7.5MB) can use artifacts via https://docs.gitlab.com/ee/ci/jobs/job_artifacts.html#with-a-cicd-job-token

Filed new issue as it's the blocker for new infra #3449463: Update DB images to prevent server restart →

Do we have an ETA for this? 🤞

This could AFAICT massively speed up CI runs, and reduce infrastructure costs!

If we don’t implement 🐛 Random HTTP timeouts for GitLab CI jobs Active this issue would be helpful to avoid failing jobs (at an expense of increase in S3 storage usage)

Cache is not a replacement for artifacts, this won’t fix the storage issue mentioned above but can help decrease runtimes by copying some temporary cache files (like phpstan analysis cache) though that will not help
issue forks (cache is per project)

If this doesn't help issue forks, do we want to look at moving phpstan analysis caches into artifacts somehow?