Consider merging the build and validate stages

I think it's a great improvement. However, in our case, as the templates are now used by hundreds of projects, dropping something like this could be really disruptive. Core's case is a bit different as it is self-contained, so it won't disrupt other projects.

Doing some searches like this https://git.drupalcode.org/search?group_id=2&scope=blobs&search=composer... show that the "composer-base" and "composer" job are extended heavily and I think it's the right place to add new requirements or packages to projects for all jobs. Probably the only "confusing" part here is that we called the jobs "composer" and "composer-base", even though we are also doing the the "yarn" install lines there, but that's not what this issue is really about.

I agree that it's a good optimization, but I think that it's not one that we can do on the templates, specially now, as it would break existing pipelines. I'll "park" it for the future though, as a possible optimization.

I want to add on from another angle:
GitLab Components,

Generally the concept appears that a GitLab Component should be self contained, with minimal to no usage of external dependencies.

While it appears with could replicate our current setup in components with a central 'build' stage if we were to skip that we could design the components to be more self contained.

I do have a bit of a worry about additional composer jobs given the issues reported in 🐛 Random HTTP timeouts for GitLab CI jobs Active however #3387117: Enable distributed caching in GitLab Runner → could mitigate some of this, or even local runner caching,

Doing some searches like this https://git.drupalcode.org/search?group_id=2&scope=blobs&search=composer... show that the "composer-base" and "composer" job are extended heavily and I think it's the right place to add new requirements or packages to projects for all jobs. Probably the only "confusing" part here is that we called the jobs "composer" and "composer-base", even though we are also doing the the "yarn" install lines there, but that's not what this issue is really about.

What if we kept the same job name, moved it to the lint stage, and then replaced the composer lint job with it. That might be backwards compatible then?

+1 for this. Would @catch's proposal in #5 work, or alternatively, an opt-out? (i.e. if the build fails, make it a one-line addition to opt out of the better new thing and fall back to the old?)

We could defo run things based on a variable (name to be determined) as that's easy enough to trigger or not full jobs (via rules) and to check in the very first part of the script for each job.

Example (pseudocode):

variables:
  NEW_VARIABLE: 1

.run-composer-steps: &run-composer-steps
- do the composer things

composer:
  rules:
    - if NEW_VARIABLE == 0
  script
    - *run-composer-steps

cspell (for example):
  # Drop the "needs" part
  script:
    - if NEW_VARIABLE == 1 OR no-asset-from-composer-found
        *run-composer-steps
      endif
   - rest of the job script

That way we could experiment and turn on/off as needed.

As there is no MR here, the status can be put back to 'active'.
This is so we can more easily identify which issues have been started and really do need work to complete.