Try to optimize test ordering when run-tests.sh is used with a mixture of test types

Created on 25 May 2024, 7 months ago
Updated 11 June 2024, 6 months ago

Problem/Motivation

Core gitlab runs use run-tests for concurrency, on only one test suite at a time (unit, kernel, functional, functional js).

In contrib gitlab runs, concurrent test runs run all groups at once, and this appears to go in the unit, kernel, functional, functional js order.

See the order of initial output on:
https://git.drupalcode.org/project/paragraphs/-/jobs/629259

i.e. If we run functional javascript -> functional -> kernel -> unit instead, we should be able to get overall faster test runs.

This is because functional and functional js tests take dozens or hundreds of times as long to run as unit tests, so if concurrency is maxed out, several unit tests can be finished in the space of one functional test. If a functional test is already running and there are spare threads for unit tests, this time is 'free', but if the unit tests all run first, then you end up with long running tests occupying just a single thread at the end of the job, with other CPUs idle.

Steps to reproduce

Proposed resolution

When multiple tests are run by run-tests.sh, reverse the order of the test groups so that slowest tests run first.


If we only change the default order of test discovery, @group #slow will continue to work to modify the order of tests regardless of type, which seems fine and makes this a very small change.

Remaining tasks

This will have no effect on core test runs, so ideally we'd see how much difference it makes with a contrib module that has a decent test suite like paragraphs.

User interface changes

API changes

Data model changes

Release notes snippet

📌 Task
Status

Fixed

Version

10.2

Component
PHPUnit 

Last updated about 22 hours ago

Created by

🇬🇧United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024