Account for test type, number of methods, and data providers in test ordering

Created on 25 August 2024, 24 days ago
Updated 5 September 2024, 13 days ago

Problem/Motivation

After 📌 [PP-1] Order tests by number of public methods to optimize gitlab job times Postponed , test classes within a test job will be ordered by number of test methods descending.

Data providers mean that the number of test method runs diverges from the number of test methods, so we currently have to use @group #slow to identify most tests with data providers (at least ones with significant setup tasks) so that they run first.

If we did some additional reflection work, we could detect the @dataProvider annotation, and add a 'bonus' to the method count, like +5 or +10 or something. Doing this would allow @group #slow to be removed from those tests which only have @group #slow due to a data provider, leaving just the ones which really are genuinely slow to run (Umami installs et al).

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

📌 Task
Status

RTBC

Version

11.0 🔥

Component
PHPUnit  →

Last updated about 4 hours ago

Created by

🇬🇧United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024