Allow known flaky tests to be automatically repeated

Created on 14 August 2020, about 5 years ago
Updated 8 August 2025, 17 days ago

Problem/Motivation

#3037436: [random test failure] Make QuickEditIntegrationTest more robust and fail proof and 🐛 [random test failure] Random failures building media library form after uploading image (WidgetUploadTest) Closed: cannot reproduce amongst others are wasting contributor time as they knock RTBC patches back to Needs Work, when in fact it was just a random test failure. These failures are false positives and always go away on a retest.

Steps to reproduce

Mark a patch RTBC and wait a few days/weeks for a random test failure.

Proposed resolution

Provide a way of marking a test as "flaky". If a flaky test fails, automatically repeat it in the test run and only actually fail if it fails the second time round.

Remaining tasks

Decide if this is a good idea.
Figure out how to implement it.

User interface changes

API changes

Data model changes

Release notes snippet

📌 Task
Status

Active

Version

11.0 🔥

Component

phpunit

Created by

🇬🇧United Kingdom longwave UK

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024