TestBot returns with "Build Successful" when there are too many test failures to report

Created on 15 April 2021, over 3 years ago
Updated 26 January 2023, almost 2 years ago

Problem/Motivation

At the moment, when TestBot discovers too many test failures to create a report for, it returns with the message `Build Successful`.
Example, see: #3157938-21: Use t() for #title and #description in tests and test modules β†’

Although this message is in grey instead of green, it gives a message of positivity (Succesfull) and people interpreted it as "Yay, all went well".

You have to dig deep (read: look at the actual console output from Jenkins, in the above example: https://dispatcher.drupalci.org/job/drupal_patches/80985/console) to see that all is not well, in fact it's very, very wrong.

Steps to reproduce

Create a patch with many test failures on TestBot.
I'm not exactly sure what the threshold is to not get the "normal" red x pass, n failed return from TestBot.

Proposed resolution

- Replace the positive text with something more suitable, like `Too many errors, the end is nigh!` or something alike.
- Make the message red instead of grey to alert users something bad happened whilst testing.

Remaining tasks

User interface changes

API changes

Data model changes

πŸ› Bug report
Status

Active

Component

Artifacts and Output

Created by

πŸ‡³πŸ‡±Netherlands spokje

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024