Test-only job does not detect failures correctly

Created on 19 December 2023, about 1 year ago
Updated 10 July 2024, 7 months ago

Problem/Motivation

Test-only job sometimes report as successfull even if there are failures in tests, see:

https://git.drupalcode.org/project/drupal/-/jobs/506479

There were 4 errors:
1) Drupal\Tests\system\Functional\Pager\PagerTest::testActiveClass
Error: Class "Drupal\Core\Cache\MemoryCache\MemoryCacheFactory" not found
...
There was 1 failure:
1) Drupal\Tests\user\Kernel\UserPassRehashTest::testUniqueHashNoPasswordValue
...

And still:

Job succeeded

From the Slack discussion with @fjgarlin:

test-only tests are run using phpunit, whereas the whole suite uses run-tests.sh, so that might explain why phpunit is not catching the exception.we have some try/catch blocks on run-tests.sh and on some test base classes that might capture that and not reported

Steps to reproduce

See: https://git.drupalcode.org/project/drupal/-/jobs/506479

Proposed resolution

Fix the error checking.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

๐Ÿ› Bug report
Status

Fixed

Version

10.3 โœจ

Component
PHPUnitย  โ†’

Last updated about 11 hours ago

Created by

๐Ÿ‡ธ๐Ÿ‡ฐSlovakia poker10

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024