Improve run-tests.sh output when deprecations an warnings are triggered

Created on 12 July 2025, 24 days ago

Problem/Motivation

PHPUnit does not include deprecations nor warnings in the JUnit output.

See Include deprecation/warning failures in JUnit logs

run-tests.sh in the parent issue 🐛 Allow run-tests.sh to report skipped/risky/incomplete PHPUnit-based tests Needs work has been changed to be more adherent to JUnit logs in its report output - and reports as passed runs that throw such deprecations or warnings.

Proposed resolution

Look at exit codes of the PHPUnit CLI processes on top of the JUnit logs to report as failed tests that throw deprecations or warnings.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

📌 Task
Status

Active

Version

11.0 🔥

Component

phpunit

Created by

🇮🇹Italy mondrake 🇮🇹

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mondrake
  • Merge request !12703Closes #3535637 → (Closed) created by mondrake
  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Failed
    24 days ago
    Total: 273s
    #545806
  • Pipeline finished with Failed
    23 days ago
    Total: 685s
    #545809
  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Success
    23 days ago
    Total: 695s
    #545839
  • 🇮🇹Italy mondrake 🇮🇹
  • Haven't had a chance yet to test locally what the junit looks like, but had a question about the need for and placement of the update hook.

  • 🇮🇹Italy mondrake 🇮🇹

    Replied inline. Yes, it's odd. Hopefully one day we will get rid of the test results database and of the --sqlite argument in the run-tests.sh script.

  • OK, tested locally, LGTM.

    11.x

    SIMPLETEST_DB="sqlite://localhost/sites/default/files/db.sqlite" php -d memory_limit=1G core/scripts/run-tests.sh --verbose --sqlite test.sqlite --color --types PHPUnit-Unit Lock
    
    Drupal test run
    
    Drupal Version:   11.3-dev
    PHP Version:      8.3.21
    PHP Binary:       /usr/bin/php
    PHPUnit Version:  11.5.22
    -------------------------------
    
    Tests to be run:
      - Drupal\Tests\Core\Lock\LockBackendAbstractTest
    
    Test run started:
      Tuesday, July 15, 2025 - 09:30
    
    Test summary
    ------------
    
       2.180s Drupal\Tests\Core\Lock\LockBackendAbstractTest                            3 passed, 1 log(s)
    
    Test run duration: 1 sec
    
    Detailed test results
    ---------------------
    
    
    ---- Drupal\Tests\Core\Lock\LockBackendAbstractTest ----
    
    
    Status      Duration Info
    --------------------------------------------------------------------------------------------------------
    Pass          0.033s testWaitFalse
    Pass          1.007s testWaitTrue
    Pass          0.002s testGetLockId
    Log           1.139s *** Process execution output ***
        PHPUnit 11.5.22 by Sebastian Bergmann and contributors.
    
        Runtime:       PHP 8.3.21
        Configuration: /var/www/html/core/phpunit.xml
    
        ...                                                                 3 / 3 (100%)
    
        Time: 00:01.044, Memory: 6.00 MB
    
        Lock Backend Abstract (Drupal\Tests\Core\Lock\LockBackendAbstract)
         ✔ Wait false
         ✔ Wait true
         ✔ Get lock id
    

    After

    SIMPLETEST_DB="sqlite://localhost/sites/default/files/db.sqlite" php -d memory_limit=1G core/scripts/run-tests.sh --verbose --sqlite test.sqlite --color --types PHPUnit-Unit Lock
    
    Drupal test run
    
    Drupal Version:   11.3-dev
    PHP Version:      8.3.21
    PHP Binary:       /usr/bin/php
    PHPUnit Version:  11.5.22
    -------------------------------
    
    Tests to be run:
      - Drupal\Tests\Core\Lock\LockBackendAbstractTest
    
    Test run started:
      Tuesday, July 15, 2025 - 09:29
    
    Test summary
    ------------
    
       2.195s Drupal\Tests\Core\Lock\LockBackendAbstractTest                            3 passed, exit code 1
    
    Test run duration: 1 sec
    
    Detailed test results
    ---------------------
    
    
    ---- Drupal\Tests\Core\Lock\LockBackendAbstractTest ----
    
    
    Status      Duration Info
    --------------------------------------------------------------------------------------------------------
    Pass          0.033s testWaitFalse
    Pass          1.008s testWaitTrue
    Pass          0.002s testGetLockId
    Failure       1.152s *** Process execution output ***
        PHPUnit 11.5.22 by Sebastian Bergmann and contributors.
    
        Runtime:       PHP 8.3.21
        Configuration: /var/www/html/core/phpunit.xml
    
        ...                                                                 3 / 3 (100%)
    
        Time: 00:01.046, Memory: 6.00 MB
    
        Lock Backend Abstract (Drupal\Tests\Core\Lock\LockBackendAbstract)
         ✔ Wait false
         ✔ Wait true
         ✔ Get lock id
    
  • Pipeline finished with Success
    18 days ago
    #551050
  • Pipeline finished with Success
    12 days ago
    Total: 784s
    #555827
  • Pipeline finished with Success
    9 days ago
    Total: 435s
    #557615
    • catch committed 4fd90637 on 11.x
      Issue #3535637 by mondrake, godotislate: Improve run-tests.sh output...
  • 🇬🇧United Kingdom catch

    Thanks for linking 🐛 Allow run-tests.sh to report skipped/risky/incomplete PHPUnit-based tests Needs work because I'd forgotten about that and would have had to go through all the same thought process all over again. Apart from not adding an update at all, and potentially breaking people's local installs, we don't have much choice here.

    This will help prevent a lot of confusion for a lot of people.

    Committed/pushed to 11.x, thanks!

Production build 0.71.5 2024