[experiment] Run Unit tests with paratest

Created on 27 September 2024, 6 months ago

Problem/Motivation

An issue for using paratest exists already, 📌 [experiment] Explore paratest to run our phpunit tests in parallel Needs work

Here a sandbox to keep attempts separate from that issue.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

📌 Task
Status

Active

Version

11.0 🔥

Component

phpunit

Created by

🇮🇹Italy mondrake 🇮🇹

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mondrake
  • Merge request !9664Experiment paratest → (Open) created by mondrake
  • Pipeline finished with Failed
    6 months ago
    Total: 116s
    #294890
  • Pipeline finished with Failed
    6 months ago
    Total: 232s
    #294902
  • Pipeline finished with Failed
    6 months ago
    Total: 116s
    #294913
  • Pipeline finished with Failed
    6 months ago
    Total: 100s
    #294921
  • Pipeline finished with Failed
    6 months ago
    Total: 115s
    #294929
  • Pipeline finished with Failed
    6 months ago
    Total: 216s
    #294942
  • Pipeline finished with Failed
    6 months ago
    Total: 240s
    #294957
  • Pipeline finished with Failed
    6 months ago
    Total: 225s
    #294968
  • Pipeline finished with Failed
    6 months ago
    #294996
  • Pipeline finished with Failed
    6 months ago
    Total: 316s
    #295643
  • Pipeline finished with Failed
    6 months ago
    Total: 246s
    #296057
  • Pipeline finished with Failed
    6 months ago
    #296059
  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Success
    6 months ago
    Total: 216s
    #296320
  • 🇬🇧United Kingdom catch

    The disconnect between paratest and run-tests.sh timings looked a bit too much, so I checked phpunit test timings in HEAD and found bugs in the run-tests.sh logic (recently introduced by me).

    📌 Try a shorter polling time in run-tests.sh Active is the issue.

    https://git.drupalcode.org/project/drupal/-/jobs/2938499 reports 42 seconds to run tests now so a better baseline for comparison.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Also, in the past when I've used paratest on projects I've had problems with deprecation testing and how Symfony adds the info. Maybe with the recent PHPUnit / Symfony PHPUnit Bridge changes this is no longer a problem but we need to confirm whether deprecation fails are still reported correctly - both that the test fails and that we can see the deprecation that causes the fail.

  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Failed
    5 months ago
    Total: 68s
    #326387
  • Pipeline finished with Canceled
    5 months ago
    Total: 194s
    #326391
  • Pipeline finished with Success
    5 months ago
    Total: 220s
    #326393
  • 🇮🇹Italy mondrake 🇮🇹

    plain PHPUnit: 1 min 54 secs
    paratest: 24 secs
    run-tests: 31 secs

  • Merge request !11573Closes #3477444 → (Open) created by mondrake
  • Pipeline finished with Failed
    16 days ago
    Total: 146s
    #455010
  • 🇮🇹Italy mondrake 🇮🇹

    MR!11573 combines 🌱 [meta] Support PHPUnit 11 in Drupal 10 Postponed with 📌 [PP-1][CI] Spin off Drupal Components tests in a job of their own Postponed and runs Drupal\Component tests using paratest without any change.

    PHPUnit CLI: PHP 8.3 43sec; PHP 8.4 43sec; PHP 8.5 44sec.

    paratest (using 72 processes): PHP 8.3 18sec; PHP 8.4 19sec; PHP 8.5 23sec.

  • Pipeline finished with Failed
    16 days ago
    Total: 202s
    #455024
  • 🇮🇹Italy mondrake 🇮🇹

    Adding a test failure and a deprecation, observations:

    • --testdox command line toggle does not seem to work
    • deprecation reporting seem to be working fine (re comment #5 above)
  • Pipeline finished with Failed
    16 days ago
    Total: 365s
    #455357
  • Pipeline finished with Canceled
    16 days ago
    Total: 421s
    #455358
  • Pipeline finished with Failed
    16 days ago
    Total: 246s
    #455370
  • 🇮🇹Italy mondrake 🇮🇹

    The missing testdox output is caused by a change in PHPUnit 11.5.12 that is not matched by paratest 7.8.3

    With some hacking I managed to fix it here, but will require being fixed upstream.

  • 🇮🇹Italy mondrake 🇮🇹

    #11 is actually fixed upstream already in 7.9.0, that is only compatible with PHPUnit 12 - this will be the bigger problem to use paratest in Drupal: it only supports the latest PHPUnit version, that Drupal is consistently late in adopting at the moment.

  • Pipeline finished with Failed
    14 days ago
    Total: 218s
    #456997
Production build 0.71.5 2024