[CI] Components tests coverage metrics differ by PHP version

Created on 28 April 2025, 15 days ago

Problem/Motivation

Follow-up of ๐Ÿ“Œ [PP-1][CI] Spin off Drupal Components tests in a job of their own Postponed .

Components tests coverage metrics differ slightly by PHP version, and apparently the last PHP version job completing the components unit test is winning the box. Not a big deal, but this can cause some confusion when looking at test results and seeing that metrics are changing even if components code/tests are not part of the MR.

Example from HEAD on Apr 28, 2025:

PHP 8.3 and 8.4

  Classes: 31.62% (37/117)   
  Methods: 46.88% (315/672)  
  Lines:   36.59% (2782/7604)

PHP 8.5

  Classes: 30.77% (36/117)   
  Methods: 46.43% (312/672)  
  Lines:   36.53% (2778/7604)

Proposed resolution

Possible solutions:

  1. Produce code coverage report only in the job running the default PHP version; no code coverage for the other PHP versions.
  2. Produce a different set of metrics for each PHP version.
  3. Investigate why the differences and try to remove them (might be impossible if there are code paths dependent on PHP version checking)

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

๐Ÿ› Bug report
Status

Active

Version

11.0 ๐Ÿ”ฅ

Component

phpunit

Created by

๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mondrake
  • Merge request !11963Closes #3521541 โ†’ (Open) created by mondrake
  • Pipeline finished with Failed
    15 days ago
    Total: 255s
    #483886
  • Pipeline finished with Canceled
    13 days ago
    Total: 152s
    #485409
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    MR implements option 1 from the issue summary. IMHO option 2 is overkill and 3 is complicated and risk regressions easily.

  • Pipeline finished with Success
    13 days ago
    Total: 3267s
    #485410
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

    Yeah only for the default PHP version sounds good to me too.

  • Pipeline finished with Success
    13 days ago
    #485593
  • Pipeline finished with Success
    12 days ago
    Total: 668s
    #486580
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States smustgrave

    Assuming this will have to be tweaked when https://www.drupal.org/project/drupal/issues/3515706 ๐Ÿ“Œ Switch the default test environment to PHP 8.4 Active lands?

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    Why so?

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States smustgrave

    Didn't mean to change status but if 8.4 is a default won't this change need to apply to 8.4

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    Ah! No itโ€™s already for 8.4, check the MR - 8.3 and 8.5 are getting the โ€œno-coverageโ€ job settings. Anyway this is about components unit tests which are getting rather independent as we speak.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States smustgrave

    Thanks for clarifying!

  • Pipeline finished with Success
    11 days ago
    Total: 499s
    #486840
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    Sorry, I found opportunities to remove duplication

  • Pipeline finished with Success
    11 days ago
    Total: 608s
    #486848
  • Pipeline finished with Success
    11 days ago
    Total: 443s
    #487076
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States smustgrave

    Clean up seems to be good

  • Pipeline finished with Failed
    10 days ago
    Total: 232s
    #487870
  • Pipeline finished with Failed
    9 days ago
    Total: 592s
    #488367
  • Pipeline finished with Success
    6 days ago
    Total: 694s
    #491325
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    Adjusted post- ๐ŸŒฑ [meta] Support PHPUnit 11 in Drupal 10 Postponed .

    Letting PHPUnit's own deprecations fail the job execution on PHPUnit 8.5 as the purpose of that job is precisely to shout out when language deprecation or changes occur in this stage of PHP development, so I thought it is a good spot where to let PHPUnit shout as well.

    With the PHPUNIT_FAIL_ON_PHPUNIT_DEPRECATION job variable it's a piece of cake to silence them if we do not manage to solve them before PHP 8.5 is released.

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น
  • Pipeline finished with Success
    5 days ago
    Total: 1225s
    #491972
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น
Production build 0.71.5 2024