sebastianbergmann/comparator:5.0.2 Introduces (for Drupal) breaking changes

Created on 16 August 2024, about 1 month ago
Updated 8 September 2024, 11 days ago

Problem/Motivation

phpunit/phpunit released 10.5.30. (https://github.com/sebastianbergmann/phpunit/releases/tag/10.5.30)
Updating to that version causes an update of sebastianbergmann/comparator to 5.0.2. (https://github.com/sebastianbergmann/comparator/releases/tag/5.0.2)

The latter update is where the "fun" starts. The changes in https://github.com/sebastianbergmann/comparator/pull/113/files#diff-40f6... causes the expected and actual arrays to be only sorted if both have only numeric keys in order 0, 1, 2, ... - so called "list mode".

This breaks some of our tests when using assertEqualsCanonicalizing().

Example:

Drupal\Tests\system\Kernel\Common\UrlTest::testLinkBubbleableMetadata
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     'contexts' => Array (
         0 => 'languages:language_interface'
-        1 => 'session'
-        2 => 'theme'
-        3 => 'user.permissions'
+        1 => 'theme'
+        2 => 'user.permissions'
+        6 => 'session'
     )
     'tags' => []
     'max-age' => -1
 )

core/modules/system/tests/src/Kernel/Common/UrlTest.php:71

Previously both arrays would be sorted before comparing, also changing the keys to be in order 0, 1, 2, ..., those days are gone with
sebastianbergmann/comparator going to 5.0.2.

Steps to reproduce

Take a look at the test results of the daily 'Updated deps' run, like this one: failed run.

, there are occurrences of Failed asserting that two arrays are equal.


Also bump the version of phpunit/phpunit which in turn bumps sebastianbergmann/comparator.
This version of PHPUnit has better checks on our tests, that already proved its value in ๐Ÿ“Œ phpunit/phpunit:10.5.30 Introduces (for Drupal) breaking changes Needs work .

Proposed resolution

Fix Failed asserting that two arrays are equal. test failures, by returning the array_values() of the method that causes the test-failure, since we're only interested in the values anyway.

There's a Slack discussion about this with @longwave here: https://drupal.slack.com/archives/C079NQPQUEN/p1723799215004749

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

๐Ÿ“Œ Task
Status

Fixed

Version

11.0 ๐Ÿ”ฅ

Component
PHPUnitย  โ†’

Last updated about 4 hours ago

Created by

๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Merge request !9240sebastianbergmann/comparator-5.0.2 โ†’ (Closed) created by Spokje
  • Pipeline finished with Canceled
    about 1 month ago
    Total: 90s
    #257203
  • Pipeline finished with Failed
    about 1 month ago
    Total: 574s
    #257204
  • Pipeline finished with Failed
    about 1 month ago
    Total: 214s
    #257312
  • Pipeline finished with Failed
    about 1 month ago
    Total: 1967s
    #257316
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Status changed to Needs work about 1 month ago
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Status changed to Needs review 30 days ago
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje

    Putting this on NR to get some confirmation if this is the way to move forward.

    The real MR needs to not-bump versions, but doing this here temporarily is making detecting test-failures much easier.

  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Issue was unassigned.
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Pipeline finished with Success
    27 days ago
    Total: 679s
    #262461
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    We have a Drupal\Component\Utility\NestedArray already. Wouldnโ€™t it make sense to extend it to add a deepSort() method? So we could also add some testing for it.

    Converting an object to an array maybe does not need an helper, better explicitly be done when needed. Itโ€™s a one line anyway.

  • Pipeline finished with Canceled
    25 days ago
    Total: 520s
    #264024
  • Assigned to Spokje
  • Status changed to Needs work 25 days ago
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Pipeline finished with Success
    25 days ago
    #264032
  • Pipeline finished with Failed
    25 days ago
    Total: 224s
    #264068
  • Pipeline finished with Failed
    25 days ago
    #264069
  • Pipeline finished with Failed
    24 days ago
    Total: 333s
    #264078
  • Pipeline finished with Canceled
    24 days ago
    Total: 125s
    #264085
  • Pipeline finished with Success
    24 days ago
    Total: 537s
    #264088
  • Status changed to Needs review 24 days ago
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje

    Thanks @mondrake, I agree mostly with your review, but that would be in the case this issue would be about actual deep sorting an array.

    It isn't, it's about a one-thing-to-fix-one-thing: Massaging data so it will be accepted by assertEqualsCanonicalizing() the same way it was before this update.

    Updated the naming (which didn't explain that at all, mea culpa) and sprinkled comments on some cleaned-up code.

    In my mind, this would benefit from being a wrapper somehow around assertEqualsCanonicalizing() to save core, and especially contrib from watching test-failures first to see where data-massaging is needed.
    Not sure if this is possible, nor even desirable for core.

    Anyway, putting this on NR to get more of your (and others) feedback.

  • Issue was unassigned.
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • Pipeline finished with Failed
    24 days ago
    Total: 293s
    #264842
  • Pipeline finished with Success
    24 days ago
    Total: 520s
    #264865
  • Merge request !9330alt fix โ†’ (Closed) created by mondrake
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    Trying an alternative approach in MR!9330 - just making sure that Cache::mergeTags() and ::mergeContexts() return actual lists i.e. after removing dupes, the resulting array is sequenced 0...n.

  • Pipeline finished with Success
    23 days ago
    Total: 519s
    #265348
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje

    Thanks @mondrake, this is exactly what @longwave intended in the original Slack-discussion.
    I misunderstood it and tried (and failed) to use array_values in the failing tests, instead of the method that causes the test failure.

    All this needs is a CR so contrib knows what to do if it hits this test-failure in its existing tests and this is RTBC.

  • Pipeline finished with Success
    23 days ago
    Total: 1098s
    #265764
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    @Spokje I am not sure about a CR here. The behavior change happened in PHPUnit, and I do not think we should echo their change, otherwise we should do that for any change in PHPUnit. The fix is an implementation detail, no one should really care about that. Happy to hear different opinions, though.

  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje

    @mondrake:

    longwave
    Aug 16th at 11:13 AM
    yeah maybe we just fix core uses, document the solution with a change record, and move on
    11:14
    to me this is a BC break in sebastianbergmann/comparator but I assume Sebastian will see it as a bug and not a feature

    @longwave in Slack (https://drupal.slack.com/archives/C079NQPQUEN/p1723799591056949?thread_t...)

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น
  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands Spokje

    I took the bold step to rewrite the CR a little, to help contrib find it more easily and advice on how this was fixed in core.

    If you agree with it (feel free to rewrite it if not), I'm willing to RTBC this.

  • Status changed to RTBC 23 days ago
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly mondrake ๐Ÿ‡ฎ๐Ÿ‡น

    Looks good. Setting to RTBC as confirmation of the intent already expressed in #28

  • Status changed to Fixed 22 days ago
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom longwave UK

    Thanks, I think explicitly returning a list here is more correct behaviour and it's cleaner than having to modify multiple tests (and perhaps some in contrib or custom code).

    Committed to 11.x. As a minor behaviour change I don't think it's worth backporting to a patch release or even down to 10.4.x given we only need it in order to pass tests.

    Also published the CR.

  • ๐Ÿ‡ฆ๐Ÿ‡นAustria drunken monkey Vienna, Austria

    Committed to 11.x. As a minor behaviour change I don't think it's worth backporting to a patch release or even down to 10.4.x given we only need it in order to pass tests.

    As a module maintainer, just wanted to chime in to say that I would appreciate this being applied to as many Drupal versions as possible (10.3+ would be best), so I can get rid of custom overrides for making my moduleโ€™s CI pass as soon as possible. Currently, Iโ€™d have to wait until the MAX_PHP_VERSION test uses Drupal 11.1 (I think?) before I can remove the custom override.

Production build 0.71.5 2024