Handle Package Manager errors and warnings more elegantly

Created on 6 June 2023, about 1 year ago
Updated 26 June 2024, 1 day ago

Problem/Motivation

Currently, we run through the validators of Package Manager in order to know if we should support UI installations. However, some of those are warnings and some of them are errors. One example is the Xdebug warning, which exists due to performance issues. This makes debugging difficult, since we have to comment out the validator.

Steps to reproduce

Enable xdebug. Attempt to install a module from the UI in Project Browser.

Proposed resolution

Do not prevent installations if the only messages coming back from Package Manager are warnings; there should be at least one ERROR to prevent the installs from running.

NOTE: Adam (phenaproxima) suggested that PM could add a hasErrors() method, if we want to pursue that upstream.

📌 Task
Status

Fixed

Version

2.0

Component

Code

Created by

🇺🇸United States chrisfromredfin Portland, Maine

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @chrisfromredfin
  • Assigned to earthday47
  • 🇺🇸United States earthday47
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    65 pass
  • Status changed to Needs review about 1 year ago
  • 🇺🇸United States earthday47

    Fix in place.
    Does this need a test?
    Also - do we want to print warnings still? At the moment this is suppressing them.

  • Status changed to Needs work about 1 year ago
  • 🇺🇸United States earthday47

    We want to print warnings

  • 🇺🇸United States chrisfromredfin Portland, Maine

    Yes there should be a test to go along with this, if possible. Not sure how to get it to fire a warning, though.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    Custom Commands Failed
  • Status changed to Needs review about 1 year ago
  • 🇺🇸United States earthday47

    Refactored the way we return messages from package manager to return errors and non-errors (warnings). If there are any errors, prevent installs and show the heading "Unable to download modules via the UI". If there are any warnings, we print them without a header.

    I've added some screenshots of the various scenarios we could get if there are multiple messages. Chris informed me that we're unlikely to get both because of the way Package Manager works, regardless, the styles are in there just in case this happens.

  • 🇺🇸United States earthday47

    Will have to work on tests later - any help in that department is appreciated!

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    65 pass
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    65 pass
  • Status changed to Needs work about 1 year ago
  • 🇺🇸United States chrisfromredfin Portland, Maine

    Tested this manually in DrupalPod and is working well. Allows me to install even if XDebug is enabled, for example, which is perfect. (And great DX.)

    Kicking back to NW because it needs tests, but it's working well. If anyone knows of a way to write a test for this to (a) cause an error from PM and (b) cause a _warning_ from PM, help appreciated.

    I will go through and do a code review also, just to get that started.

  • 🇪🇸Spain fjgarlin

    Also left a review in the MR. Nothing big, but worth addressing or discussing.

  • 🇺🇸United States chrisfromredfin Portland, Maine

    We may want to wait on https://www.drupal.org/project/project_browser/issues/3376202 📌 Error messages should utilize Drupal's messages Needs work to come in.

  • 🇺🇸United States chrisfromredfin Portland, Maine
  • 🇺🇸United States pfrilling Minster, OH

    pfrilling → changed the visibility of the branch 3365180-pm-warnings to hidden.

  • 🇺🇸United States pfrilling Minster, OH

    pfrilling → changed the visibility of the branch 3365180-pm-warnings to active.

  • Status changed to Needs review 7 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    54:02
    45:33
    Running
  • 🇺🇸United States pfrilling Minster, OH

    Attached is a patch that _should_ add test coverage for both warnings and errors coming from package manager. I also attempted to merge the changes from the MR into this patch. I admin, I'm no Svelte expert, so there is a good chance I didn't do the JS stuff correctly.

  • First commit to issue fork.
  • Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update 7 months ago
    Waiting for branch to pass
  • 🇺🇸United States andy-blum Ohio, USA

    Moved Phil's patch into the MR and fixed PHPCS errors

  • 🇺🇸United States andy-blum Ohio, USA

    Test failures appear to be aligned with current failures against 1.0.x

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update 6 months ago
    Patch Failed to Apply
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 5.7
    last update 6 months ago
    55 pass, 4 fail
  • 🇦🇺Australia sime Canberra

    Plan to refresh/rebase and review as this touches the code i'm working on here https://www.drupal.org/project/project_browser/issues/3451863 📌 Too slow to validatePackageManager() on every page Needs review

  • Pipeline finished with Failed
    23 days ago
    Total: 856s
    #191470
  • Pipeline finished with Failed
    23 days ago
    Total: 752s
    #191584
  • Pipeline finished with Failed
    23 days ago
    Total: 722s
    #191603
  • Assigned to sime
  • Status changed to Needs work 23 days ago
  • 🇦🇺Australia sime Canberra
  • Pipeline finished with Failed
    23 days ago
    Total: 760s
    #191830
  • Pipeline finished with Failed
    22 days ago
    Total: 715s
    #192270
  • Pipeline finished with Failed
    22 days ago
    Total: 864s
    #192276
  • Pipeline finished with Failed
    22 days ago
    Total: 633s
    #192282
  • Pipeline finished with Failed
    21 days ago
    Total: 724s
    #193220
  • 🇦🇺Australia sime Canberra

    I'm still in progress cleaning up tests on this please message @sime in slack if you want to contribute or take over.

  • Pipeline finished with Failed
    20 days ago
    Total: 787s
    #194140
  • Pipeline finished with Failed
    19 days ago
    Total: 570s
    #194545
  • Pipeline finished with Failed
    19 days ago
    Total: 3142s
    #194625
  • Pipeline finished with Failed
    19 days ago
    Total: 462s
    #194687
  • Pipeline finished with Failed
    19 days ago
    Total: 372s
    #194713
  • Pipeline finished with Failed
    19 days ago
    Total: 555s
    #194853
  • Pipeline finished with Failed
    18 days ago
    Total: 341s
    #195509
  • Pipeline finished with Failed
    18 days ago
    Total: 333s
    #195534
  • Pipeline finished with Failed
    18 days ago
    Total: 394s
    #195543
  • Pipeline finished with Failed
    18 days ago
    Total: 1949s
    #195551
  • Pipeline finished with Success
    18 days ago
    Total: 1155s
    #195564
  • 🇦🇺Australia sime Canberra

    OK. So this was meant to be just a rebase, and it still is a rebase because I didn't really change anything from the work 6 months ago.

    Code logic

    I haven't really touched the code logic, but i did have to update the message handling in the svelte, since we are using the Drupal.messages() since the last work on this.

    I'm hoping to get things back to where the code reviews were at, as I do think there is ab it more work here, but I think it needs a review as well.

    Tests

    1. There were event listeners which triggered errors when calling ->validatePackageManager(). This allows the code to spend a lot of time in the package_manager code, so i mocked InstallReadiness instead which means we are focussed on testing PB login instead of PM logic.
    2. I was able to remove a todo from PackageManagerFixtureUtilityTrait->initPackageManager() removing a fair bit on module installation and move those modules back to the tests's $modules property
    3. Note that this currently includes 📌 phpstan.neon.dist fixes - Get phpstan to level 1 RTBC just to help me wade through the errors.
  • Status changed to Needs review 18 days ago
  • 🇦🇺Australia sime Canberra
  • 🇦🇺Australia sime Canberra

    I still have this assigned to myself just because if someone wants to pick it up it would be great to talk through the approach.

    Some thoughts on next steps

    1. Review the 6 month old unresolved comments
    2. I think we should change the wording "messages" to "warnings" the same way that "errors" => "errors"
    3. I think I have a plan for removing the need to isntall package manager, thus hopefully further reducing flakiness
  • Pipeline finished with Failed
    15 days ago
    Total: 443s
    #197849
  • Merge request !510Handle warnings/messages in UI (for 2.0.x) → (Merged) created by sime
  • 🇦🇺Australia sime Canberra

    sime → changed the visibility of the branch 3365180-pm-warnings to hidden.

  • 🇦🇺Australia sime Canberra

    Rebased and created new PR on 2.0.x branch.

  • Pipeline finished with Failed
    15 days ago
    Total: 414s
    #197869
  • 🇦🇺Australia sime Canberra

    I mostly resolved @fjgarlin's comments from 1.x MR

  • Status changed to Needs work 15 days ago
  • 🇦🇺Australia sime Canberra

    Non working state, removing from needs review.

  • Pipeline finished with Failed
    15 days ago
    Total: 866s
    #198016
  • Pipeline finished with Failed
    15 days ago
    Total: 509s
    #198031
  • Pipeline finished with Failed
    13 days ago
    Total: 175s
    #199521
  • Pipeline finished with Failed
    13 days ago
    Total: 854s
    #199524
  • Pipeline finished with Failed
    13 days ago
    Total: 423s
    #199565
  • Pipeline finished with Failed
    13 days ago
    Total: 378s
    #199568
  • Pipeline finished with Failed
    13 days ago
    Total: 397s
    #199575
  • Pipeline finished with Failed
    13 days ago
    Total: 845s
    #199588
  • Pipeline finished with Failed
    13 days ago
    Total: 440s
    #199614
  • Pipeline finished with Success
    13 days ago
    Total: 839s
    #199626
  • Pipeline finished with Success
    13 days ago
    Total: 797s
    #199638
  • Issue was unassigned.
  • Status changed to Needs review 13 days ago
  • 🇦🇺Australia sime Canberra

    Updated to 2.0.x (new branch). Old branch kept.

  • 🇦🇺Australia sime Canberra

    Just re-summarising this progress. The previous progress is still here against 1.x.

    Testing strategy

    Compared to the 1.x progress, I changed the way errors were mocked so that test logic did not need to go deep into Package Manager. So the updated tests are testing scenarios InstallReadiness generates errors, rather than testing how PM generates errors.

    The reason i changed the tests was because it was very flaky (intermittent time outs and failures) and slow for me (even without xdebug). I think this remains a bigger problem with a lot of PB tests seem to do full integration not always for obvious reasons. In these case we are flexing the PM code just to see how ProjectBrowser.svelte handles these messages.

    Long story short, I can revert how the tests are architected if asked. it was a journey for me to get them to work but i'm not wedded to my approach.

    Message strategy

    Instead of just setting status messages in PHP, we are turning them into dumb strings and passing them to the svelte app, which then uses the javascript equivalent of setting the messages. We could just set them in PHP. Basically the current 2.x code is still doing the same thing as a the 1.x coee, I just wanted to point that out.

    Couple other points:

    1. I did change the svelte app so that it clears and current status messages just to avoid the screen getting filled with messages.
    2. I aligned variable names so that everything from package_manager is prefixed as such.

    Installing test dependencies

    A big thing in this MR is that it seems like we no longer need to install package_manager in a weird way via initPackageManager() and we can just use the $modules property. That seems like a good bit of technical debt removed.

  • Pipeline finished with Success
    12 days ago
    Total: 817s
    #200100
  • First commit to issue fork.
  • Pipeline finished with Failed
    10 days ago
    Total: 448s
    #201173
  • Pipeline finished with Failed
    10 days ago
    Total: 428s
    #201182
  • Pipeline finished with Success
    10 days ago
    Total: 461s
    #201187
  • Pipeline finished with Failed
    10 days ago
    Total: 231s
    #201200
  • Pipeline finished with Canceled
    10 days ago
    Total: 52s
    #201202
  • Pipeline finished with Failed
    10 days ago
    #201204
  • Pipeline finished with Canceled
    10 days ago
    Total: 69s
    #201208
  • Pipeline finished with Canceled
    10 days ago
    Total: 282s
    #201211
  • Pipeline finished with Canceled
    10 days ago
    Total: 89s
    #201213
  • Pipeline finished with Success
    10 days ago
    Total: 430s
    #201215
  • 🇺🇸United States phenaproxima Massachusetts
  • 🇺🇸United States phenaproxima Massachusetts
  • 🇦🇺Australia sime Canberra

    @phenaproxima has tightened the scope and it is looking good to me. I like the improvements in variable handling.

    I have created a few extra tickets out of the things I learnt in the course of working on it.

    📌 Resolve technical debt in initPackageManager() Active
    📌 ServiceProviderBase::alter shouldn't call parent::register Active
    📌 Identify and action solutions for flaky tests Active

  • Status changed to Needs work 9 days ago
  • 🇦🇺Australia sime Canberra

    I'm just going to combined these up a little. It would be rare to have multiple errors here, I believe, so we'll just prefix each one and skip the extra ones?

  • 🇦🇺Australia sime Canberra

    Changes I made result in this:

    The worst case scenario is two messages of the same type, thus showing the same prefix, but I think this is acceptable as it's very rare and looks fine.

  • Status changed to Needs review 9 days ago
  • 🇦🇺Australia sime Canberra
  • Pipeline finished with Failed
    9 days ago
    Total: 582s
    #202449
  • Pipeline finished with Success
    9 days ago
    Total: 976s
    #202511
  • Pipeline finished with Success
    8 days ago
    Total: 864s
    #203242
  • 🇦🇺Australia sime Canberra

    For anyone who wants to manually test this, a simple way is to install the project_browser_test module.

    1. You need this in your settings.php, note that it will start showing test modules/themes in the UI.

    $settings['extension_discovery_scan_tests'] = TRUE;

    2. Then install project_browser_test, I just use drush.

    3. Now in TestInstallerReadiness.php in the statusCheck method, just change the code to manual create whatever messages you want to create.

  • Status changed to RTBC 8 days ago
  • 🇦🇺Australia sime Canberra

    Ya know, I think this is RTBC. Feels solid to me. Will let Chris have the final say.

  • Status changed to Needs work 7 days ago
  • 🇺🇸United States chrisfromredfin Portland, Maine

    With manual testing, prior to applying this MR - all messages show (two status messages and the error). When applied, only the error shows (now correctly as a warning, though). What happened to the other messages?

  • Pipeline finished with Success
    7 days ago
    Total: 389s
    #204865
  • Pipeline finished with Success
    7 days ago
    Total: 386s
    #204869
  • Status changed to RTBC 7 days ago
  • 🇦🇺Australia sime Canberra
  • Pipeline finished with Success
    1 day ago
    Total: 675s
    #208719
  • Pipeline finished with Skipped
    1 day ago
    #208734
  • Status changed to Fixed 1 day ago
  • 🇺🇸United States chrisfromredfin Portland, Maine

    Oh man just being able to have Xdebug on is such a win. ;)

Production build 0.69.0 2024