[PP-1] \Drupal\automatic_updates\Validation\StatusCheckRequirements makes admin/reports/status 10x slower!

Created on 11 May 2023, over 1 year ago
Updated 15 June 2023, over 1 year ago

Problem/Motivation

Accessing /admin/reports/status takes:

  1. ๐Ÿ”ด 5 seconds when Automatic Updates is installed! ๐Ÿ˜ฑ
  2. ๐ŸŸข 0.5 seconds with automatic_updates_requirements()' $requirements = $status_check_requirement->getRequirements(); commented out

And this is on a trivial Drupal project on a very fast machine. The numbers would likely be worse in real world contexts.

Steps to reproduce

See above.

Proposed resolution

In beginning of \Drupal\automatic_updates\Validation\StatusCheckRequirements::getRequirements if the results are NULL
and the we are setup up to use console the we just tell the user

Unattended updates are configured to run via the console, but do not appear to have run recently.

But if the results are NULL
and the we are setup up to use web we just run the status checker
see
$results ??= $this->statusChecker->run()->getResults();
We should instead give the user a message and just give the user a link to run the status checks \Drupal\automatic_updates\Validation\StatusCheckRequirements::createRunLink()

Remaining tasks

  1. Logging that measures how long each of the event subscribers called by \Drupal\automatic_updates\Validation\StatusCheckRequirements takes.
  2. TBD!

User interface changes

A reasonably fast UI

API changes

TBD

Data model changes

None.

๐Ÿ› Bug report
Status

Postponed

Version

3.0

Component

Code

Created by

๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

  • needs profiling

    It may affect performance, and thus requires in-depth technical reviews and profiling.

Sign in to follow issues

Comments & Activities

  • Issue created by @wim leers
  • Status changed to Postponed over 1 year ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA

    In ๐Ÿ“Œ Add new setting for how unattended updates will be run Fixed we will probably change to only showing the cached results on the status report page. So then this would not affect the page load

    Once that issue is fixed can re-open this issue to test if it is still a problem. I think it will not be and we can just close the issue.

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    ๐Ÿ‘

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA

    actually ๐Ÿ“Œ Add new setting for how unattended updates will be run Fixed is very big so lets just do this after that issue is done.

  • Status changed to Active over 1 year ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    Blocker is in.

  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    omkar.podey โ†’ made their first commit to this issueโ€™s fork.

  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    running a kernel test gives me

    total time package_manager.validator.environment_support =>>> 0.00011515617370605
    total time package_manager.validator.composer =>>> 0.47905015945435
    total time package_manager.validator.disk_space =>>> 0.00065398216247559
    total time package_manager.validator.file_system =>>> 0.0002140998840332
    total time package_manager.validator.multisite =>>> 8.9883804321289E-5
    total time package_manager.validator.symlink =>>> 0.013756036758423
    total time package_manager.validator.stage_not_in_active =>>> 0.00015902519226074
    total time package_manager.validator.base_requirements_fulfilled =>>> 0.00034904479980469
    total time package_manager.validator.pending_updates =>>> 0.17668294906616
    total time package_manager.validator.lock_file =>>> 0.00026702880859375
    total time package_manager.validator.allowed_scaffold_packages =>>> 0.28983497619629
    total time package_manager.validator.staged_database_updates =>>> 0.00043702125549316
    total time package_manager.validator.settings =>>> 9.8943710327148E-5
    total time Drupal\package_manager\Validator\RsyncValidator =>>> 0.00078988075256348
    total time package_manager.validator.composer_plugins =>>> 0.24100685119629
    total time package_manager.validator.patches =>>> 0.4860999584198
    total time package_manager.validator.php_extensions =>>> 0.00017690658569336
    total time package_manager.validator.php_extensions =>>> 0.00015687942504883
    

    the validators that stand out

    1. package_manager.validator.composer
    2. package_manager.validator.allowed_scaffold_packages
    3. package_manager.validator.composer_plugins
    4. package_manager.validator.patches
  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    from a build test

    total time package_manager.validator.environment_support =>>> 0.00042605400085449
    total time package_manager.validator.composer =>>> 0.62173891067505
    total time package_manager.validator.disk_space =>>> 0.00076508522033691
    total time package_manager.validator.file_system =>>> 0.00023698806762695
    total time package_manager.validator.multisite =>>> 0.00013208389282227
    total time package_manager.validator.symlink =>>> 4.5157771110535
    total time package_manager.validator.stage_not_in_active =>>> 0.00046586990356445
    total time package_manager.validator.base_requirements_fulfilled =>>> 0.00059890747070312
    total time automatic_updates.cron_frequency_validator =>>> 0.00032591819763184
    total time Drupal\automatic_updates\Validator\AutomatedCronDisabledValidator =>>> 0.0001530647277832
    total time automatic_updates.validator.version_policy =>>> 0.7344229221344
    total time automatic_updates.validator.cron_server =>>> 9.5844268798828E-5
    total time package_manager.validator.pending_updates =>>> 0.011720895767212
    total time package_manager.validator.lock_file =>>> 0.00018596649169922
    total time package_manager.validator.allowed_scaffold_packages =>>> 0.14549493789673
    total time package_manager.validator.staged_database_updates =>>> 0.00029420852661133
    total time package_manager.validator.settings =>>> 0.00016403198242188
    total time Drupal\package_manager\Validator\RsyncValidator =>>> 0.00024795532226562
    total time package_manager.validator.composer_plugins =>>> 0.30691409111023
    total time package_manager.validator.patches =>>> 0.63482904434204
    total time package_manager.validator.php_extensions =>>> 0.00059890747070312
    total time package_manager.validator.php_extensions =>>> 0.00015783309936523
    
    1. the validators that stand out
    2. package_manager.validator.composer
    3. package_manager.validator.symlink
    4. automatic_updates.validator.version_policy
    5. package_manager.validator.composer_plugins
    6. package_manager.validator.patches
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States phenaproxima Massachusetts

    Crediting Omkar for the profiling.

  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    the code i used to get results by modifying \Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher::dispatch

              if ($event_name == StatusCheckEvent::class) {
                if (array_key_exists('service', $definition)) {
                  $name = $definition['service'][0];
                  $time = microtime(TRUE);
                  file_put_contents('/Users/omkar.podey/www/auto_updates_dev/sites/test.txt', "total time $name =>>> ", FILE_APPEND);
                }
              }
              call_user_func($definition['callable'], $event, $event_name, $this);
              if ($event_name == StatusCheckEvent::class) {
                if (array_key_exists('service', $definition)) {
                  file_put_contents('/Users/omkar.podey/www/auto_updates_dev/sites/test.txt', microtime(TRUE) - $time . "\n", FILE_APPEND);
                }
              }
    
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA
  • Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    Not currently mergeable.
  • @tedbow opened merge request.
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    783 pass, 3 fail
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA

    I pushed a start. We will need tests.

    I think we should the profiling but in any case I don't think we should run the status checks unless they click the link.

    In most cases there should be some results. If not then it probably means they don't have cron or the console command running often enough

  • Status changed to Needs work over 1 year ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA
  • Assigned to omkar.podey
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    785 pass, 3 fail
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    800 pass
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    @omkar.podey GREAT work on #8 & #9! ๐Ÿ‘ ๐Ÿคฉ

    If not then it probably means they don't have cron or the console command running often enough

    Great point, @tedbow! ๐Ÿ‘ Agreed.

    99% done. Just one @todo that needs to be addressed.

  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    Custom Commands Failed
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    800 pass
  • Status changed to Needs review over 1 year ago
  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    I can't login into drupal code the @todo

     // @todo Link to the documentation on how to set up unattended updates
              //   via the terminal in https://drupal.org/i/3362695.

    I don't think there is a point unless we have documentation and right now we don't. ๐Ÿ“Œ Create documentation for using the auto-update terminal command Active

  • Status changed to Needs work over 1 year ago
  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    #18: that is not the @todo I commented on. I commented on the other one:

    // @todo Do better text.
    
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    800 pass
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    800 pass
  • Issue was unassigned.
  • Status changed to Needs review over 1 year ago
  • Assigned to omkar.podey
  • Status changed to Needs work over 1 year ago
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA

    Looking good! Needs Work for MR comments

  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    794 pass, 1 fail
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    794 pass, 1 fail
  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    794 pass, 2 fail
  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia omkar.podey

    Still fails as the status can't be fetched The project 'drupal' can not be updated because its status is not-fetched

  • Issue was unassigned.
  • Status changed to Postponed over 1 year ago
Production build 0.71.5 2024