\Drupal\plugin\PluginType\PluginTypeManager::getPluginTypes() does not use file cache correctly

Created on 22 January 2024, about 1 year ago
Updated 8 February 2024, about 1 year ago

Problem/Motivation

Unfortunately \Drupal\plugin\PluginType\PluginTypeManager::getPluginTypes() does not use FileCache correctly and this results in a massive amount of extra memory usage for no reason during a site install.

Steps to reproduce

Install a site with many modules and plugin among them. Run site install from config using drush -vvv.

Before this change:
[success] Installation complete. User name: dru_admin User password: ********* [68.05 sec, 1.05 GB]

After this:
[success] Installation complete. User name: dru_admin User password: ******** [71 sec, 224.18 MB]

Proposed resolution

Do not vary the file cache by provider it is misunderstanding how file cache works.

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

πŸ› Bug report
Status

Needs review

Version

2.0

Component

Code

Created by

πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @alexpott
  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ
  • Status changed to Needs review about 1 year ago
  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ
  • πŸ‡«πŸ‡·France andypost

    Sounds like needs backport to 10.2

  • Merge request !8Use FileCache correctly β†’ (Open) created by berdir
  • Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MariaDB 10.3.22
    last update about 1 year ago
    Waiting for branch to pass
  • Status changed to RTBC about 1 year ago
  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Change makes sense, no idea what's going on with those numbers, I managed to get it to report 1.6GB, although I really wonder if those numbers are accurate or if it's confused somehow. top seems to match that.

    Before:

     [success] Installation complete.  User name: admin  User password: **** [61.66 sec, 1.61 GB]
    

    After:

     [success] Installation complete.  User name: admin  User password: **** [78.85 sec, 226.16 MB]
    
  • Status changed to Needs work about 1 year ago
  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ

    I guess another question here is why does the plugin module cause 10mb of memory usage...

    Before I applied this change:

     [info] plugin module installed. [137.17 sec, 108.92 MB]
     [info] productdb_client module installed. [138.22 sec, 109.52 MB]
     [info] productdb_paragraphs module installed. [139.23 sec, 119.06 MB]
     [info] publication_date module installed. [140.3 sec, 130.55 MB]
     [info] redirect module installed. [141.41 sec, 141.22 MB]
     [info] schema_metatag module installed. [142.44 sec, 149.17 MB]
     [info] schema_article module installed. [143.48 sec, 159.83 MB]
    

    After

     [info] plugin module installed. [13.74 sec, 98.41 MB]
     [info] productdb_client module installed. [13.85 sec, 98.94 MB]
     [info] productdb_paragraphs module installed. [13.95 sec, 109.19 MB]
     [info] publication_date module installed. [14.08 sec, 109.74 MB]
     [info] redirect module installed. [14.22 sec, 105.78 MB]
     [info] schema_metatag module installed. [14.32 sec, 108.63 MB]
     [info] schema_article module installed. [14.43 sec, 110.34 MB]
    

    Oh I see.... this module results in the container being stored in static cache many many times... that's why the memory ballooned. But it should not even go in once!

    More work incoming.

  • Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MariaDB 10.3.22
    last update about 1 year ago
    Waiting for branch to pass
  • Status changed to Needs review about 1 year ago
  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ

    The current patch now results in less memory usage all the time:

    Before patch

     [info] plugin module installed. [14.28 sec, 98.56 MB]
     [info] prefetcher module installed. [14.45 sec, 96.48 MB]
     [info] preserve_page_cache module installed. [14.55 sec, 107.57 MB]
     [info] productdb_client module installed. [14.64 sec, 119.94 MB]
     [info] productdb_paragraphs module installed. [14.74 sec, 129.66 MB]
     [info] publication_date module installed. [14.85 sec, 141.18 MB]
     [info] redirect module installed. [14.98 sec, 152.28 MB]
     [info] schema_metatag module installed. [15.08 sec, 160.24 MB]
     [info] schema_article module installed. [15.18 sec, 173.11 MB]
    

    Before the commit to fix #8

     [info] plugin module installed. [14.57 sec, 98.55 MB]
     [info] prefetcher module installed. [14.73 sec, 100.8 MB]
     [info] preserve_page_cache module installed. [14.83 sec, 108.28 MB]
     [info] productdb_client module installed. [14.96 sec, 110.28 MB]
     [info] productdb_paragraphs module installed. [15.07 sec, 110.36 MB]
     [info] publication_date module installed. [15.22 sec, 110.81 MB]
     [info] redirect module installed. [15.37 sec, 106.63 MB]
     [info] schema_metatag module installed. [15.49 sec, 109.59 MB]
     [info] schema_article module installed. [15.62 sec, 110 MB]
    

    After

     [info] plugin module installed. [13.8 sec, 97.46 MB]
     [info] prefetcher module installed. [13.95 sec, 96.81 MB]
     [info] preserve_page_cache module installed. [14.07 sec, 97.65 MB]
     [info] productdb_client module installed. [14.19 sec, 99.53 MB]
     [info] productdb_paragraphs module installed. [14.32 sec, 99.62 MB]
     [info] publication_date module installed. [14.46 sec, 99.23 MB]
     [info] redirect module installed. [14.6 sec, 100.63 MB]
     [info] schema_metatag module installed. [14.73 sec, 98.8 MB]
     [info] schema_article module installed. [14.86 sec, 100.7 MB]
    
  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ

    I think we should add tests of installing multiple modules with plugin types so we can test that the caches work as expected.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    190 pass, 2 fail
  • Status changed to Needs work about 1 year ago
  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Rebased and reviewed.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    205 pass
  • Pipeline finished with Success
    about 1 year ago
    #82508
  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    197 pass, 16 fail
  • Pipeline finished with Failed
    about 1 year ago
    #82512
  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    205 pass
  • Pipeline finished with Success
    about 1 year ago
    #82529
  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    205 pass
  • Pipeline finished with Success
    about 1 year ago
    #82530
  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    WOW, those numbers! 🀯

  • Status changed to Needs review about 1 year ago
  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ
Production build 0.71.5 2024