Entity access check results can cause memory exhaustion during sitemap generation

Created on 11 April 2025, 4 months ago

Problem/Motivation

Entity access check results are stored in a static cache and affect memory consumption during sitemap generation. See \Drupal\Core\Entity\EntityAccessControlHandler::setCache() for details.

Steps to reproduce

Use performance_test.php and check the memory consumption when increasing the number of entities in the sitemap.

Proposed resolution

Clear cached access checks during generation.

📌 Task
Status

Active

Version

4.0

Component

Code

Created by

🇷🇺Russia walkingdexter

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @walkingdexter
  • 🇷🇺Russia walkingdexter

    Running performance_test.php with 55k entities.

    Before:

     [info] Running memory usage tests:
     [info] Executing: /var/www/vendor/bin/drush batch-process 1238 --uri=http://example.com
    >  [notice] Memory: 97.2 MB, non-peak mem: 90.5 MB
    >  [notice] Memory: 122.45 MB, non-peak mem: 114.5 MB
    >  [notice] Memory: 137.77 MB, non-peak mem: 127.5 MB
    >  [notice] Memory: 150.63 MB, non-peak mem: 137.5 MB
    >  [notice] Memory: 156.25 MB, non-peak mem: 143.5 MB
    >  [notice] Memory: 166.26 MB, non-peak mem: 153.5 MB
    >  [notice] Memory: 166.26 MB, non-peak mem: 153.5 MB
     [info] Generation completed in: 95.36 seconds
     [info] Executing: /var/www/vendor/bin/drush batch-process 1239 --uri=http://example.com
    >  [notice] Memory: 94.5 MB, non-peak mem: 92.5 MB
    >  [notice] Memory: 146.19 MB, non-peak mem: 135.5 MB
    >  [notice] Memory: 168 MB, non-peak mem: 163.5 MB
     [info] Generation completed in: 44.71 seconds
     [info] Running query count tests:
     [info] Executing: /var/www/vendor/bin/drush batch-process 1240 --uri=http://example.com
    >  [notice] Query count: 20212
    >  [notice] Query count: 40563
    >  [notice] Query count: 56204
    >  [notice] Query count: 67989
    >  [notice] Query count: 79562
    >  [notice] Query count: 91456
    >  [notice] Query count: 103885
    >  [notice] Query count: 116314
    >  [notice] Query count: 118477
     [info] Generation completed in: 127 seconds
     [info] Executing: /var/www/vendor/bin/drush batch-process 1241 --uri=http://example.com
    >  [notice] Query count: 21904
    >  [notice] Query count: 61511
    >  [notice] Query count: 97748
    >  [notice] Query count: 112545
     [info] Generation completed in: 52.29 seconds
    

    After:

     [info] Running memory usage tests:
     [info] Executing: /var/www/vendor/bin/drush batch-process 1253 --uri=http://example.com
    >  [notice] Memory: 72.5 MB, non-peak mem: 68.5 MB
    >  [notice] Memory: 80.42 MB, non-peak mem: 72.5 MB
    >  [notice] Memory: 84.71 MB, non-peak mem: 73 MB
    >  [notice] Memory: 84.71 MB, non-peak mem: 73 MB
    >  [notice] Memory: 89.24 MB, non-peak mem: 75 MB
    >  [notice] Memory: 89.24 MB, non-peak mem: 75 MB
    >  [notice] Memory: 89.24 MB, non-peak mem: 75 MB
     [info] Generation completed in: 93.93 seconds
     [info] Executing: /var/www/vendor/bin/drush batch-process 1254 --uri=http://example.com
    >  [notice] Memory: 86.5 MB, non-peak mem: 84.5 MB
    >  [notice] Memory: 97.25 MB, non-peak mem: 87 MB
    >  [notice] Memory: 97.25 MB, non-peak mem: 89 MB
     [info] Generation completed in: 42.31 seconds
     [info] Running query count tests:
     [info] Executing: /var/www/vendor/bin/drush batch-process 1255 --uri=http://example.com
    >  [notice] Query count: 20426
    >  [notice] Query count: 42275
    >  [notice] Query count: 55988
    >  [notice] Query count: 67882
    >  [notice] Query count: 79134
    >  [notice] Query count: 91135
    >  [notice] Query count: 103350
    >  [notice] Query count: 115028
    >  [notice] Query count: 118477
     [info] Generation completed in: 129.34 seconds
     [info] Executing: /var/www/vendor/bin/drush batch-process 1256 --uri=http://example.com
    >  [notice] Query count: 22006
    >  [notice] Query count: 64367
    >  [notice] Query count: 103872
    >  [notice] Query count: 112545
     [info] Generation completed in: 49.01 seconds
    
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024