Page and block caches are cleared the same way on cron as on content change

Created on 24 August 2010, over 14 years ago
Updated 16 March 2023, almost 2 years ago

For more background, see http://www.metaltoad.com/blog/how-drupals-cron-killing-you-your-sleep-si..., which was written for D6, but the same problem exists in D7. Here's a summary:

  • system_cron() calls cache_clear_all(NULL, $table) for a bunch of tables, including 'cache_page'.
  • comment_form_submit(), node_form_submit() and a bunch of other functions call cache_clear_all() with no arguments, which results in cache_clear_all(NULL, 'cache_page') being called.
  • Records in cache_page have the expire field set to CACHE_TEMPORARY, because that's what drupal_page_set_cache() sets it to.
  • The default class that implements the 'cache_page' cache bin is DrupalDatabaseCache, whose clear() method when NULL is passed as $cid deletes all records with an expire of CACHE_TEMPORARY.
  • There is a cache_lifetime system variable, settable on the admin/config/performance page, that can be used to ensure that DrupalDatabaseCache::clear() doesn't run more than once within the corresponding interval, regardless of how often cron runs or website content gets changed.

This setup doesn't make sense to me. If I setup cron.php to run once an hour, why should my page cache be emptied each hour? Maybe I have a site whose content is only changed once per week on average: why should my page cache be emptied on cron at all? And while, yes, I can tune the cache_lifetime variable to be longer than my cron interval, doing so also means that when content is changed (i.e., a node or comment form is submitted), then my page cache really is stale for up to cache_lifetime.

And while, yes, I can override DrupalDatabaseCache with some other class, I'm not sure that helps, since I don't think it would have any way to distinguish whether clear() was called due to a node/comment submit or to a cron run.

Does anyone have thoughts on this? Knowledge of why system_cron() clears caches at all?

Related issues:
#1279654: Page cache is CACHE_TEMPORARY and does not honor cache_lifetime β†’

πŸ› Bug report
Status

Needs review

Version

7.0 ⚰️

Component
CacheΒ  β†’

Last updated 2 days ago

Created by

πŸ‡ΊπŸ‡ΈUnited States effulgentsia

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

  • Contributed project blocker

    It denotes an issue that prevents porting of a contributed project to the stable version of Drupal due to missing APIs, regressions, and so on.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ΊπŸ‡ΈUnited States brad.bulger

    Yet another reroll to bump the fake update hook number for 7.95.

  • πŸ‡ΊπŸ‡ΈUnited States brad.bulger

    Is continually bumping up this fake update_N hook in the system module preventing the real hooks from running? Like, d7-cache_garbage_collection-891600-119.patch defined system_update_7086(). Now core Drupal 7.98 has a real system_update_7086() function. But won't my installation think that it has already run?

  • πŸ‡ΈπŸ‡°Slovakia poker10

    Yes, that is a good point, thanks!

    In case someone has run the update hook from this patch, it is likely that the real system_update_7086 introduced by πŸ“Œ Improve security of session ID against DB exposure or SQL injection Fixed in Drupal 7.98 will not run. This will cause that these sites would not use the security improvement, because the session hashing will not be enabled. See: https://www.drupal.org/node/3364841 β†’

    I recommend that all sites using this patch should carefully check that change record and a committed patch and take measures to ensure that they have the session hashing enabled.

  • πŸ‡ΊπŸ‡ΈUnited States brad.bulger

    I've never been quite clear about the system_update_N function in this patch. It tries to find all existing cache tables and replaces any existing "expire" index with an "expire_created" index. Running in an update function like this, that's a one-time check, right? If new cache tables are created with an "expire" index, this would do nothing about that. I don't know if that would happen. But if a one-time check will suffice, then it seems like instead of interfering with the real update hooks of the system module, that check should move somewhere else.

  • πŸ‡ΈπŸ‡°Slovakia poker10

    There are two changes in the system.install file. Change in system_schema:

    function system_schema() {
         ...
         'indexes' => array(
    -      'expire' => array('expire'),
    +      'expire_created' => array('expire', 'created'),
    

    And the new update function system_update_7086().

    These two changes cover two usecases - if the site already has cache tables, then the update hook will update the indexes there. On installation (or creation of a new cache_xx table) the system_schema() comes to play and it will ensure that all new tables would have expire_created index. This is because all other cache tables are using the main cache table definition. See:

    $schema['cache_block'] = drupal_get_schema_unprocessed('system', 'cache');
    
    $schema['cache_form'] = $schema['cache'];
    

    So both changes in the install file are needed - one for new installations (and new cache tables) and the update hook for existing tables.

  • πŸ‡ΊπŸ‡ΈUnited States brad.bulger

    OK, so if future cache table use (or are supposed to use) the basic cache table schema, then there's no need to redo any of these changes for future updates. Is that right?

    It seems like putting the update hook into a new module would be a better way to handle that. There are some alternate caching modules mentioned above, it's over my head to evaluate them though.

  • πŸ‡ΈπŸ‡°Slovakia poker10

    Yes, that should be correct. In case the full patch was/is still applied, all existing cache tables should have been updated in the past and new are created by the system_schema() code. So there is no need to run the update hook again. Sites just needs to ensure that the new system_update_7086() from D7.98 have run.

  • last update over 1 year ago
    Drush setup of Drupal Failed
  • πŸ‡ΊπŸ‡ΈUnited States brad.bulger

    I've moved the code to replace 'expire' indexes with 'expire_create' from the fake hook_update_N function into system_requirements(). In the runtime phase (eg when you load the Status report on your site), it will report a warning that the update script needs to be run if it finds at least one 'expire' index on a cache table. In the update or install phases, if it finds an 'expire' index on a cache table, it will drop it and then create an 'expire_create' index if one does not already exist, reporting all of these actions to the user. So running drush updatedb or the update.php page should clean up the indexes even if there are no update hooks needing to be run.

    I reset the system module schema number back to before the first hook from this patch that we applied to our site - 7074 - and reran all of the real system module hooks since then, without much incident. YMMV

  • πŸ‡ΊπŸ‡ΈUnited States brad.bulger

    reroll to apply to 7.103

Production build 0.71.5 2024