Follow-up for #2752961: automatically deleting compiled Twig templates across multiple webheads

Created on 14 June 2018, over 6 years ago
Updated 4 January 2024, 11 months ago

Problem/Motivation

#2752961: No reliable method exists for clearing the Twig cache solved a big problem. See the CR at https://www.drupal.org/node/2908461 .

The remaining problem is now the deletion of stale (unused, irrelevant) compiled Twig templates on multi-webhead setups where each web server has its own compiled Twig template cache (because most shared file systems are slow).

The change record ( https://www.drupal.org/node/2908461 ) says:

Note: the new approach invalidates files but doesn't delete files across multiple webheads. If you are running Drupal in a multi webhead environment, you might want to delete stale files from the storage manually. To read more about a potential solution read comments #102–#123 at #2752961-102: No reliable method exists for clearing the Twig cache , which contains a couple of suggestions for potential strategies.
In other words: newly deployed Twig templates will be picked up as expected across all webheads (as long as \Drupal::service('twig')->invalidate(); is called after code deployment). But if each webhead is doing its own (local) caching, then stale compiled Twig templates will remain there. So over time, this could accumulate to a significant size.

(Emphasis added.)

Proposed resolution

@greg.1.anderson wrote at #2752961-121: No reliable method exists for clearing the Twig cache :

If I am not mistaken, then the purpose of #112 is to implement the solution proposed in #7. In terms of converting a single-head solution into a double-head solution, what hosting providers need is:

1. The ability to put the twig cache files in a completely different directory each time they are cleared.
- When our scavenger deletes stale cache files, it can just delete entire directories at a time.
2. The ability to know which directory the current cache files are stored in.
- When scavenging stale cache files, we want to skip the directory that is active.
- We currently do this by forcing the directory calculation used in 1 so that we do not need to query Drupal.
- We could also potentially do this simply by examining the creation date of the cache folder, and presume that the newest is the active one.
3. The ability to stop Drupal from deleting the files in the twig cache
- If we have a scavenger, then we don't need to spend time during the web request to delete large directories.

Remaining tasks

TBD

User interface changes

None.

API changes

TBD

Data model changes

TBD

📌 Task
Status

Active

Version

11.0 🔥

Component
Theme 

Last updated about 20 hours ago

Created by

🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

Live updates comments and jobs are added and updated live.
  • Needs issue summary update

    Issue summaries save everyone time if they are kept up-to-date. See Update issue summary task instructions.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇧🇷Brazil dungahk Balneário Camboriú

    @Eric_A That issue is different, I initially thought they were the same too, but this one is specifically about making sure inactive Twig compiled files are deleted across multiple webheads, that other issue did not fix this, one still needs to delete them separately.

Production build 0.71.5 2024