Twig cache invalidated almost every request with multiple web servers caching on shared storage

Created on 21 October 2019, about 5 years ago
Updated 5 March 2024, 10 months ago

A bug is occurring where the twig cache directory is getting a unreasonable amount of files (20k+) and performance crawls to a halt, eventually making the server unresponsive.

The problem is caused by a very significant set of circumstances as outlined:

  1. The Drupal application is being hosted across multiple application servers.
  2. The application servers share database and twig cache directories but not application code (e.g. each application server code is deployed by git pull separately per server)
  3. Users are hitting URLs across all application servers that are hitting uncached twig files.

The problem exists in the code https://git.drupalcode.org/project/drupal/blob/8.8.x/core/lib/Drupal/Cor... such that if the timestamps in both application servers don't match, the appservers fight with setting the twig_extension_hash and the cached files are continuously reset. Depending on load and when the twig cache cleanup task runs, it's not uncommon to see 20k+ files in this directory.

Changing filemtime to md5_file fixes the issue but I'm not 100% sure of the performance impact, though I have been running the fix for 6+ months in production with no issues.

🐛 Bug report
Status

Active

Version

11.0 🔥

Component
Theme 

Last updated 7 minutes ago

Created by

🇦🇺Australia nterbogt

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024