Purge image derivatives

Created on 17 June 2024, 7 months ago
Updated 15 September 2024, 4 months ago

Problem/Motivation

We are using the module for invalidating cached files and images in Cloudflare. It works correctly in these cases, but one scenario has not been considered: the derivative images that are generated when there are possibilities of multiple image styles. Those cases remain cached.

The module Varnish Image Purge, Varnish Purge submodule, implements this functionality, perhaps we can adapt this approach in the purge file module.

Proposed resolution

Implement a new function in purge_file.module that included the image derivative:

  • Check that the object file is a image object.
  • Get the different image styles and the uris for the object file.
  • Check if the image derivative exits to try to reduce the number of petitions to the API.
  • Add the image derivative URLs to the list of URLs that need to be purged
Feature request
Status

Active

Version

1.0

Component

Code

Created by

🇪🇸Spain lpeidro Madrid

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @lpeidro
  • 🇪🇸Spain omarlopesino

    This problem should have been fixed at 1.1.0 with this issue: https://www.drupal.org/project/purge_file/issues/3308665 Include Query String Variants in Purge Request Fixed . May you enable the wildcard option at the purge file configuration, and check if that solves the issue?

    If that solves the problem, we could improve the documentation to explain how to solve problems with image style derivatives.

  • 🇪🇸Spain lpeidro Madrid

    The wildcard functionality is not sufficient to purge the image files generated by image styles. In this case, the URLs of the derived images vary in their middle structure, so a wildcard at the end of the original image URL would not apply to the derived ones.

    The derived images are not constituted as file entities, so when the file entity is deleted, the Styles module directly deletes all the files that have been generated that are derived images.

    I have created a proposal that has a fundamental problem related to the number of image styles that may exist in an installation.

    When there are many image styles, the system to check if they exist can take 2-3 seconds. This is because the check is done exclusively in the context of deleting the entity and by making a HEAD request to the image URL. If the image is cached and needs to be purged, it will return a 200 status; otherwise, it can be omitted since it does not exist in the cache. This check is done to avoid send unnecessary requests to the API.

    Perhaps this could be solved by implementing queues, and there is an issue about this.

    On the other hand, maybe is not so serious to send requests to the API asking to remove cache objects that does not exists.

    Anyway, I think that the implementation of queues should be mandatory for this implementation, to avoid that in this border cases (sites with many Image Styles) can happen bugs.

  • Issue was unassigned.
  • 🇦🇺Australia dpi Perth, Australia

    To me, it sounds like this is not within the scope of this project.

    There should be a project dedicated to image derivatives, or Purge itself should support it, since it is a core feature.

  • 🇪🇸Spain jonhattan Plasencia

    Some thoughs:

    1. Implement a wildcard purge based on the styles folder: `public://styles/*/filename-path` (not sure now but I think it may be generalized to use the default stream wrapper instead of hardcoding public://)

    Note: wildcard purge is not available in all cache/cdn solutions, so it must be a configurable option and fallback to iterating all image styles.

    2. I think asking to purge images in despite of they don't exists is better from a performance viewpoint, because a HEAD on inexistent resource in the cache will trigger a call to the backend... ending in a hit to the db server.

    3. I think it should be a submodule and avoid mixing image specific stuff with file basic stuff. Perhaps

    4. To check file is an image use `Image::isValid` as in:

    $image_factory = \Drupal::service('image.factory');
    $image = $image_factory->get($file->getFileUri());
    if ($image->isValid()) {
     // ...
    }
    
Production build 0.71.5 2024