Crop API is not appending a hash when the image styles are converted to WEBP

Created on 1 July 2022, over 2 years ago
Updated 21 August 2023, over 1 year ago

Problem/Motivation

Steps to reproduce

1. install and set the image_widget_crop โ†’
2. for the sake of demonstration change the thumbnail image style or use this guide: https://gorannikolovski.com/blog/drupal-92-will-support-webp-images-out-box
3. add Convert effect
4. set WEBP as Extension
5. upload image and set a crop for it
6. display it in this image style in a node field

When we don't add the WEBP convert to the image style, these kind of URLs are generated:

/sites/default/files/styles/thumbnail/public/2022-07/landscape-wallpaper-340.jpg?h=9d04aecf&itok=45Abo5kO
/sites/default/files/styles/thumbnail/public/2022-07/landscape-wallpaper-340.jpg?h=39e9e1d9&itok=45Abo5kO

But if we convert our image styles to WEBP, the URLs look like these:

/sites/default/files/styles/thumbnail/s3/2022-07/landscape-wallpaper-340.jpg.webp?itok=aLAJKJ0O
/sites/default/files/styles/thumbnail/s3/2022-07/landscape-wallpaper-340.jpg.webp?itok=aLAJKJ0O

Why is this a problem?
We're caching each URLs with CDN or with S3 bucket. If I change the crop, then the newly cropped image doesn't appear to the visitors until the cache gets invalidated. For one site we're serving image files from Google Cloud Storage. There by default every requested URL is cached for 60 minutes. If a cropping change doesn't change anything in an image URL, then again the same problem happens.

Proposed resolution

In crop.module this line checks if there's a crop available for an image:

if ($crop = Crop::getCropFromImageStyleId($file_uri, $image_style_id)) {

There the $file_uri will contain this:

public://styles/media_library/public/2022-07/landscape-wallpaper-340.jpg.webp

Add an extra check that if that doesn't give crop results, then also check it without the webp extension like this:

public://styles/media_library/public/2022-07/landscape-wallpaper-340.jpg

Remaining tasks

User interface changes

API changes

Data model changes

๐Ÿ› Bug report
Status

RTBC

Version

2.0

Component

Code

Created by

๐Ÿ‡ธ๐Ÿ‡ฐSlovakia kaszarobert

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States recrit

    This patch with working for me. Thanks @kaszarobert!

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States recrit
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States chrissnyder Maryland

    Does this impact those using the contrib WebP โ†’ module and not using core's "Convert" image style effect?

  • ๐Ÿ‡จ๐Ÿ‡ญSwitzerland berdir Switzerland

    Implementation looks OK now. Uses camelCase for the variable name, doesn't really fit in with the rest in the same function (coding standards say you can use camelCase, but the whole file should be consistent). But that's minor.

    A test for this would be good. based on our tests, this is also required for the core convert option, so it should be possible adjust/duplicate the existing tests when they also have a convert operation.

  • ๐Ÿ‡ช๐Ÿ‡ธSpain pcambra Asturies

    Confirming RTBC, in my case crop_file_url_alter was adding a h query parameter and then FileUrlGeneratorInterface::generateAbsoluteString was not adding the itok param because there was something already in there.

  • ๐Ÿ‡ต๐Ÿ‡ฑPoland szy

    Hello there.

    I'm using these two lines in settings.php:

    $config['image.settings']['suppress_itok_output'] = TRUE;
    $config['image.settings']['allow_insecure_derivatives'] = TRUE;

    Does it matter for this patch?

    Now, despite your patch, converting to WEBP still makes crop disappear (using a cropped image in a view).

    D 10.2.2, Crop API 8.x-2.3, ImageWidgetCrop 8.x-2.4, Image Effects 8.x-3.6.

    Thanks!

    Szy.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom rossb89 Bristol

    Patch is working great, thanks!

  • ๐Ÿ‡ต๐Ÿ‡ฑPoland besek

    I can confirm, patch from #4 works perfect, thanks!

  • Status changed to Needs review 6 months ago
  • ๐Ÿ‡จ๐Ÿ‡ญSwitzerland sraLton

    I have this issue with conversion to PNG too.
    Therefore I propose a more generic solution to pass the conversion type directly in to the function.
    I haven't tested this patch with Webp-Files.

  • Open in Jenkins โ†’ Open on Drupal.org โ†’
    Core: 10.2.x + Environment: PHP 8.2 & MySQL 8
    last update 6 months ago
    7 pass
  • ๐Ÿ‡ซ๐Ÿ‡ทFrance DrDam

    Hi,

    I think I've found a solution that I think is โ€˜simplerโ€™ (less intrusive and based on what the core does in generating thumbnails).

    The main idea is to retrieve the code from the ImageStyleDownloadController::deliver :

        // Don't try to generate file if source is missing.
        if (!$this->sourceImageExists($image_uri, $token_is_valid)) {
          // If the image style converted the extension, it has been added to the
          // original file, resulting in filenames like image.png.jpeg. So to find
          // the actual source image, we remove the extension and check if that
          // image exists.
          $path_info = pathinfo(StreamWrapperManager::getTarget($image_uri));
          $converted_image_uri = sprintf('%s://%s%s%s', $this->streamWrapperManager->getScheme($derivative_uri), $path_info['dirname'], DIRECTORY_SEPARATOR, $path_info['filename']);
          if (!$this->sourceImageExists($converted_image_uri, $token_is_valid)) {
    

    and inject this controle in the crop::getCropFromImageStyleId method

  • ๐Ÿ‡ซ๐Ÿ‡ทFrance DrDam

    In Drupal 10.3+ , we have a new static method to do that

    $converted_original_uri = ImageStyleDownloadController::getUriWithoutConvertedExtension($original_uri);
    

    I'm open a merge request to add this

  • @drdam opened merge request.
  • ๐Ÿ‡ซ๐Ÿ‡ฎFinland hartsak

    I don't exactly know how it works, but the solution from #15 helped me out.

    My problem was Cloudflare which was displaying the old image even after Focal point was saved (using Focal point essentially crops the image). And by adding the fix from #15 the image uri receives the missing ?h parameter.
    I'm also converting the image to webp format which seems to be one of the reasons for this problem. Without webp conversion the ?h parameter appears, but with the patch I'm able to get the ?h parameter there even with webp conversion enabled.
    I'm using Drupal 10.3.9 and Crop API 8.x-2.4.

  • Status changed to RTBC 19 days ago
  • ๐Ÿ‡ซ๐Ÿ‡ทFrance DrDam

    Thanks @hartsak, I pass the ticket to RTBC

  • ๐Ÿ‡ท๐Ÿ‡บRussia zniki.ru

    nikolay shapovalov โ†’ made their first commit to this issueโ€™s fork.

  • @nikolay-shapovalov opened merge request.
  • ๐Ÿ‡ท๐Ÿ‡บRussia zniki.ru

    I create test to demonstrate the issue, created Draft MR #21 to run pipelines.
    Also applied changes from MR #21 to MR #20 to test changes.

    My suggestion to create followup to refactor CropFunctionalTest::doTestFileUriAlter().

    $this->assertTrue(strpos($image_style_url, $shortened_hash) !== FALSE, 'The image style URL contains a shortened hash.');
    // Suggestion.
    $this->assertStringContainsString($shortened_hash, $image_style_url, 'The image style URL contains a shortened hash.');
    
  • ๐Ÿ‡ท๐Ÿ‡บRussia zniki.ru

    Hide patch files.

  • ๐Ÿ‡ซ๐Ÿ‡ทFrance DrDam
Production build 0.71.5 2024