Only use coalesceImages() when needed

Created on 1 February 2024, 5 months ago
Updated 16 February 2024, 4 months ago

I have a project that uses the Imagick toolkit. Clients can upload images.
I received a message from a client that she was unable to upload an image.
The specific image was 12 000 x 11 234 pixels and was 10,6 MB.
The error displayed on screen is:
Oops, something went wrong. Check your browser's developer console for more details.
The error in the console is:
POST https://mywebsite.com/media/add/image?element_parents=field_media_image/widget/0&ajax_form=1&_wrapper_format=drupal_ajax 500 (Internal Server Error)
and:

"
An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: /media/add/image?element_parents=field_media_image/widget/0&ajax_form=1
StatusText: error
ResponseText: The website encountered an unexpected error. Try again later."

See screenshot 1 for this.

When this error message is shown, it is usually accompanied by the following error in watchdog:
ImagickException: CacheResourcesExhausted `/data/sites/web/mywebsitecom/production/private/assets/images/test-11000x10298_0.jpg' @ error/cache.c/OpenPixelCache/4095 in Imagick->coalesceImages() (line 205 of /data/sites/web/mywebsitecom/production/web/modules/contrib/imagick/src/Plugin/ImageToolkit/ImagickToolkit.php).

The image file size does not exceed any php or drupal limits.
There are no max or min resolutions set in Drupal.
I did more testing and uploading the same image on a local copy of the website is no problem.
So the problem exists on the production server, but not on my local machine.
The production server is limited to 1024MB memory. I cannot increase that.
The server is PHP 8.1 just like my local machine. I tried changing it to php 8.2 but that makes no difference.

To see what resolution does work, I created variants of the image of 5000, 6000, 7000, 8000, 9000, 10 000 and 11000 pixels wide.
The 5000, 6000 and 7000 pixels wide variants upload fine.

When searching the internet the solution seems to be to modify imagemagicks policy.xml file.
I spend all day testing all different values for memory, map, width, height, area, disk, file, thread, throttle, ... resources in the XML file.
Via ssh and the magick identify -list policy command I verifiy that my custom policy.xml file is in use.
With all the adjustments, the maximum image I could get uploaded at the end of the day was the 8000 pixel variant.
Larger image uploads usually gave me either an error like in screenshot 1 with about the same watchdog error, or an error like screenshot 2 which makes it look like the problem that the file extension is not valid. But that is a big lie. When that error is displayed, there is no console and no watchdog messages.

So after a whole day of playing with the values of the policy.xml file, i couldstill not upload an image with a resolution of more than 8000 pixels. So I looked coser to the error itself:
What does Imagick->coalesceImages() do exactly and why does it prevent me from uploading an jpg image ?
According to this, this is what the coalesceImages function does:
Composites a set of images while respecting any page offsets and disposal methods. GIF, MIFF, and MNG animation sequences typically start with an image background and each subsequent image varies in size and offset. Returns a new Imagick object where each image in the sequence is the same size as the first and composited with the next image in the sequence.

So I wondered: a .jpg file is not 'a set of images' is it ? So for testing, I edited web/modules/contrib/imagick/src/Plugin/ImageToolkit/ImagickToolkit.php and changed:
return $this->getResource()->coalesceImages()->getImageGeometry()['width'];
into:
return $this->getResource()->getImageGeometry()['width'];
and
return $this->getResource()->coalesceImages()->getImageGeometry()['height'];
into:
return $this->getResource()->getImageGeometry()['height'];

And I tried again uploading all my different resolution image vaiants.
This time I could successfully upload the 9000, 10 000 and 11 000 pixel variants.
I must admit that I was still unable to upload the original 12 000 pixel variant, that resulted in screenshot 2 so without any console or watchdog messages. But I find boosting up the maximum image width from 8000 to 11000 a big difference.

So I think coalesceImages() must only be used for file extensions where it makes sense like GIF, and not for every uploaded file like JPG.

πŸ› Bug report
Status

Fixed

Version

1.0

Component

Code

Created by

πŸ‡§πŸ‡ͺBelgium flyke

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @flyke
  • πŸ‡§πŸ‡ͺBelgium flyke
  • πŸ‡§πŸ‡ͺBelgium flyke

    Maybe we could add a check like one that already exists at:
    web/modules/contrib/imagick/src/Plugin/ImageToolkit/Operation/imagick/ImagickOperationTrait.php line 36:

        if (isset($image_format) && in_array($image_format, ['GIF', 'WEBP'])) {
          // Get each frame in the GIF
          $resource = $resource->coalesceImages();
          do {
            if (!$this->process($resource, $arguments)) {
              $success = FALSE;
              break;
            }
          } while ($resource->nextImage());
    
          $resource->deconstructImages();
        }

    And then we could change inside ImagickToolkit.php this:

      public function getWidth() {
        if (!$this->getResource()) {
          return NULL;
        }
    
        return $this->getResource()->coalesceImages()->getImageGeometry()['width'];
      }

    into something like:

      public function getWidth() {
        $resource = $this->getResource();
        if (!$resource) {
          return NULL;
        }
    
        // If preferred format is set, use it as prefix for writeImage
        // If not this will throw a ImagickException exception
        try {
          $image_format = $resource->getImageFormat();
        } catch (ImagickException $e) {}
    
        // Use coalesceImages() for images with multiple frames.
        if (isset($image_format) && in_array($image_format, ['GIF', 'WEBP'])) {
          return $this->getResource()->coalesceImages()->getImageGeometry()['width'];
        }
    
        // Otherwise, get with without using coalesceImages().
        return $this->getResource()->getImageGeometry()['width'];
      }

    and the same goes for the getHeight() function.

  • πŸ‡§πŸ‡ͺBelgium flyke

    Attempt at a patch.

  • Status changed to Fixed 5 months ago
  • πŸ‡§πŸ‡ͺBelgium bceyssens Genk πŸ‡§πŸ‡ͺ

    thank you for the patch @flyke

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.69.0 2024