GD toolkit & operations should fail gracefully in case of memory issues

Is there proposed solution?

@andypost not really, no. In 🐛 GD toolkit & operations should catch \Throwable to fail gracefully in case of errors Fixed I was proposing to pre-calculate the memory required before executing the operation and fail in case it would exceed the available memory reported by PHP, but that proposal did not get through, https://www.drupal.org/project/drupal/issues/2583041#comment-14725453 🐛 GD toolkit & operations should catch \Throwable to fail gracefully in case of errors Fixed .

Actually, when 🐛 GD toolkit & operations should catch \Throwable to fail gracefully in case of errors Fixed was opened, we were in PHP 5 and the failure was quite evident. We're in PHP8land now and it seems this is occurring less, but still sometimes it does. #2583041-124: GD toolkit & operations should catch \Throwable to fail gracefully in case of errors → is a case when the error occurred in a PostgreSql based test runner, when all other tests run fine. So it maybe something related to PHP build, but that will be hard to find.

So, to start right with, we probably need new steps to reproduce.

both tests somehow passed

No, PHP8.1 with pgsql fails. What has the db to do with GD failure who knows.

Uhm. pgsql fails in StatementWrapper... not in GD... could be this has to do with test's tear down and db cleanup. It would interesting to check this combined with 🐛 Implement statement classes using \Iterator to fix memory usage regression and prevent rewinding Fixed .

The last commit in the MR now shows the failure.

Of the two test cases,

(1)

    $this->assertFalse($image->createNew(2000000, 2000000));

is actually false because createNew fails and fires a warning like imagecreatetruecolor(): product of memory allocation multiplication would exceed INT_MAX, failing operation gracefully

while
(2) the second test (limiting the memory available for PHP and trying to create an image that goes beyond that), fails with the memory error

PHP Fatal error:  Allowed memory size of 13631488 bytes exhausted (tried to allocate 61440 bytes) in /var/www/html/core/modules/system/src/Plugin/ImageToolkit/Operation/gd/CreateNew.php on line 89
Fatal error: Allowed memory size of 13631488 bytes exhausted (tried to allocate 61440 bytes) in /var/www/html/core/modules/system/src/Plugin/ImageToolkit/Operation/gd/CreateNew.php on line 89

Now, while I can reproduce (1) also locally, I can't reproduce (2).

I suspect that is because in my testing environment I have a PHP built with external GD library on 2.3.3, whereas the one used by DrupalCI bots uses the embedded GD library that is equivalent to the 2.1 version (IIRC). Given this https://github.com/libgd/libgd/issues/122 happened in 2.3, it could be that this explains the different behaviour.

From the linked libgd issue, it appears this problem could be solved by adding 64-bit support, unless I'm totally misinterpreting things. It's something I've come across a few times, on images that don't appear at first to be that large.

I think I am running into this on what doesn't seem like a very large image (5100 × 3404 and 471K). The linked GD issue calls out some extremely large dimensions (specifically 60000x60000). But I am running into it on an Acquia host, with a memory_limit of 192M. With that said, it doesn't replicate in my local ddev environment with a matching memory limit.

I am running standalone GD 2.3.3 in ddev.

Acquia uses an older GD, 2.1.0 bundled with FreeType. The FreeType version they use (as of this comment, on the server I'm looking at) is 2.6.1. Is this even supported? I don't even see an available release for that at https://download.savannah.gnu.org/releases/freetype/?C=N&O=A

Are other's running into this issue exclusively on Acquia? Perhaps they just need to update FreeType/GD? But I'm finding it difficult to find documentation on what GD versions are bundled with what FreeType versions.

@bburg we also run in such issues without Acquia hosting, but we also use GD 2.1.0 in our current Docker images. Also uploaded images are in allowed dimension and sizes. In most cases I recognized that users uploaded images with a very special color schema. When I convert these uploaded images than to something like sRGB or RGB color schema all is working fine with them.

GD-Bibliothek
bundled (2.1.0 compatible)
Supported image file formats: GIF, JPEG, PNG.
Unsupported image file format: WEBP.
Check the PHP GD installation documentation if you want to add support.

I could also double check with GD 2.3.x in our local DDEV environment if we also run in such memory issues with newer GD toolkit version.

I believe we are seeing some variation of this from time to time.
We are seeing it on images based on php:fpm-alpine, but unable to reproduce the issues in DDEV where the image is based on debian:slim.
Main difference there is Alpine use musl vs Debian is glibc.
Haven't had a chance to test yet, but I wonder if what we see can be related to what is discussed in https://github.com/php/php-src/issues/13877

OK, I was wrong in my last assumption.
As already mentioned by @mondrake, the memory exhaustion issue we see, appear to be related to GD version bundled with PHP and can be resolved with building the GD extension using the "--with-external-gd" flag.

I.e on debian based something like this:

...
apt-get install ... libgd-dev ... \
&& docker-php-ext-configure gd --with-freetype --with-jpeg --with-webp --with-avif --with-external-gd
...

And on Alpine based:

...
apk add --no-cache ... gd-dev ... \
&& docker-php-ext-configure gd --with-freetype --with-jpeg --with-webp --with-avif --with-external-gd
...