Does not work with Object Storage accessed via Stream Wrappers (FlySystem)

Created on 16 January 2021, almost 4 years ago
Updated 9 September 2024, 4 months ago

When configuring a Media Entity to utilize Remote Object Storage (such as AWS S3) using a StreamWrapper such as FlySystem, this module is generating an uncaught PHP Exception:

Uncaught PHP Exception Drupal\\Core\\Database\\DatabaseExceptionWrapper: "SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction: INSERT INTO {cache_discovery} ("cid", "expire", "created", "tags", "checksum", "data", "serialized") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6) ON DUPLICATE KEY UPDATE "cid" = VALUES("cid"), "expire" = VALUES("expire"), "created" = VALUES("created"), "tags" = VALUES("tags"), "checksum" = VALUES("checksum"), "data" = VALUES("data"), "serialized" = VALUES("serialized"); Array\n(\n    [:db_insert_placeholder_0] => last_write_timestamp_cache_discovery\n    [:db_insert_placeholder_1] => -1\n    [:db_insert_placeholder_2] => 1610762807.437\n    [:db_insert_placeholder_3] => \n    [:db_insert_placeholder_4] => 0\n    [:db_insert_placeholder_5] => d:1610762807.437;\n    [:db_insert_placeholder_6] => 1\n)\n" at /var/www/html/web/core/lib/Drupal/Core/Database/Connection.php line 896, referer: https://www.example.com.com/media/add

If the module is disabled, the message does not occur.

If you are using the public:// file system for file storage, the thumbnail is generated successfully.

PHP 7.4, Drupal 9.1, ImageMagick 6.9.10-23, GhostScript 9.27

πŸ› Bug report
Status

Fixed

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States lhridley

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ΊπŸ‡ΈUnited States jsutta United States

    Hello,

    On the site I'm working on currently, we use the s3fs module to take over the public/private file system and store files in an S3 bucket, with the temporary file directory still local (see https://www.drupal.org/project/s3fs/issues/3325298 β†’ for why the temporary directory should remain local).

    As of today we're using Drupal 10.0.9, version 8.x-1.0-beta1 of Media Thumbnails, and version 2.0.0-beta2 of this module. On my site I have a requirement to generate thumbnails of PDF files as they're uploaded, which is where our use of this module originated. However, with ImageMagick's limitation around stream wrappers I had to find a solution for our site. Now that my patch is working and I've completed functional testing, I'm attaching it to this issue since it was created first and may be of use to others. It's based on a patch from the ImageMagick module's issue queue ( Allow ImageMagick to work with remote streamwrappers (s3) β†’ ). The patch was originally written for Drupal 7, so I had to update it for Drupal 8/9/10.

    Some notes from my experience setting up this module:

    • Make sure to install ghostscript so ImageMagick can read PDFs (see https://www.php.net/manual/en/imagick.requirements.php for more information)
    • If thumbnails aren't being generated and you see a message like this in the log: cache resources exhausted `/tmp/magick-fCoRMvS5SPEH7_klIKyId3VgmNM24XZL346' @ error/cache.c/OpenPixelCache/4095, you need to increase ImageMagick's memory allocation. To do this, find ImageMagick's policy.xml file (for reference, mine is located at /etc/ImageMagick-6/policy.xml) and change the following settings, depending on your needs:
      • <policy domain="resource" name="memory" value="256MiB"/> - I increased my value to 2GiB
      • <policy domain="resource" name="disk" value="1GiB"/> - I increased my value to 2GiB
    • If thumbnails aren't being generated and you see a message like this in the log: attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/421, you need to change ImageMagick's security policy to allow it to access PDFs (see https://stackoverflow.com/questions/52998331/imagemagick-security-policy...). To do this, find ImageMagick's policy.xml file (for reference, mine is located at /etc/ImageMagick-6/policy.xml) and uncomment the line that looks like this: <policy domain="module" rights="read|write" pattern="{PS,PDF,XPS}" /> (the rights attribute may be set only to read; if it is, update it to read|write).
  • πŸ‡ΊπŸ‡ΈUnited States jsutta United States

    Rerolled the patch and incorporated the change from https://www.drupal.org/project/media_thumbnails_pdf/issues/3414282 πŸ“Œ Convert CMYK colors to RGB for generated thumbnail RTBC due to encountering CMYK/RGB colorspace issues on our own site.

  • Status changed to Fixed 4 months ago
  • πŸ‡«πŸ‡·France Renrhaf πŸ“ Strasbourg 🐦🦜

    Fixed in 2.x version, thank you all !

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024