Contrib.social

Integrate Drupal's Batch API when using plupload

Open on Drupal.org →
Created on 9 March 2023, almost 2 years ago
Updated 15 March 2023, almost 2 years ago

Problem/Motivation

We have a Drupal 9 site where users can upload photos to albums using the plupload widget. After upload, images are resized to max 800x600 with JPGs quality 75%. Unfortunately, we keep running into WSOD due to timeouts (as previously reported in this issue → ).

After much testing on both Pantheon and Platform.sh, the bottleneck seems to be the host's PHP setting of MAX_EXECUTION_TIME. For Pantheon, this is hard capped at 59 seconds, while Platform.sh lets you increase it to 300 seconds.

We've tried to resize and compress all images in Photoshop to max 800x600 at 70% quality, but even uploading 300 of these images (totalling about 25MB) often results in a WSOD. Strangely enough, all images usually do seem to end up in the album after a while.

Steps to reproduce

  1. Create a free Pantheon site
  2. Install Drupal 9
  3. Install and enable the Album Photos contrib module
  4. Go to /admin/config/media/photos and enable Plupload and set maximum image resolution to 800x600
  5. Create a new album and add a large number of photos (50 to 500 depending on size)
  6. Plupload will start uploading the files one by one
  7. Once Plupload finishes uploading all files, click 'Confirm Upload'
  8. On a Pantheon site, after exactly 59 seconds, you will see a WSOD with Application did not respond in time. (Results may vary depending on your server capacity and settings.)

Proposed resolution

We're hoping that integrating Drupal's Batch API → (when using plupload) will improve the process of uploading large amounts of images.

Remaining tasks

  • Determine how to integrate Drupal's batch API in this module
  • Make the necessary changes
  • Test and review
  • Commit!

User interface changes

Probably none

API changes

Probably none

Data model changes

Probably none

✨ Feature request
Status

Fixed

Version

6.0

Component

Code

Created by

🇳🇱Netherlands tinto Amsterdam

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @tinto
  • Status changed to Needs review almost 2 years ago
  • Comment almost 2 years ago →
    🇺🇸United States nathaniel

    Started some code for this. Not thoroughly tested, but seems to work.

  • Comment almost 2 years ago →
    🇳🇱Netherlands tinto Amsterdam

    Wow, that's fast! Thanks @nathaniel!

    I've tested patch #2 on a fresh Drupal install - both on my local machine and on a new Pantheon site - and it seems to work like a charm on both sites. Here's the combination of modules/versions that I used:

    • Drupal 9.5.4
    • Album Photos 6.0.x-dev
    • Plupload 2.0@beta
    • Patch from #2 →
    • (our custom workaround module for the plupload file extension error)
  • Comment almost 2 years ago →
    🇳🇱Netherlands frontmobe Amsterdam

    I have applied the patch provided and can confirm that the Plupload uploads now work flawlessly with the batch API!

    I tested this locally as well as remotely with a high number of large files, combined with short PHP timeouts configured. The new pluploadSubmitBatch() and batchFinished() functions in PhotosUploadForm.php work and the integration with batch API like progress messages work as expected. No errors show up in the php error log when using the module with the patch applied.

    I had to manually apply the patch to Photos module 6.0.2, because the patch is created against de dev version of the module. Any chance of maybe tagging a new release once this patch is confirmed to work ok?

    Many thanks for this nice extension of the module, using batch API for bulk uploads really increases stability.

  • Status changed to RTBC almost 2 years ago
  • Comment almost 2 years ago →
    🇺🇸United States nathaniel

    Sounds good to me!

    Tested "Maximum number of photos per album" and that still works as expected. I'll prep a new release.

  • Comment almost 2 years ago →
    System Message
    • 1a259e26 committed on 6.0.x 
      Issue #3347017 by Nathaniel: Batch API when using plupload.
  • Status changed to Fixed almost 2 years ago
  • Comment almost 2 years ago →
    🇺🇸United States nathaniel
  • Comment almost 2 years ago →
    System Message
    • 49f7fc4e committed on 6.0.x 
      Issue #3347017 by Nathaniel: Work on minor coding standards issues.
  • Comment over 1 year ago →
    System Message

    Automatically closed - issue fixed for 2 weeks with no activity.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024