Contrib.social

Support delivering local image styles until remote upload is complete

Open on Drupal.org →
Created on 3 February 2016, almost 9 years ago
Updated 2 August 2023, over 1 year ago

Problem/Motivation

To quote @deviantintegral:

For Drupal 7, we found that the S3 API was very slow for image styles - on the order of several seconds by the time all of the stat and write calls were done. What I ended up doing was initially serving the image style from a local temp file, and then uploading the file to S3 in a shutdown handler. We should investigate something similar here:

https://github.com/justafish/drupal_amazons3/blob/7.x-2.x/amazons3.modul...

Setting 'cache' => TRUE should help for some of this, but image styles are indeed still slow to generate.

The original issue came up in the flysystem_s3 issue queue, #2528630: Support image style generation. → . It was later also reported in the flysystem_gcs issue queue, 🐛 Saving and loading content takes way too long when multiple image styles are used Needs review .

Steps to reproduce

Taken from 🐛 Saving and loading content takes way too long when multiple image styles are used Needs review :

1. set one image field to have multiple values and GCS as storage. Also, for faster testing, you can disable the required alt field.
2. set an image style or responsive image style for displaying the image field value.
3. upload at least 10 images. (this is the first place where the slowdowns occur)
4. click save.

After clicking save the user is redirected to the canonical node page, so the image styles are generated (probably). The user has to wait until every image is generated to get a response from the server. This takes more than 30 seconds sometimes. If a site has responsive image styles, then this process will run even longer because of the multiple image styles for multiple breakpoints. Also, if someone generates jpg and webp variants (with imageapi_optimize_webp), then again this needed time multiplies until infinity.

Proposed resolution

Don't wait until the remote upload is complete before sending the generated image derivative to the client:

  1. If an non-existent image derivative is requested (any path starting with styles/), a temporary URL is generated to a Flysystem Drupal route. This URL is returned to the browser instead of the direct image URL.
  2. The browser requests this URL to get the image.
  3. A check is done to make sure the source image exists. If it doesn't, a 404 is returned and no further action is taken
  4. If in the meantime the image derivative was created (should be rare), redirect to the remote adapter URL.
  5. If the image still doesn't exist, generate the derivative and store it locally in a temporary folder. The temporary file is now sent to the browser without having to go through the remote adapter. Once the response is sent and the image is displayed in the browser, Drupal will copy the temporary image to the remote adapter.
  6. If the image is requested again and the temporary file still exists, that file is returned, which saves us a trip to the remote adapter.
  7. Once the temporary file is copied to the remote adapter, that URL will be sent to the browser instead of the Flysystem Drupal route.

Remaining tasks

  • Review the proposed implementation
  • Write tests
  • Update the documentation

User interface changes

None.

API changes

None.

Data model changes

None.

Release notes snippet

None.

✨ Feature request
Status

Needs work

Version

2.1

Component

Code

Created by

🇺🇸United States twistor

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024