Force creating a /s3 link for images (for API/indexing purposes)

Created on 29 February 2024, about 1 year ago
Updated 1 March 2024, about 1 year ago

Problem/Motivation

We have an API returning an image using a specific image style. When s3fs is used, the image URL is different depending on if the image style already exists or if it doesn't. If we have to flush the image styles, all the indexed image URLs are then broken as the image styles are removed and new images aren't created as the /s3 URL isn't called.

Proposed resolution

Would it be possible to make it easier to force the generation of a /s3 URL; those are of course a bit more resource-intensive for existing image styles, but in this kind of situation it'd be better to have a bit more resource intensive URL than a non-working one.

Feature request
Status

Active

Version

4.0

Component

Code

Created by

🇫🇮Finland ZeiP

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @ZeiP
  • 🇺🇸United States cmlara

    Assigning to 4.x as we need all features to be added there first.

    I can understand the issue and agree that finding a non-breaking URL is indeed a necessity for your situation. I will note that loading the style generation urls does have a larger performance hit than one might initially think. We currently clear the cache of every page that the source image is utilized on in order to prevent caching generation URL's. This will increase the burden to the site, how much of course depends upon your usage and how frequently the generation URL's are being used and how often the files are used elsewhere on the site.

    I'm wondering hos common a need this is as it would, on initial glance, require an API addition on the StreamWrapper plugins level since we can't assume that all plugins will use the s3fs_streamwrapper controller.

    Assuming it users the s3fs provided controller it would be something like
    Obtain the ITOK from the ImageStyle API.
    Generate a link using the s3fs_streamwrapper.image_styles.linkgen route and with the query parameter set to the ITOK.
    (similar can be done with the 3.x s3fs.image_styles route although the path needs to be appended manually)

    With the flexible plugin architecture of 4.x one could create a plugin that will always generate the S3 urls to avoid the need to manually generate them. This could even be extended to use a custom controller to change caching behavior.

    Alternatively, perhaps an image style warmer is a better solution? https://www.drupal.org/project/image_styles_generator is an option that shows up on quick search.

Production build 0.71.5 2024