Contrib.social

Cloudinary Stream Wrapper caches items in the database (Database Storage) but never actually checks if an item is inside the storage

Open on Drupal.org →
Created on 18 April 2023, about 1 year ago
Updated 20 June 2024, 9 days ago

Problem/Motivation

The Cloudinary plugin writes items after retrieving them from the API to the database storage. But it will never check if the item is inside the DB storage when retrieving them. For pages that only have 1-2 images at most, you will not notice the issue directly but if we have a page with a vast collection it can take over 40 seconds (800ms - 1s per image) for a page to render and load.

Steps to reproduce

Clean install the plugin (2.1.3), and retrieve on a page a huge amount of images, the more the better the result. You will notice an increase in page load when the amount of images increases that are loaded through Cloudinary.

Proposed resolution

The cloudinary_stream_wrapper_resource_prepare code is never called anywhere within the module causing every request for an API call.

Partial patch that resolves this issue with cloudinary_stream_wrapper_load_file, The page I used locally to test took 43 seconds to load before applying the patch and afterward 430ms.

Remaining tasks

Probably still needs additional checks for folders and other options. This was just in our use case the only call necessary to have high performance.

User interface changes

n/a

API changes

n/a

Data model changes

n/a

🐛 Bug report
Status

Needs review

Version

3.0

Component

Code

Created by

🇳🇱Netherlands jneuteboom

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment about 1 year ago →
    🇺🇸United States B_man California, USA

    Re-published, not spam.

  • Comment about 1 year ago →
    🇳🇱Netherlands jneuteboom
  • Comment 6 months ago →
    🇳🇱Netherlands jneuteboom

    Since there has not been a reply here recently on this issue. Just wanted to let you know this issue is still relevant for the new 3.x branch. The patch above is still working without any modifications.

  • Comment 6 months ago →
    🇳🇱Netherlands jneuteboom
  • Comment 9 days ago →
    🇧🇪Belgium klaasvw

    There are multiple problems related to this issue:

    * The meta data is never loaded from storage (which is addressed by the previous patch)
    * The meta data is never fetched if the asset's public_id contains a file extension
    * Because the meta data is always re-fetched a fatal error is thrown if the original file is not found via the Cloudinary admin api: Uncaught InvalidArgumentException: A path was passed when a fully qualified domain was expected
    * Pages containing multiple cloudinary assets (e.g. the file admin view, entities with a lot of cloudinary media entities, etc) get a big performance impact because the cloudinary_stream_wrapper module calls the cloudinary api for every asset that is displayed.

  • Merge request !29Resolve #3354894 "Cloudinary stream wrapper" → (Open) created by klaasvw
  • Assigned to klaasvw
  • Status changed to Needs review 9 days ago
  • Comment 9 days ago →
    🇧🇪Belgium klaasvw

    Attached is a patch that addresses these issues:

    • The patch introduced earlier
    • Keep the file extension in the public_id
    • Fall back to the homepage URL if the asset cannot be fetch from cloudinary when generating an external URL
contrib.social Blog FAQ Discussions
Production build 0.69.0 2024