Follow-Up: Default public scheme is wrongly not considered as a local filesystem

Created on 29 November 2024, 3 months ago

Problem/Motivation

After the upgrade to consumer_image_styles version 4.0.10 I observed performance regressions on a project using the public stream wrapper from S3 File System β†’ module. The regression was similar to the fixed issue πŸ› Follow-up: Very slow JSON:API responses when images are stored on AWS bucket Needs review .

Steps to reproduce

Use the public stream wrapper from s3fs and debug \Drupal\consumer_image_styles\ImageStylesProvider::buildDerivativeLink().

The function should return before determining the image dimensions (from local file system):

if ($this->streamWrapperManager->getViaScheme(StreamWrapperManager::getScheme($uri))->getType() & ~StreamWrapperInterface::LOCAL_NORMAL) {
      return $info;
}

It does not. But it worked before the fix from πŸ› Default public scheme is wrongly not considered as a local filesystem Active , with following change:

diff --git a/src/ImageStylesProvider.php b/src/ImageStylesProvider.php
index bd04542..e629f41 100644
--- a/src/ImageStylesProvider.php
+++ b/src/ImageStylesProvider.php
@@ -124,7 +124,7 @@ class ImageStylesProvider implements ImageStylesProviderInterface {
];
// Sites with external images cannot afford to download the image to the
// webserver in order to inspect the image dimensions.
- if ($this->streamWrapperManager->getViaScheme(StreamWrapperManager::getScheme($uri))->getType() & ~StreamWrapperInterface::LOCAL) {
+ if ($this->streamWrapperManager->getViaScheme(StreamWrapperManager::getScheme($uri))->getType() & ~StreamWrapperInterface::LOCAL_NORMAL) {
return $info;
}
$image = $this->imageFactory->get($uri);

The s3fs module alters the stream_wrapper.public from the default one Drupal\Core\StreamWrapper\PublicStream to Drupal\s3fs\StreamWrapper\PublicS3fsStream, whose type is StreamWrapperInterface::NORMAL, which "does not include StreamWrapperInterface::LOCAL" (see documentation). If I understand correctly it's not a local stream wrapper, so it's usage should be fine.

To get a better understanding of the above mentioned if statement I did the following check:

StreamWrapperInterface::LOCAL_NORMAL;  // 0000000000011101
~StreamWrapperInterface::LOCAL_NORMAL; // 1111111111100010
StreamWrapperInterface::LOCAL;         // 0000000000000001
~StreamWrapperInterface::LOCAL;         // 1111111111111110
StreamWrapperInterface::NORMAL;        // 0000000000011100

As the statement uses the bitwise operation AND it will evaluate to false (= local) for StreamWrapperInterface::LOCAL_NORMAL, which was the intention/fix of πŸ› Default public scheme is wrongly not considered as a local filesystem Active :

StreamWrapperInterface::LOCAL_NORMAL & ~StreamWrapperInterface::LOCAL_NORMAL
0000000000011101
1111111111100010
0000000000000000

Before the fix it would have falsely evaluated to true (=not local):

StreamWrapperInterface::LOCAL_NORMAL & ~StreamWrapperInterface::LOCAL
0000000000011100
1111111111111110
0000000000011100

In that case the fix was fine. But in case of StreamWrapperInterface::NORMAL on the left site it will falsely evaluate to false (=local)

StreamWrapperInterface::NORMAL & ~StreamWrapperInterface::LOCAL_NORMAL
0000000000011100
1111111111100010
0000000000000000

BTW: Before the fix it worked fine, evaluated to true (=not local):

StreamWrapperInterface::NORMAL & ~StreamWrapperInterface::LOCAL
0000000000011100
1111111111111110
0000000000011100

Proposed resolution

As johnny5th commented in [#3252023-25] β†’ it should actually be:

if (!($this->streamWrapperManager->getViaScheme(StreamWrapperManager::getScheme($uri))->getType() & StreamWrapperInterface::LOCAL)) {
  return $info;
}

That is also much easier to read. The inner comparison will always evaluate to true (=local) for streamwrapper types where the right bit is set to 1, no matter what the other bits are:

***************1
0000000000000001
0000000000000001

The negation will turn it to true (= NOT local).

πŸ› Bug report
Status

Active

Version

4.0

Component

Code

Created by

πŸ‡©πŸ‡ͺGermany osopolar πŸ‡©πŸ‡ͺ GER 🌐

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @osopolar
  • πŸ‡©πŸ‡ͺGermany osopolar πŸ‡©πŸ‡ͺ GER 🌐

    Patch from MR!27 for composer.

  • πŸ‡©πŸ‡ͺGermany osopolar πŸ‡©πŸ‡ͺ GER 🌐
  • Pipeline finished with Success
    3 months ago
    Total: 376s
    #353776
  • πŸ‡©πŸ‡ͺGermany osopolar πŸ‡©πŸ‡ͺ GER 🌐
  • Yesterday i updated the module on an installation where we are also using S3 Module β†’ :

    drupal/consumer_image_styles (4.0.8 => 4.0.10)

    After the upgrade i get

    Error: Call to a member function getType() on false in Drupal\consumer_image_styles\ImageStylesProvider->buildDerivativeLink() (line 128 of /var/www/html/htdocs/modules/contrib/consumer_image_styles/src/ImageStylesProvider.php

    i tried the patch from MR #27 , but still got the error.

    Problem seems that

    $this->streamWrapperManager->getViaScheme(StreamWrapperManager::getScheme($uri) returns false now.

  • πŸ‡©πŸ‡ͺGermany osopolar πŸ‡©πŸ‡ͺ GER 🌐

    @makkus183 Can you (x)debug and check why it returns false? Which StreamWrapper do you use? I assume it's a different issue. This one is about figuring out if it's a local file or not. In your case it's about wrong behavior of $this->streamWrapperManager->getViaScheme(StreamWrapperManager::getScheme($uri), which of cause will make the detection fail but for different reasons. May you open a new issue and also link it here?

  • πŸ‡ΊπŸ‡ΈUnited States tom friedhof

    I can confirm the issue posted in #6. Reverting back to 4.0.8 resolves the issue that @makkus183 brought up.

    It's because `StreamWrapperManager::getScheme($uri)` returns empty when processing the `/no-thumbnail.png` file that I believe comes from the Media module.

  • @osopolar i did some (x)debugging and found the reason. We applied a patch which solved a validation error in JSON:API if an image name has a space in the filename. (see uri-reference in vendor/justinrainbow/json-schema/src/JsonSchema/Constraints/FormatConstraint.php)This patch urlencoded the $uri which caused the StreamWrapperManager::getScheme($uri) to return false as mentioned. I removed the patch and it is working as expected. The error which occured when a filename contains a space does not appear anymoree either.
    So you were right, it was another issue in my case.

Production build 0.71.5 2024