Support generating image styles for read-only streamWrappers

I’m not sure I understand this request.

A full URL is
https://wew.example.org/s3/files/styles/$image_style_name/$scheme/styles/$source_scheme/path/to/image

The Drupal rules are that:

• A read/write StreamWrapper $scheme will(must) be the same as the $source _scheme.
• A read-only StreamWrapper $scheme will(must) be the site default scheme.

In both cases for s3fs it should only be “public” or “s3” for $scheme (private files use the code provided controller).

Later we validate that $source_scheme is suppose to use an s3fs StreamWrapper, if not we reject the request to prevent access bypass vulnerabilities.

Can you clarify your request?

We have a custom read-only streamwrapper that is used to reference files on another server to allow us to use Drupal image styles on them.

We're currently using the URL pattern /s3/files/styles/image_style_name/streamwrappername/202/15481/upload_000922.jpg , which works now that I added streamwrappername to the PathProcessor. So basically the idea is that the client requests the file from Drupal, which in turn uses the stream wrapper to fetch the image (if necessary) and create the image style version of it, saving that to S3 and serving it from there.

It's certainly possible that I've misunderstood something and there's a better way to do this. Did I understand correctly that in this case an URL pattern like /s3/files/styles/image_style_name/public/styles/streamwrappername/202/81/upload_02.jpg should work instead? I tried this and at least on a first try it doesn't work.

There is also the ITOK access control as well.

Taking a step back, ImageStyle links should not be manually generated, instead Drupal Core ImageStyleInterface::buildUrl() should be used.

I left the itok arguments from my examples but yeah, they're there. We're indeed using ImageStyleInterface::buildUrl() to generate the image style URL's with something like streamwrappername://202/15481/upload_000922.jpg as the URI, and we're getting the URLs I mentioned ie. ones that have streamwrappername as the scheme.

That sounds like either an error in a class overriding the ImageStyle entity class or possibly in the non s3fs StreamWrapper.

S3fs doesn’t modify either of those.

I would be curious what your StreamWrapper getType() and getExternalUrl() methods contain as that would narrow the cause.

(Calling it a night, will check in tomorrow)

Our custom stream wrapper's getType returns StreamWrapperInterface::READ_VISIBLE. getExternalUrl returns FALSE with the intent of indicating that no direct URL is available (the method documentation indicates that this would be a web-accessible URL, while we want to serve the file through Drupal). Should this be something else? I'll admit that I didn't find much documentation regarding implementing a custom stream wrapper, but the current version is working except for the needed change in s3fs.

I should also mention that we do have the imageapi_optimize_webp module enabled which does something to the image style controller route.

Thanks for being so helpful so far, really appreciate it!

It appears I remembered incorrectly how links are structured.

Your links are indeed correct and there is not two different schemes present in the ImageStyle link.

My apologies for leading you down the incorrect path for this issue.

We are incompatible with imageapi_optimize_webp, see [#219495]. I'm not sur if that is the cause or not.

I need to fire up the s3fs lab again and look at this before I comment more. I do believe we handled this in our code in the past and that it should work, though perhaps it was corrupted along the way.

Setting back to active as there is at least something that needs to be looked at and moving to Bug Report as if this isn't caused by imageapi_optimize_webp than it should be something that does work.

Note: I am going to edit previous posts to strikeout incorrect statements to reduce the risk of them being misquoted in the future.

Your streamWrapper code sounds fine for getType(). I will note that I have seen a FALSE return on getExternalUrl cause WSOD as it is usually expected to return a string.

Confirmed your initials report about this being inside S3fsPathProcessorImageStyles (note to self: double check the class name mentioned in an issue, it wasn't S3fsImageStyleDownloadController).

I'm thinking we might completely remove the isValidScheme check in the PathProcessor.

Core doesn't validate the scheme is valid at the PathProcessor, it is done in the controller. We have the same check inside of S3fsImageStyleDownloadController.

We will want to validate ImageStyleDownloadController securely handles the schemes. There have been a number of vulnerabilities regarding ImageStyleDownloadControllers in the past few years in Core and Contrib with our limit of only accepting public:// and s3:// has protected us.

@zeip: have you been able to test the proposed changes to confirm they solve the issue in your deployment?

Automatically closed - issue fixed for 2 weeks with no activity.