Does this work with private files ?

Created on 29 January 2024, 12 months ago
Updated 23 February 2024, 11 months ago

Problem/Motivation

Does not seem to work for private files ?

Steps to reproduce

/admin/config/media/file-system
- Public file system path: sites/default/files
- Private file system path: ../private
-> save, make sure a folder 'private' exists one level above the drupal website root folder

/admin/structure/media/manage/image/fields/media.image.field_media_image (edit the image field of your Image Media type)
- Field storage: Upload destination: Private files
- File directory: assets/images

When it tries to render a crop when I test it, it seems that it gets this URI:
private://assets/images/demo-008_AdobeStock_60027303.jpeg
And turns it into:
https://mywebsite.com/system/files/contextual/styles/crop_16_9/212/priva...

Which is a 404 page not found.

Is something wrong with my setup, or is this reproducible and a general problem that cropping does not work with private files at the moment ?

🐛 Bug report
Status

Fixed

Version

2.0

Component

Code

Created by

🇧🇪Belgium flyke

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @flyke
  • 🇫🇷France DrDam

    It is reproducible and it never was tested...

    I'll check the pathprocessor

    • 14609345 committed on 2.0.x
      Issue #3417740: fix private files access
      
  • 🇫🇷France DrDam

    Pushed in 2.0.x,

    just a side effect of the correction to #3416473.
    For private resources, we must add a "file" parameter in the request.

  • 🇧🇪Belgium flyke

    Just to be clear: does cropping work with private files for you, for the settings as mentioned in the issue summary ?
    I still cant seem to get it to work in the latest version. But it could be because of the project, so I'd like confirmation that normally it should work.

  • 🇫🇷France DrDam

    If I test a "standard image field" set to private files, Yes "standard CROP" work (you can test with a "basic field image with crop", in cas of some file permission problem)

    this seems coherent, because for Drupal "file mediation" there is no differencies between public/private or any other "file stream scheme" (ex : S3 file system).

    So this should work, and I making it working on my local (config as you describe it) with 2.0.x need a whole cache flush.

  • 🇫🇷France DrDam

    Lets say you have all information needed, so you have a Crop entity, you have all width, height, x, y, context info, you have the image uri etc. Which is the exact code to programmatically render a Crop?

    How it's work

    something like that :

    $image_style = ImageStyle::load('my_style');
    
    $original_image_uri = "public://path/to/source/file.jpg"; // public for the exemple
    $derivative_uri =  "public://path/to/the/future/derivative/file.png"; // may be generated by something like ImageStyle::buildUri()
    
    $image_style->createDerivative($original_image_uri, $derivative_uri);
    

    If your image style use a crop effect, the crop effect will check in all crop entities which entity have as the couple "crop type / original_image_uri".

    => so no context effect here, if you test that without my mess, you cannot apply a context on the crop process, it will use the first couple "crop type / original_image_uri" found.

    My first approach : create copies of original files, and crop on it.

    My first approach of the multiple crop issue was to "troll" this association, by "creating copies" of the original_image in the file system (1 copy by context) so for the CROP module, there still have one unique couple "crop type / image uri" , except the image_uri are no longer the "source image" (just a copy)

    So in term of code, juste have a "formater" or "altering the image formater" will create a "copy" of the image source and create a CROP entity with contextual setting targeting this copy.

    You can check 1.x branch of the collection (or multi_crop module) for more details.
    have found this approach was a mess for keeping the file system & the crop entity table "clean" (when the image will change in the media, when media are deleted, or the image_style edited...)

    The second approach : without copies by new controller and CROP_ID injection

    for this approach, I have add a patch to the CROP process :

    public static function findCrop($uri, $type) {
    -    return \Drupal::entityTypeManager()->getStorage('crop')->getCrop($uri, $type);
    +    $contextual_crop = \Drupal::request()->query->get('contextual_crop');
    +
    +    if($contextual_crop) {
    +      return \Drupal::entityTypeManager()->getStorage('crop')->load($contextual_crop);
    +    }
    +    else {
    +      return \Drupal::entityTypeManager()->getStorage('crop')->getCrop($uri, $type);
    +    }
    }
    

    If there is a "contextual_crop" (I.E. a CROP_id) in the request, we just load it, instead searching for a "crop type / image uri" couple.

    The "cost" of that, are I cannot use the "standard imageDeliver method, because I need the "crop_id" in a parameters (not in the query), so it's a new route. In order to limit security issues, I have copy the whole "imageDeliver" process (routing, pathprocessing & controller).

    I'm always open to new implementation proposal. I'm in love with this problem, not with my solution...

  • 🇧🇪Belgium flyke

    I was unable to get any cropped image to programmatically generate with private files.
    I am not sure if it is actually because if they are private or because something else.

    Via some code I load (or create first) a correct Crop entity ($crop).
    I also have the original image URI.
    This is my code to render a cropped image:

    ...
    // Get crop URI.
    /** @var \Drupal\media_contextual_crop\MediaContextualCropService $mcc_service */
    $mcc_service = \Drupal::service('media_contextual_crop.service');
    $cropped_uri = $mcc_service->createContextualizedDerivativePath($originalUri, $crop_image_style, $crop->id(), TRUE);
    // Render image.
    $preview_image = [
      '#theme' => 'image',
      '#uri' => $cropped_uri,
    ];

    The value of $cropped_uri = private://contextual/styles/crop_16_9/215/private/assets/images/demo-013_adobestock_243699323_nl.jpg

    This generates an image tag like:
    <img src="/system/files/contextual/styles/crop_16_9/215/private/assets/images/demo-013_adobestock_243699323_nl.jpg" alt="" class="img-fluid">

    This does not render any image. If I open the image in a new tab then the url = https://mysite.com/system/files/contextual/styles/crop_16_9/215/private/...

    But that returns a 404 page not found.
    After long invetigation, the ContextualImageStyleDownloadController process function is never triggered.
    That seems to be because a /system/files/contextual/styles/{image_style}/{context}/{scheme} path is never requested.
    That seems to be because the processInbound function of media_contextual_crop/src/PathProcessor/PathProcessorImageStyles.php does not set file to the query element.

    It seems that the processInbound function is triggered twice when opening the https://mysite.com/system/files/contextual/styles/crop_16_9/215/private/assets/images/demo-013_adobestock_243699323_nl.jpg url (see screenshots)
    The first time all is good and the path contains file parameter:
    The value of $path = /system/files/contextual/styles/crop_16_9/215/private/assets/images/demo-013_adobestock_243699323_nl.jpg
    The value of $path_prefix = /system/files/contextual/
    The code reaches into this part of the code: if (substr_count($rest, '/') >= 4) { an dsuccessfully sets the 'file' query parameter.

    BUT then the url in the browser is automatically changed into:
    https://mysite.com/system/files/contextual/styles/crop_16_9/215/private

    This triggers the processInbound function again (see screenshot).
    This time the file path is missing from $path:
    The value of $path = /system/files/contextual/styles/crop_16_9/215/private
    and the code never reaches inside if (substr_count($rest, '/') >= 4) {.
    The code reaches the last else { condition. No 'file' query parameter is set this time.

    This cause the process function of ContextualImageStyleDownloadController.php to never trigger.

    For testing purposes, I can Add the following line into the last else { condition:
    $request->query->set('file', 'private/assets/images/demo-013_adobestock_243699323_nl.jpg');

    If I do this, then the process function of ContextualImageStyleDownloadController.php is triggered.

    This is a small step forward, but... Still no image is rendered.
    This is because then inside the process function, the request opbject has no 'itok' parameter (IMAGE_DERIVATIVE_TOKEN). So when the code performs $this->checkToken it fails and no image is rendered.

    This can be fixed however by adding this piece of code into the process function:

        // Set 'itok' query parameter if it does not exist yet.
        // Otherwise checkToken will fail and no image will be generated.
        if (is_null($request->query->get(IMAGE_DERIVATIVE_TOKEN))) {
          $normalize_old_original_uri = $this->streamWrapperManager->normalizeUri($original_image_uri);
          $request->query->set(IMAGE_DERIVATIVE_TOKEN, $image_style->getPathToken($normalize_old_original_uri));
        }

    Now the $this->checkToken works, and finally, a cropped image is shown!

    Its good to finally see a cropped image but setting a fixed 'file' query parameter is of course not good.
    Its the redirecting from https://mysite.com/system/files/contextual/styles/crop_16_9/215/private/... to https://mysite.com/system/files/contextual/styles/crop_16_9/215/private that should be addressed I think

  • 🇧🇪Belgium flyke

    Ok, I found new code to programmatically generate a cropped image. This time I don't need to adjust anything inside ContextualImageStyleDownloadController.php. Also, the crop is now showing the correct cropped part of the image. Previous code always cropped top left of the image so the resulting displayed image was not correct, but now it is.

    The executing twice of processInbound is still a problem, so for now I still need to explicitly set a 'file' query parameter inside the last else { condition.

    Updated code (after getting or creating a $crop entity, and for a given $media entity - I use this inside a custom FieldFormatter for Media entities that will auto crop correct no matter what crop the user has applied.):

        /** @var \Drupal\media_contextual_crop\MediaContextualCropUseCasePluginManager $use_case_plugin_manager */
        $use_case_plugin_manager = \Drupal::service('plugin.manager.media_contextual_crop_use_case');
        $use_case_plugin = $use_case_plugin_manager->createInstance('references');
        $crop_data = Json::decode($media->_referringItem->overwritten_property_map);
        // ... here is code to load or create Crop entity based on $crop_data.
    
          $crop_type = $crop->type->entity->id();
          $crop_image_style = $layoutbuilder_helper->getCropImageStyle($crop_type, FALSE); // Custom function to get matching image style based on crop type.
    
          $image_item = $media->field_media_image->first();
          $context_settings = $use_case_plugin->getCropSettings($image_item);
          $context_settings['plugin'] = $use_case_plugin->getPluginId();
          $context_settings['item_values'] = $image_item->getValue();
          $context_settings['plugin_id'] = 'image_widget_crop';
          $context_settings['context'] = $context; // This is defined earlier in my code. Value is something like: block_content:starterkit_block_image:76.image_image.0
          $context_settings['base_crop_folder'] = 'media_contextual_crop';
          $context_settings['crop_setting'] = $crop_data['field_media_image'][0]['image_crop'];
          $cropped_image_uri = $use_case_plugin->getContextualizedImage($context_settings, $originalUri, $crop_image_style);
    
            $preview_image = [
              '#theme' => 'image',
              '#uri' => $crop_uri,
            ];
  • 🇧🇪Belgium flyke

    I added a patch that fixes the problem inside the processInbound function where the second time its loaded, there is no file query parameter causing cropping to not work. I am unsure of why the controller is only triggered if the request has a file query parameter (because if you have context / crop id you can load Crop entity and get file uri from there) but at least this patch seems to fix my problems with programmatically generating crops.

  • 🇫🇷France DrDam

    For the "file" parameters in the request object, it is to prevent the "PathProcessorFiles" from taking charge of the file downloading.
    It seems I still don't understand the black magic hiding in pathprocessors ...

    • flyke authored 90692ac8 on 2.0.x
      Issue #3417740 by flyke: Does this work with private files ?
      
  • 🇫🇷France DrDam

    Thanks for your time and your support

  • Status changed to Fixed 12 months ago
  • Status changed to Fixed 11 months ago
Production build 0.71.5 2024