Contrib.social

(XLS) Batch mode does not work with S3FS

Open on Drupal.org →
Created on 5 December 2022, about 2 years ago
Updated 4 March 2024, 10 months ago

Problem/Motivation

Working with views_data_export and S3FS module as your file system is causing problems on generating/downloading file.

When the batch operation starts it second iteration, module fails trying to retieve realpath from S3.

If you check S3FS realpath method, it return FALSE because this wrapper simply does not support:

/**
   * {@inheritdoc}
   *
   * This wrapper does not support realpath().
   *
   * @return bool
   *   Always returns FALSE.
   */
  public function realpath() {
    return FALSE;
  }

Then if you check module DataExport, it tryies to use on the second iteration:

$vdeFileRealPath = \Drupal::service('file_system')->realpath($context['sandbox']['vde_file']);

the variable will be initializated as FALSE, and an exception will be thrown from IOFactory::load($vdeFileRealPath); when tryng to load the file.

Steps to reproduce

- Create a export view page.
- View page must execute operation with batch mode (this is important)
- Active S3FS module
- Execute download operation

--
Note: this will only breaks when your explicit configure your views to generate data via batch operation.

Proposed resolution


Fix URI that goes on IOFactory::load()

🐛 Bug report
Status

Needs work

Version

1.2

Component

Code

Created by

Yago Elias

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • First commit to issue fork.
  • Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 8.0 & MySQL 5.7
    last update over 1 year ago
    Not currently mergeable.
  • @zuernbernhard opened merge request.
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 8.0 & MySQL 5.7
    last update over 1 year ago
    2 pass, 2 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 8.0 & MySQL 5.7
    last update over 1 year ago
    2 pass, 2 fail
  • @zuernbernhard opened merge request.
  • Comment over 1 year ago →
    🇩🇪Germany zuernbernhard

    Here is the patch attached. It is not possible anymore to work with s3:// as ätreamwarapper but we can use public:// or private:// in our filename-option form the views display and the s3fs module will handle everything. so something like:

    private://views_data_export/export.csv

    in the "FILE STORAGE/DOWNLOAD SETTINGS" in the view-displays path-configuration for filename and uncheck "Store file in public files directory" will work.

    ATTENTION: If you use the public filesystem checkbox and a public:// stream wrapper the generated file download-link will be the S3-Bucket URL. Users should not see it and they will also not be allowed to access ist. But if you use the private filesystem s3fs handles everything properly.

  • Comment over 1 year ago →
    🇩🇪Germany zuernbernhard

    sry messed up the patch file

  • Comment 10 months ago →
    🇨🇦Canada vincent signoret

    I think I have a similar issue with Azure. Has anyone been able to use views_data_export with Azure Blob Storage? I'm wondering if this patch could work with the Azure File System ?

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024