Support chunked uploads

Created on 7 April 2020, over 4 years ago
Updated 27 March 2024, 9 months ago

I will be submitting a patch against the latest dev version to support the "chunking" functionality of Dropzone (see https://www.dropzonejs.com/#config-chunking) to enable uploading large files on slow connections.

✨ Feature request
Status

Needs review

Version

2.7

Component

Code

Created by

πŸ‡©πŸ‡ͺGermany semiaddict

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MySQL 5.7
    last update over 1 year ago
    Patch Failed to Apply
  • πŸ‡·πŸ‡΄Romania abautu

    I have prepared and tested a smaller patch. It doesn't offer all the features from the previous patches, but I'm adding it here in case it helps anyone.

    It does only these things:
    - enables chunking on all dropzone elements (using default chunk size of 2M from DropzoneJS)
    - fixes missing response in the succes callback (in 5.7, the response parameter of success callback is an empty string for chunked uploads; this has been changed in 5.9 to be consistent between chuncked and regular uploads)
    - fixes the upload callback to correctly write/append each chunk.

    I tested it by uploading files from 5 to 200MB on a PHP server with 20MB upload limit.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MySQL 5.7
    last update over 1 year ago
    4 pass
  • πŸ‡·πŸ‡΄Romania abautu

    Patch from #36 failed to apply to dev. I updated it here.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MySQL 5.7
    last update over 1 year ago
    4 pass
  • πŸ‡³πŸ‡±Netherlands Nicasso

    I made some additional changes to the patch #33, that @dishabhadra provided:

    - I force the usage of chunking.
    - I added a new settings value (max_filesize_default) to be able to set a new maximum file size, thus not depening on the PHP upload_max_filesize value)
    - Changed the getting of CSRF tokens, to make chunking work for the Media library widget

    With these changes chunking now works for me in the following 2 places:

    - Bulk upload (admin/content/media/bulk-upload)
    - Media library widget

    Patch applies on Dropzonejs version 2.8.0 and I used it with in combination with the Dropzone library version 5.9.3.

    Complete patch + interdiff with #33 provided.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MySQL 5.7
    last update about 1 year ago
    4 pass
  • πŸ‡³πŸ‡±Netherlands Nicasso

    Patch file in #38 contained a mistake with info.yml files.

    Correct it here in this patch.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 9.5.5 + Environment: PHP 7.3 & MySQL 5.7
    last update about 1 year ago
    Patch Failed to Apply
  • πŸ‡³πŸ‡±Netherlands Nicasso

    When using lightning media, you might also want to include this patch in combination with the path from #39.

  • πŸ‡¨πŸ‡¦Canada vincent signoret

    I applied patch #39 with

    • - drupal 10.2.1
    • - dropzonejs v2.8
    • - entity_browser v2.10



    I had a media 'document' containing a field of type File. This field has a maximum file size of 1000 MB. I added a widget to the display form of this field of type Entity Browser. This widget has the following configuration settings:

        settings:
          submit_text: 'SΓ©lectionner le document'
          auto_select: false
          upload_location: 'public://[date:custom:Y]-[date:custom:m]'
          dropzone_description: 'Drop files here to upload them'
          max_filesize: 1000M
          extensions: 'pdf odt ods xls xlsx doc docx xhtml xbrl zip'
          clientside_resize: false
          resize_width: null
          resize_height: null
          resize_quality: 1.0
          resize_method: contain
          thumbnail_method: contain
          chunking: true
          force_chunking: true
          chunk_size: 2000000
          parallel_chunk_uploads: true
          retry_chunks: true
          retry_chunks_limit: 3
    

    When I tried to upload a file (900MB) I always have the error "File is too big (XXXMiB). Max filesize: XXXMiB."

    With some debuging in DropzoneJsEbWidget.php line 165 the $config = $this->getConfiguration(); returned the configuration I had set in the entity browser widget, except for the max_file_size. The max_file_size alwayss returned the value of my upload_max_filesize in my php.ini

    I have tried previous patches and different versions of enyo/dropzone, but I always face the same issue.

  • πŸ‡³πŸ‡±Netherlands uberengineer

    Patch #39 does not apply to dropzonejs 2.10.0

  • πŸ‡©πŸ‡ͺGermany semiaddict

    Just create a fork and a merge request on the latest version: https://git.drupalcode.org/project/dropzonejs/-/merge_requests/14

  • Pipeline finished with Success
    9 months ago
    Total: 220s
    #124141
  • πŸ‡³πŸ‡±Netherlands Nicasso

    @uberengineer, hereby an updated patch for 2.10.0

Production build 0.71.5 2024