Support chunked uploads

I have prepared and tested a smaller patch. It doesn't offer all the features from the previous patches, but I'm adding it here in case it helps anyone.

It does only these things:
- enables chunking on all dropzone elements (using default chunk size of 2M from DropzoneJS)
- fixes missing response in the succes callback (in 5.7, the response parameter of success callback is an empty string for chunked uploads; this has been changed in 5.9 to be consistent between chuncked and regular uploads)
- fixes the upload callback to correctly write/append each chunk.

I tested it by uploading files from 5 to 200MB on a PHP server with 20MB upload limit.

Patch from #36 failed to apply to dev. I updated it here.

I made some additional changes to the patch #33, that @dishabhadra provided:

- I force the usage of chunking.
- I added a new settings value (max_filesize_default) to be able to set a new maximum file size, thus not depening on the PHP upload_max_filesize value)
- Changed the getting of CSRF tokens, to make chunking work for the Media library widget

With these changes chunking now works for me in the following 2 places:

- Bulk upload (admin/content/media/bulk-upload)
- Media library widget

Patch applies on Dropzonejs version 2.8.0 and I used it with in combination with the Dropzone library version 5.9.3.

Complete patch + interdiff with #33 provided.

Patch file in #38 contained a mistake with info.yml files.

Correct it here in this patch.

When using lightning media, you might also want to include this patch in combination with the path from #39.

I applied patch #39 with

• - drupal 10.2.1
• - dropzonejs v2.8
• - entity_browser v2.10

I had a media 'document' containing a field of type File. This field has a maximum file size of 1000 MB. I added a widget to the display form of this field of type Entity Browser. This widget has the following configuration settings:

    settings:
      submit_text: 'Sélectionner le document'
      auto_select: false
      upload_location: 'public://[date:custom:Y]-[date:custom:m]'
      dropzone_description: 'Drop files here to upload them'
      max_filesize: 1000M
      extensions: 'pdf odt ods xls xlsx doc docx xhtml xbrl zip'
      clientside_resize: false
      resize_width: null
      resize_height: null
      resize_quality: 1.0
      resize_method: contain
      thumbnail_method: contain
      chunking: true
      force_chunking: true
      chunk_size: 2000000
      parallel_chunk_uploads: true
      retry_chunks: true
      retry_chunks_limit: 3

When I tried to upload a file (900MB) I always have the error "File is too big (XXXMiB). Max filesize: XXXMiB."

With some debuging in DropzoneJsEbWidget.php line 165 the $config = $this->getConfiguration(); returned the configuration I had set in the entity browser widget, except for the max_file_size. The max_file_size alwayss returned the value of my upload_max_filesize in my php.ini

I have tried previous patches and different versions of enyo/dropzone, but I always face the same issue.

Patch #39 does not apply to dropzonejs 2.10.0

Just create a fork and a merge request on the latest version: https://git.drupalcode.org/project/dropzonejs/-/merge_requests/14

@uberengineer, hereby an updated patch for 2.10.0

Rerolled the lightning_media patch for version 5.0.0.

What’s the holdup on merging the PR? We use this in production across multiple websites. What can we do to get this moved into a new release?

@glynster, I believe tests are still missing.

Ah ha good to know semiaddict. As you are the creator of this wonderful gem will you be adding the testing or waiting on the maintainers?

I tested MR14 but have the same issue as #41

if php upload_max_filesize is smaller the file size I am trying to upload, the error that file is too big referring at the limit of the PHP Limit.

Testing this as we finally have a use case for this.

The chunked upload itself seems to work fine, however, then nothing happens.

Debugging a bit, I can see that despite the comment saying otherwise, dropzoneInstance.on('success' is in fact called with a response, on the last part, *after* Drupal.dropzonejs.chunksUploaded, and it has result filename with the last part. This overrides file.processedName, this is then submitted and obviously the server then doesn't find that anymore.

This is with dropzone 6.x beta2 so there might have been a change with that.

Commenting out that line as a quickfix solves that and the media can then be edited. But it needs a better check, maybe by checking if file.processedName is already set and then assuming that it was set by the chunk response so that it's compatible with different versions?

Also, if you have problems with max file size, then you might be on an old core version, #2867336: File size validator should only respect the explicitly configured maximum file size → should have resolved that.