Support JSON/REST multipart/form-data uploads

Created on 20 November 2018, almost 6 years ago
Updated 30 August 2024, 3 months ago

Problem/Motivation

This is a follow-up of #1927648: Allow creation of file entities from binary data via REST requests โ†’ which added support for uploading file through a REST endpoint.

The implementation only support file transferred with the Content-Type: application/octet-stream.

It's also possible (and widely used) to upload file using Content-Type: multipart/form-data.

Some client libraries, such as ngx-uploader for Angular projects, only support the multipart way.

Proposed resolution

It's valid to consider that the burden should be on client library and that it's up to it to be compatible with the simple binary way, but it's also valid to want to have both use cases supported by Drupal.

I think we should add a basic support for multipart/form-data uploads, which can at first be limited to only 1 file per request to reduce scope & complexity.

Remaining tasks

Declare a new format alongside "bin" to match multipart content-type and handle such requests.

โœจ Feature request
Status

Needs work

Version

11.0 ๐Ÿ”ฅ

Component
File moduleย  โ†’

Last updated 3 days ago

Created by

๐Ÿ‡จ๐Ÿ‡ฆCanada garphy

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • ๐Ÿ‡ป๐Ÿ‡ณVietnam Chฦฐฦกng Phแบกm

    The patch work for me but I have to change

    $fileKey = 'file';
    

    to

    $fileKey = $request->files->getIterator()->key();
    
  • ๐Ÿ‡ซ๐Ÿ‡ทFrance Guicom

    Update patch for Drupal 10.1.5

  • last update about 1 year ago
    Custom Commands Failed
  • last update about 1 year ago
    Patch Failed to Apply
  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia _utsavsharma

    Patch for 11.x.

  • last update about 1 year ago
    Custom Commands Failed
  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia vsujeetkumar Delhi

    Fixed the failures (CCF Issues) in #27.

  • last update about 1 year ago
    30,423 pass, 10 fail
  • I think the patch needs to be updated, it uses getContentType() to check for bin_multipart. The getContentType is now deprecated and we need to use getContentTypeFormat. That uses format and has a fixed set of keys mapped to mime type.

    static::$formats = [
      'html' => ['text/html', 'application/xhtml+xml'],
      'txt' => ['text/plain'],
      'js' => ['application/javascript', 'application/x-javascript', 'text/javascript'],
      'css' => ['text/css'],
      'json' => ['application/json', 'application/x-json'],
      'jsonld' => ['application/ld+json'],
      'xml' => ['text/xml', 'application/xml', 'application/x-xml'],
      'rdf' => ['application/rdf+xml'],
      'atom' => ['application/atom+xml'],
      'rss' => ['application/rss+xml'],
      'form' => ['application/x-www-form-urlencoded', 'multipart/form-data'],
    ];
    

    So It will always return 'form' in this use-case, so I think we need to use the new method and replace checks for bin_multipart with form.
    Since I'm running into this problem for my project I'll see I can put a new patch together.

  • As described above, I've update the patch for 10.1 to check for 'form' format instead of the bin_multipart that was added in previous patches in this issue.

  • last update 11 months ago
    Patch Failed to Apply
  • ๐Ÿ‡ซ๐Ÿ‡ทFrance Guicom

    Update patch for Drupal 10.2.3 including fix in #30

  • ๐Ÿ‡ซ๐Ÿ‡ทFrance Guicom

    Fix previous patch, wrong argument order in jsonapi.services.yml.

  • My multipart/form-data upload is still failing with the latest patch.
    The Content-Disposition header with filename is now mandatory, which I can somewhat understand. It is just annoying that a different party is using our REST interface and they need to change their code.

    But even with the Content-Disposition header set, the upload still fails on trying to move the temporary file. It claims it doesn't exist but that is because it uses the OriginalName to find the tmp file and not the actual path of the tmp file.

    In FileUploadResource:
    $temp_file_path = $this->streamUploadData($request);

    The patch now returns
    return basename($file->getClientOriginalName());

    Shouldn't it return $file->getPathname();
    I would say yes, but as I just glanced at the Symfony class I might be missing something.
    It would be a simple fix, so I'm testing it and will upload a new patch soon.

    I am a bit worried about Drupal 10.3 though, will look into that soon as it seems to refactor some relevant code and rerolling the patch will probably take a bit of time.

  • Added a new patch for Drupal 10.2.
    Like I said in my previous comment, changed the return value of streamUploadData to return the path instead of the originalName.

    I'll soon try to make some time to look into how drupal 10.3 affects this issue.

  • ๐Ÿ‡ฆ๐Ÿ‡บAustralia kim.pepper ๐Ÿ„โ€โ™‚๏ธ๐Ÿ‡ฆ๐Ÿ‡บSydney, Australia

    This might be better as a separate controller?

  • The previous patch is for FileUploadResource which is deprecated in 10.3 and will be removed in 11.0.
    FileUploadHandler will now be used: https://www.drupal.org/node/3402032 โ†’

    What that means for multipart/form-data upload, well my test doesn't fail but I do end up with a 0 bytes file.
    So I need to debug and see why the file is empty.

  • I Really want to embrace the new MR way of doing things, but there currently is a branch for 9.2.x or something and starting a new branch results in a page not found. So if someone can help with setting up a merge request for 10.3.x and/or 11.x.
    Who knows, it might one day be implemented...

    For now I've added a patch based on the 10.3 branch and tested it with 10.3.1.

  • So this is becoming a conversation with myself. I mentioned that FileUploadResource is deprecated, that is not fully through. A lot of its methods are, but the class still sticks around for now and re-uses the FileUploadHandler. That is a good thing for my project as it extends the FileUploadResource. But I did confuse myself as I mention the FileUploadHandler and then my patch mainly touches the FileUploadResource. So this is as much a reminder for myself as anyone else that is in the unlucky situation to have to support multipart/formdata uploads (but I might be the only one).

    I thought I tested the previous patch, but I most have had some uncommitted changes in my editor as it fails on the routing that now doesn't accept the format form anymore.

    I've re-added the form to the requirements in FileUploadResource and to JSONAPI routing.

  • ๐Ÿ‡ฆ๐Ÿ‡บAustralia kim.pepper ๐Ÿ„โ€โ™‚๏ธ๐Ÿ‡ฆ๐Ÿ‡บSydney, Australia

    In #35 I suggested this should be a separate controller.

Production build 0.71.5 2024