Send/Upload files?

Hi! Any update on getting this into a release?

Warning : Do ​​not install the Webform REST 4.0.2 update, if you have this patch.

This completely broke my project in production.

@Monster971 did you apply this patch to version 4.0.2 before uploading it to production? Did it apply to 4.0.2 successfully and cause issues? Did it fail to apply? Is further work needed on the code contained in the patch? Please try to be helpful with your comments. Open-source software is built by all of us working together to make things better.

Hello @Arrow,

Sorry for the lack of additional information to help the community.

I have a Drupal 9.5.5 (Apache/2.4.6 (CentOS) PHP/7.4.33) project that relies heavily on Webform REST module patch #37.

When I tried to update Webform REST 4.0.1 to Webform 4.0.2 with the following command:

composer update drupal/webform_rest

I got the following error:

Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2021-04-30/webform_rest-add_file_upload_resource-2899902-37.patch

As well as in my Drupal project the following PHP error which crashes my project:

Drupal\Component\Plugin\Exception\PluginException: Plugin (webform_rest_file_upload) instance class "Drupal\webform_rest\Plugin\rest\resource\WebformFileUploadResource" does not exist. in Drupal\Component\Plugin\Factory\DefaultFactory::getPluginClass() (line 97 of /var/www/html/myproject/web/core/lib/Drupal/Component/Plugin/Factory/DefaultFactory.php).

Here is the information from my side.

Updating the patch from #47 to work with Drupal 10 where MimeTypeGuesser::guess was removed and MimeTypeGuesser:: guessMimeType is used.

https://www.drupal.org/node/3126004 → for reference.

Attached re-rolled patch that include changes from #55 for 4.0.2 and 4.x version

rerolled patch to work with v4.0.3

Hello,

I have a question, for a long time I was using patch #42 which saved my files in /files/private/webform/webform_id/, a week ago I applied patch #56-v4.x, I found that the files are now saved in /files/private/webform_id_tmp/.

Is this normal?

enhanced code for validation implements

Updating the patch from #59 to replace a deprecated function call MimeTypeGuesser::guess() with MimeTypeGuesser::guessMimeType().

The patch in #61 is malformed because your .editorconfig stripped trailing spaces from the file. I hate when that happens. :p

I pushed @rhristov's change to GitLab.

I am sending attached files successfully from NextJs to Drupal Webform. When I try to preview the files in Drupal the PDF files are perfect but .doc and .docx are corrupted.
Any thoughts what can be the problem?

@tolibg Same problem here. We think that all file formats are affected but PDFs possibly more forgiving. A coworker is checking into it.

Also we need to remove \Drupal\Component\Utility\Bytes::toInt() (change record linked below) for Drupal 10 compatibility. Uploads are failing for us without this change.

\Drupal\Component\Utility\Bytes::toInt() is deprecated in favor of \Drupal\Component\Utility\Bytes::toNumber()

Sorry, my brain is tired from all of the Drupal 10 migrations!! I think another version check may still be needed on the second occurrence of toInt() (line 203).

I've added the check suggested in #66

fyi Bytes:toInt is deprecated on drupal10
so ive mannualy patch WebformFileUplodResource.php->getElementValidators

  protected function getElementValidators(array $element) {
    $validators = [
      // Add in our check of the file name length.
      'file_validate_name_length' => [],
    ];

    // Cap the upload size according to the PHP limit.
    if (version_compare(\Drupal::VERSION, '9.1', '<')) {
      $max_filesize = Bytes::toInt(Environment::getUploadMaxSize());
    }
    else {
      $max_filesize = Bytes::toNumber(Environment::getUploadMaxSize());
    }

    if (version_compare(\Drupal::VERSION, '9.1', '<')) {
    if (!empty($element["#max_filesize"])) {
      $max_filesize = min(
        $max_filesize,
        Bytes::toInt($element['#max_filesize'] * 1024 * 1024)
      );
    }
    }
    else {
      if (!empty($element["#max_filesize"])) {
        $max_filesize = min(
          $max_filesize,
          Bytes::toNumber($element['#max_filesize'] * 1024 * 1024)
        );
      }
    }

    // There is always a file size limit due to the PHP server limit.
    $validators['file_validate_size'] = [$max_filesize];

    // Add the extension check if necessary.
    if (!empty($element['#file_extensions'])) {
      $validators['file_validate_extensions'] = [$element['#file_extensions']];
    }

    return $validators;
  }

@sl45sms Super but this was already addressed in #67.

@othermachines yep, late nite, brain fog, and lost in thousand drupal 10 patches I missed that. Thanks.

I successfully applied the webform_rest-add_file_upload_resource-2899902-61.patch #61 on Drupal Version 9.5.10 and PHP Version 8.1.26, and it worked fine. Thank you for the patch!

However, when testing on Drupal Version 10.1.6, I encountered an error related to Bytes:toInt. The issue seems to be resolved with the update in patch #69.

  protected function getElementValidators(array $element) {
    $validators = [
      // Add in our check of the file name length.
      'file_validate_name_length' => [],
    ];

    // Cap the upload size according to the PHP limit.
    if (version_compare(\Drupal::VERSION, '9.1', '<')) {
      $max_filesize = Bytes::toInt(Environment::getUploadMaxSize());
    }
    else {
      $max_filesize = Bytes::toNumber(Environment::getUploadMaxSize());
    }

    if (version_compare(\Drupal::VERSION, '9.1', '<')) {
    if (!empty($element["#max_filesize"])) {
      $max_filesize = min(
        $max_filesize,
        Bytes::toInt($element['#max_filesize'] * 1024 * 1024)
      );
    }
    }
    else {
      if (!empty($element["#max_filesize"])) {
        $max_filesize = min(
          $max_filesize,
          Bytes::toNumber($element['#max_filesize'] * 1024 * 1024)
        );
      }
    }

    // There is always a file size limit due to the PHP server limit.
    $validators['file_validate_size'] = [$max_filesize];

    // Add the extension check if necessary.
    if (!empty($element['#file_extensions'])) {
      $validators['file_validate_extensions'] = [$element['#file_extensions']];
    }

    return $validators;
  }

Thank you for the continued efforts in maintaining and improving this patch.

Best regards,

malikarslanasif131

Has anyone faced the file upload size validation issue? I set the webform field file upload max size to 5MB, but it allows sending files larger than 5 MB.

We need to add an access check, in the current implementation everybody can upload whatever files they want through this endpoint, even anonymous users.

@as_vard I just checked this with the latest version of the MR and it seems to work as expected. I am getting a 422 error with the following message:

Unprocessable Entity: file validation failed. The file is 2 MB exceeding the maximum file size of 1 MB.

I am working on tests. I will make sure to add coverage for uploading invalid files.

Added tests, addressed review remarks, fixed the security hole. This is ready for review again.

The tests are not running because our configuration is outdated. I created a ticket to restore the tests: 🐛 Restore test runs Needs review .

pfrenssen → thanks for your replay. But even after the last changes, I am still can upload files larger than the maximum file size for that field and upload files whose format is not accepted for those fields. I am using Drupal 10.2.1. Tested on Postman: Screenshot
Could you please describe the steps that you use to test the webform_rest file upload API?

@as_vard, thanks for the report. I wrote the tests using the standard REST testing. It works with Guzzle which uses Curl under the hood. Here is the test for an upload that exceeds the file size limit: https://git.drupalcode.org/project/webform_rest/-/merge_requests/9/diffs...

Do you get a failure when you run the tests?

@pfrenssen → I did not run the tests.

Hello, I am using this patch on D10.1.5, I followed the documentation step by step, but when I try to send the file I got following error:

Symfony\Component\HttpKernel\Exception\MethodNotAllowedHttpException: No route found for "GET http://example.com/webform_rest/job_applications/upload/cv": Method Not Allowed (Allow: POST) in Symfony\Component\HttpKernel\EventListener\RouterListener->onKernelRequest() (line 131 of /var/www/html/vendor/symfony/http-kernel/EventListener/RouterListener.php).

Has anyone faced this before, I appreciate any suggestions?

@gdurglishvili Error seems pretty explicit and is not in the module but in the code you use to send the file.
Server is expecting a POST request, and you are doing a GET.

Validation code seems indeed suspicious. I'm getting "Array to string conversion" warnings.
It seems we are sending to validation an array of validators instead of an array of fields?
Don't know much about validation, so I might be totally wrong.

Warning: Array to string conversion in Drupal\file\Plugin\rest\resource\FileUploadResource->validate() (line 45 of /var/www/experience/dev/web/core/modules/rest/src/Plugin/rest/resource/EntityResourceValidationTrait.php)
#0 /var/www/experience/dev/web/core/includes/bootstrap.inc(164): _drupal_error_handler_real()
#1 [internal function]: _drupal_error_handler()
#2 /var/www/experience/dev/web/core/modules/rest/src/Plugin/rest/resource/EntityResourceValidationTrait.php(45): array_diff()
#3 /var/www/experience/dev/web/modules/contrib/webform_rest/src/Plugin/rest/resource/WebformFileUploadResource.php(129): Drupal\file\Plugin\rest\resource\FileUploadResource->validate()
#4 [internal function]: Drupal\webform_rest\Plugin\rest\resource\WebformFileUploadResource->post()

FYI, patch no longer works with Drupal 10.3.0. I have not had the time to look into.

Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException: You have requested a non-existent service 'mime_type.guesser'

I cannot clone branch 2899902-sendupload-files so I've created a new (full) patch based on 4.x, with the changes from branch 2899902-sendupload-files and a fix for D10.3.

It's probably useful to attach the patch as well.

and for people who do not know, creating a patch from a MR is simple, just add the suffix ".patch" to the actual MR.

so this here is the patch: https://git.drupalcode.org/project/webform_rest/-/merge_requests/20.patch

Is this upload endpoint working for you for files with special characters in them? I.e. "filename™.pdf".

To be able to upload files with special characters in the filename, the filename in the Content-Disposition header should be encoded or it will fail even before uploading:

"Content-Disposition": `file; filename="${ encodeURI(file.name) }"`,

When I do this and upload a file with spaces in it, the filename is saved in Drupal with %20 instead of spaces, and as a result it throws an error when trying to download it:

InvalidArgumentException: The filename fallback cannot contain the "%" character. in Symfony\Component\HttpFoundation\HeaderUtils::makeDisposition() (line 182 of /app/vendor/symfony/http-foundation/HeaderUtils.php).

When I remove the encodeURI part, it works for most files but the POST will fail for files with special characters in them:

Cannot convert argument to a ByteString because the character at index 52 has a value of 8482 which is greater than 255.

@markwittens that is exactly what i do, but inside by external typescript application.

fileUploadHeaders.append( 'Content-Disposition',`file; filename="${encodeURIComponent(fileData.file.name)}"`);

I tested it with your example filename, and it works like a charm :)

Maybe you should concider rename it directly in the file webform field by using tokens?

@gooddev Are you renaming the file uploads using the "Rename files" option on the file upload component in Webforms? That might be the difference and why it works for you and not for me.

I've I added a patch for the issue branch that fixes the issue by decoding the filename before saving it. The patch also cleans up the function a bit and a I have replaced a deprecated function. Can you try if this works for your case as well?

I am currently working on a feature that involves submitting data to Drupal using the FormData format, particularly for forms that include file uploads. However, I've noticed that while data sent as JSON is processed correctly, the data sent using FormData is not being accepted by Drupal.

Could you please advise if there are any specific configurations or steps required on the Drupal side to ensure that FormData submissions are handled correctly? I want to ensure that I'm following best practices for handling such data submissions.

import { GetStaticPaths, GetStaticProps } from "next";
import { DrupalClient, DrupalNode } from "next-drupal";
import { useState, FormEvent } from "react";
import { useTranslation } from "next-i18next";
import { useRouter } from "next/router";

import { Body } from "@/components/body";
import { HeadingPage } from "@/components/heading--page";
import { Meta } from "@/components/meta";
import {
  CommonPageProps,
  getCommonPageProps,
} from "@/lib/get-common-page-props";

import { env } from "@/env";

interface MetaTag {
  tag: string;
  attributes: {
    name?: string;
    content?: string;
    rel?: string;
    href?: string;
  };
}

interface careersProps extends CommonPageProps {
  career: DrupalNode;
}

export default function careers({ career }: careersProps) {
    console.log('career' , career);

  const { t } = useTranslation();
  const [submissionStatus, setSubmissionStatus] = useState<"success" | "error" | null>(null);
  const router = useRouter();

  const imageLink = (career.metatag as MetaTag[] | undefined)?.find(
    (meta: MetaTag) =>
      meta.tag === "link" && meta.attributes.rel === "image_src",
  )?.attributes.href;

  async function handleSubmit(event: FormEvent<HTMLFormElement>) {
    event.preventDefault();
    
    const formData = new FormData(event.currentTarget);
    formData.append('webform_id', 'careers');
    formData.append('entity_id', career.id);

    try {
      const response = await fetch(
        `${env.NEXT_PUBLIC_DRUPAL_BASE_URL}/webform_rest/submit`,
        {
          method: "POST",
          body: formData,
        }
      );

      if (response.ok) {
        setSubmissionStatus("success");
        setTimeout(() => router.push('/careers'), 3000);
        console.log('Submission successful:', await response.json());
      } else {
        setSubmissionStatus("error");
        console.error('Submission error:', response.statusText);
      }
    } catch (error) {
      setSubmissionStatus("error");
      console.error('Submission failed:', error);
    }
  }

  return (
    <>
      <Meta title={career.title} metatags={[]} />
      <HeadingPage>{career.title}</HeadingPage>
      <div className="mt-8 space-y-6">
        {imageLink && (
          <div className="overflow-hidden rounded-lg shadow-lg">
            <img
              src={imageLink}
              alt={career.title}
              className="w-full h-auto object-cover"
            />
          </div>
        )}
        <div className="prose max-w-none">
          <Body value={career.body.processed} />
        </div>
        {submissionStatus === "success" ? (
          <div className="p-4 text-green-700 bg-green-100 border border-green-700 rounded">
            {t("Your application has been submitted successfully.")}
          </div>
        ) : submissionStatus === "error" ? (
          <div className="p-4 text-red-700 bg-red-100 border border-red-700 rounded">
            {t("There was an error submitting your application. Please try again.")}
          </div>
        ) : (
          <form onSubmit={handleSubmit} className="space-y-6">
            <div>
              <label className="block text-sm font-medium text-gray-700" htmlFor="subject">{t("Subject")}</label>
              <input
                type="text"
                id="subject"
                name="subject"
                required
                className="mt-1 block w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700" htmlFor="firstName">{t("First Name")}</label>
              <input
                type="text"
                id="firstName"
                name="firstName"
                required
                className="mt-1 block w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700" htmlFor="lastName">{t("Last Name")}</label>
              <input
                type="text"
                id="lastName"
                name="lastName"
                required
                className="mt-1 block w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700" htmlFor="email">{t("Email")}</label>
              <input
                type="email"
                id="email"
                name="email"
                required
                className="mt-1 block w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700" htmlFor="note">{t("Note")}</label>
              <textarea
                id="note"
                name="note"
                required
                className="mt-1 block w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
              ></textarea>
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700" htmlFor="cv">{t("CV")}</label>
              <input
                type="file"
                id="cv"
                name="cv"
                required
                className="mt-1 block w-full text-sm text-gray-500 file:mr-4 file:py-2 file:px-4 file:rounded-md file:border-0 file:text-sm file:font-semibold file:bg-indigo-50 file:text-indigo-700 hover:file:bg-indigo-100"
              />
            </div>
            <button
              type="submit"
            //   className="w-full inline-flex justify-center py-2 px-4 border border-transparent rounded-md shadow-sm text-sm font-medium text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500"
            >
              {t("Submit")}
            </button>
          </form>
        )}
      </div>
    </>
  );
}

export const getStaticPaths: GetStaticPaths = async () => {
  const drupalClient = new DrupalClient(env.NEXT_PUBLIC_DRUPAL_BASE_URL, {
    auth: {
      clientId: env.DRUPAL_CLIENT_ID,
      clientSecret: env.DRUPAL_CLIENT_SECRET,
    },
  });

  const careersResponse = await drupalClient.getResourceCollection<DrupalNode[]>(
    "node--career_position",
    {
      params: {
        "fields[node--career_position]": "id",
        "page[limit]": 50,
      },
    },
  );

  const paths = careersResponse.map((career) => ({
    params: { id: career.id },
  }));

  return {
    paths,
    fallback: "blocking",
  };
};

export const getStaticProps: GetStaticProps<careersProps> = async (
  context,
) => {
  const { id } = context.params as { id: string };
  const drupalClient = new DrupalClient(env.NEXT_PUBLIC_DRUPAL_BASE_URL, {
    auth: {
      clientId: env.DRUPAL_CLIENT_ID,
      clientSecret: env.DRUPAL_CLIENT_SECRET,
    },
  });

  try {
    const career = await drupalClient.getResource<DrupalNode>(
      "node--career_position",
      id,
    );

    return {
      props: {
        ...(await getCommonPageProps(context)),
        career,
      },
      revalidate: 60,
    };
  } catch (error) {
    console.error("Error fetching career post:", error);
    return {
      notFound: true,
    };
  }
};

mime_type.guesser service not found, Drupal 10.3.5

This patch fixes the following error in the patch attached in #98

```
Call to undefined method Drupal\Component\Utility\Bytes::toInt()
```

in a couple of places. And it works correctly with Drupal 10.3.10

Ready for review.

Thank you!

Please add proposed changes to the MRs instead of posting them in patches, it is easier to review commits in MRs. It is counterproductive to download patches and comb through them line by line to discover which lines have changed.

It looks like something went wrong with MR #20. It contains ~20000 lines of unrelated code. I propose we close that one and backport the Drupal 10.3 fixes into the original MR.

pfrenssen → changed the visibility of the branch 2899902-drupal-10-3-1-renamefile to hidden.

In Drupal 10.3 the FileUploadResource class has been completely refactored (ref. [#3402032]). The new code is much cleaner and leverages a FileUploadHandler which abstracts away much of the complexity of uploading files.

Unfortunately this change means that we will lose compatibility with Drupal 10.2 and below. This might be controversial since we are at the moment still compatible even with Drupal 8.7.

I personally think we should go ahead with this since Drupal 10.2 and below are unsupported. I will make a new MR though so that projects on older versions of Drupal can still use MR #9.

Unfortunately this change means that we will lose compatibility with Drupal 10.2 and below.

It might be worth creating a new major version for 10.3 and up.

pfrenssen → changed the visibility of the branch 2899902-sendupload-files to hidden.

In the meantime Webform REST 4.2.0 → has been released and it already requires Drupal 10.3+ as a minimum, so this is not controversial any more :) The old release 4.1.0 still supports older versions of Drupal.

We also have test coverage again which is great!

I have backported the changes from @markwittens and updated the code to be conform the latest changes in Drupal 10.3+. Fixed tests and cleaned up.

As far as I can see everything discussed in the last bunch of comments has been addressed. I will also upload a patch with the latest state of the MR for the patch lovers.

What is not included is the support for filenames with special characters which was discussed in #2899902-94: Support file uploads → - #2899902-95: Support file uploads → - #2899902-96: Support file uploads → . I noticed that the FileUploadResource from the core File module doesn't do anything special to handle special characters, which prompted me to do some further research.

1. It was suggested in comment #94 to URL encode the filename, but this is not supported by the W3C standard (ref. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Dispos...). Probably there was some confusion between the filename and filename* parameters. The filename parameter we use should be quoted, not URL encoded.
2. Drupal core already offers developers a way to handle filename sanitization: the FileUploadSanitizeNameEvent event listener. The idea behind this is that uploaded files are sanitized consistently across the whole site, no matter which module is handling the uploads.
3. There are already multiple modules that offer filename sanitization using the above event. Examples are Transliterate Filenames → and Filename Transliteration → .
4. Drupal 11 has native support for handling special characters, this is amazing! It has been added in ✨ Provide options to sanitize filenames (transliterate, lowercase, replace whitespace, etc) Fixed

So in conclusion we should not handle filename sanitizing ourselves but let dedicated modules handle it. Drupal 11 has it built in, while users of Drupal 10 and below can use the Transliterate Filenames or Filename Transliteration modules.

In case you have been using an older version of the patch, you'll need to update the plugin ID of the file upload REST resource, since the latest versions of the patch have switched from the webform_rest_file_upload to webform_rest:file_upload to follow the convention used in the Attributes based plugin definition in Drupal 10.3.

You can use an update hook to fix this, something similar to this should do the trick:

/**
 * Update the plugin name of the Webform REST file upload resource.
 */
function mymodule_update_10101() {
  // In recent versions of the patch we are using to add file upload capability
  // to the Webform REST resource, the plugin ID was changed.
  // @see https://www.drupal.org/project/webform_rest/issues/2899902
  $config = \Drupal::configFactory()->getEditable('rest.resource.webform_rest_file_upload');
  $config->set('plugin_id', 'webform_rest:file_upload');
  $config->save();
}

Very thorough changes, look great to me :)

You can also use drush after updating to rename the plugin ID:

drush php-eval '$config = \Drupal::configFactory()->getEditable("rest.resource.webform_rest_file_upload");
  $config->set("plugin_id", "webform_rest:file_upload");
  $config->save();'

I'm coming from an older patch. There seems to be merge conflict with MR #25. I'd try to fix it but I'm still getting acquainted with the new (well, not new anymore) way of doing things.

What is the patch in 107? For 10.2 and below?

Thx -

Solved the merge conflict.

The patch in #107 is an export of the MR for people who prefer to use a fixed patch rather than the moving head of an MR. It should work against the latest 4.2.0 release of the module on supported versions of Drupal (10.3 and up).

I appreciate that.

webform_rest 4.2.0 depends on Webform 6.3.0-beta, so I wasn't planning on going to 4.2.0 just yet. Is there a patch for those using 10.3.x and webform_rest 4.1.0?