drupal_move_uploaded_file() produces twice as much I/O than necessary

Created on 5 January 2012, over 12 years ago
Updated 14 January 2024, 6 months ago

drupal_move_uploaded_file() currently attempts to call move_uploaded_file() with an unresolved destination URI. If this fails it will fall back to passing a resolved URI. This is to support safe_mode and open_basedir.

If drupal_move_uploaded_file() is passed an unresolved URI it will perform a file copy followed by an unlink. This produces a considerable amount of I/O. If it is passed a resolved URI (a filepath) it will perform a file rename which produces negligible I/O.

The attached patch means that it always attempts to resolve the destination URI that is passed to move_uploaded_file(). If the URI does not resolve (it is a remote scheme) then move_uploaded_file() will fall back to copying and deleting. This will not cause write I/O because the scheme is remote.

I tested the performance changes of the patch by uploading an Ubuntu .iso file to a file field and measuring the I/O using

sudo iotop -Pao

Without the patch it takes 1390.60 MB of writes.
With the patch it takes 695.30 MB of writes (the file is only written to disk once).

On a loaded server the improvement would be even more. During the testing, my laptop had sufficient free RAM for the written file to be cached in RAM. This meant it could be written to the destination without having to be read. If there was no free RAM there would have to be 695.30 MB of reads.

So - this patch reduces the amount of I/O required for a file upload by 50 - 66 %.

There are already extensive tests for uploading files to local and remote URI schemes.

πŸ› Bug report
Status

Needs work

Version

11.0 πŸ”₯

Component
File systemΒ  β†’

Last updated about 8 hours ago

Created by

πŸ‡¬πŸ‡§United Kingdom jbrown

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • First commit to issue fork.
Production build 0.69.0 2024