Uploaded files are impossible to replace

Has anyone attempted to solve this problem by automatically creating a redirect from the old file URL to the new file URL when Media is edited and a new file is uploaded? Are there any obvious reasons it couldn't work?

I've seen some modules that create a redirect from a constant URL to the actual file URL, but these all rely on content editors understanding that they should never link directly to the file URL. I don't think content editors should have to remember this and I don't trust that they will.

The old file URL will still exist on the filesystem in some cases, in which case a Drupal-created redirect will not work. Your web server won't even involve Drupal when someone requests a file that exists in the filesystem.

#73 I still think, performance tradeoff aside, permanent URI a la node/123 is preferable over raw file URLs. They will always work, until that media item is deleted. Whatever file it holds, its name or path is irrelevant. As bkosborne points out in #74, the application won't have a chance to respond when a file like that is requested.

Ah, I hadn't thought of that. Dang.

I can see how a permanent URI would be the ideal solution for a site without many editors and where everyone adheres to the rules. It won't work for my situation unfortunately. I am testing your media_entity_file_replace module bkosborne, and it seems to be working great so far. Hopefully I can convince the team that being able to replace files and maintain the URL is more important than revisions on media. Seems like a fair trade based on how frustrated editors are with the current file replace situation. Thanks for your work!

I've been able to replace and rename files in Drupal 9 without any custom code by using the File (Field) Paths module → . Following this awesome blog post, Better File Management for Drupal 8 and Drupal 9 – Part 1 I was able to give content editors the power to rename media files with normal sentence case and spacing, but have ideally managed filenames renamed by File Field Paths:

Media name:
Product Handbook 2023

Filename:
product-handbook-2023.pdf

In addition, if this were a media entity type Document and had a taxonomy term attached to this e.g. onboarding, I was able to get FFP to automatically place these in a structured location for easy file management:

/sites/default/files/documents/onboarding/product-handbook-2023.pdf

The only difference between what I did and what you'll find in the blog post is I have unchecked "Create Redirect" and "Retroactive update" and only have checked "Active updating" under the FFP settings in the file field settings. Retroactive update is useful if you have existing files that you want to update, and you'll only have to check this box once, and after it runs it unchecks itself. I've also used this to change file systems from private > public or public > private and update the file paths for my media files.

I am also using the Media file delete module → so that users are managing media entities rather than Files themselves. If the media entity is deleted, so is the file. I set the default value to be checked:

/**
 * Implements hook_form_BASE_FORM_ID_alter().
 */
function MYMODULE_form_media_confirm_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  // Set confirm delete default value for media files.
  $form['also_delete_file']['#default_value'] = 1;
}

Finally, I have the following in my settings.php file, so that if I want to make sure orphaned files are removed, I can either wait for the next cron run or trigger a manual cron job.

/**
 * File settings (Custom).
 *
 * This will remove orphaned (deleted) files from the file system on the
 * next cron run.
 */
$config['file.settings']['make_unused_managed_files_temporary'] = TRUE;
$config['system.file']['temporary_maximum_age'] = 1;

Out of scope, but:

> $config['system.file']['temporary_maximum_age'] = 1;

This is wrong and will delete files that users are in process of creating if cron runs between uploading a file and saving the entity. Introducing this setting at all was a mistake, this should always be set to the same as the form cache max age (6h by default), that's the reason temporary files aren't immediately deleted. And we should also reintroduce the logic that immediately deletes files as the change from permanent to temporary/lose all usages.

Has anyone come up with a way to make the file replacements when uploading media in bulk? I'm surprised that has not been brought up.

Created a related ticket for better authorisation 🐛 File entity update is allowed only for user who has uploaded the file Active

Been 2 months with no word so upping to framework manager

I did try testing the feature but when I go to replace the file I get a fatal error

Error: Call to a member function getFileName() on false in Drupal\file\FileForm->validateForm() (line 108 of core/modules/file/src/FileForm.php).

Not getting a fatal error.

but tagging for usability review now as it's not clear how it's suppose to function

Assumed I could replace the file with whatever but that doesn't work, has to be the same name apparently but had to read several threads to figure that out.

Also on an Umami install the edit link doesn't appear for any of the image/jpeg files created. Not sure why

@smustrave: the Problem / motivation states

There seems to be no way in Drupal to overwrite a file, and keeping the same filename.

But how is anyone suppose to know that without reading this entire ticket?

Anyway 100% needs usability review before going any further

Am moving to NW as not all files are getting the edit button.

One detail i've noticed while initially testing the MR is that after you have replaced an image for the first time, the image "sticks" to that image from this point on. Meaning i've tried to replace the image two more times, so i had replace 1, replace 2, and replace 3. when you click on the name on admin/content/files after the third replace you see the image of the first replace while the thumbnail on admin/content/media shows the image of the third replace. Is that a caching issue?

@smustgrave
1. umami issue: For the files created by the install, the UID column in file_managed is null. That coupled with issue 🐛 File entity update is allowed only for user who has uploaded the file Active results in Edit button not shown in umami profile
2. All files are not getting edit button: This is due to the current permission implementation. Check 🐛 File entity update is allowed only for user who has uploaded the file Active
3. w.r.t. needing the same file name: It's a bit of a learning curve. So let's await for usability review.

@rkoller
yes it's a caching issue. That's fixed too. If you pull the latest code, kindly clear cache before testing.

thanks @sukr_s, i can confirm that the caching issue is being fixed by your changes. tested replacing an image three times in a row, each time the new replacement got shown correctly now.

Usability review

We discussed this issue at 📌 Drupal Usability Meeting 2024-08-16 Needs work . That issue will have a link to a recording of the meeting. For the record, the attendees at today's usability meeting were @benjifisher, @rkoller, and @simohell.

In general we've had a clear consensus that the issue is solving a longstanding problem and the general direction looks good. During our testing and the subsequent discussion, the following points emerged:

• If the user clicks the default option Edit on a drop button on admin/content/files the person gets to a page with Edit [file name] | [Site name] in the page title and Edit [file name] in the h1, but the page itself only contains the Replace file component. The default option for a drop button on admin/content/media is also Edit with Edit image [file name] in the h1 and Edit image [file name] | [Site name] in the page title. The page itself mirrors the drop button options as local tasks (edit, delete, revision, translate), and contains an image field set, plus vertical tabs for "Revision information", "URL alias", and "Authoring information", as well as a checkbox for the publish state. So strictly speaking the user is provided with two completely different sets of actions/functionality clicking a button labeled Edit. In the context of Files, Edit is sort of a mislabel.
• As already noted the page for a media item mirrors the available options in a drop button with edit, delete, revision, translate. For files no local items exist, you only have the separate edit/replace page while for delete you only get a confirmation dialogue.
  If you try to replace a file by picking the desired new file (which has a different file name) you run into the error, that the name does not match the existing file. If you new adjust the filename of your desired new file in the file system and directly retry to upload and save you end up with the replaced file as well as a file with a temporary status which is used in zero places - it has the file name of the first failed replacement attempt.
• If set the maximum image dimension in image field settings of your image media type to for example 500x600px and then add an image with those exact dimensions and now try to replace that file with an image with the dimensions of 900x1200px that runs through without any complaint. That might be in particular problematic in case the maximum dimension might been picked for a reason, due to for example memory limitations on the server in regards of PHP.

In the following a list of recommendation we've agreed on:

1. On admin/content/files change the button label from Edit to Replace and change the page title from Edit [file name] | [Site name] to Replace [file name] | [Site name] and the h1 from Edit [file name] to Replace [file name].
2. On the replace page for a file add the local items Replace and Delete analogous to media items since Revisions and Translate are not available for Files.
3. Spare the user the worry to make sure upfront that the file the person wants to upload is matching the filename of the file that should be replaced. The person knows the current file and knows with which file it should be replaced. The filename shouldn't be a barrier that leads into an error and annoyance. The user should never run into that error message, Drupal should handle the replacement and naming of the file if possible.
4. The replacement function should check and respect minimum and maximum dimension constraints in case they are set in the image field settings.

I'll remove the "Needs usability review" tag again

and the MR also needs a rebase due to the following commit https://git.drupalcode.org/project/drupal/-/commit/8b368d712d83900765744... which introduced a fix for an issue with a naming case which lead to the following error when i try to checkout the feature branch

$> git checkout 2648816-uploaded-files-are                                  
error: The following untracked working tree files would be overwritten by checkout:
	core/modules/system/tests/modules/driver_test/src/Driver/Database/DrivertestMysql/Connection.php
	core/modules/system/tests/modules/driver_test/src/Driver/Database/DrivertestMysql/Install/Tasks.php
	core/modules/system/tests/modules/driver_test/src/Driver/Database/DrivertestMysqlDeprecatedVersion/Connection.php
	core/modules/system/tests/modules/driver_test/src/Driver/Database/DrivertestMysqlDeprecatedVersion/Install/Tasks.php
	core/modules/system/tests/modules/driver_test/src/Driver/Database/DrivertestPgsql/Connection.php
	core/modules/system/tests/modules/driver_test/src/Driver/Database/DrivertestPgsql/Install/Tasks.php
Please move or remove them before you switch branches.
Aborting

a big big big thanks to @rfay who helped to figure out the root cause of this error on checkout. i was unable to figure it out myself i 've only had a hunch a rebase "might" solve due to the time since the last commit and that there "might" be something introduced since then. but randy figured out the what part. thanks again.

I cleaned up the code a bit and used same terminology as media_entity_file_replace module is using.

Addresses the usability review recommendation #96.3 by not requiring the user to use same file name as the existing file. However requires the mime type to be same as existing file.

#96.1 and #96.2 might not be feasible since EntityBase does not offer "replace" function.
#96.4: That would require reliable entity-usage system, so I'd suggest creating a separate issue for it and postponing it by one of the entity usage issues.

Generally in support of the feature but I think a bit more thought needs to be put into the access side.

The route requires edit access to the file, which is only allowed (with core) by the owner of the file. I also don't see much test coverage around access either?

Do we want to focus efforts on 🐛 File entity update is allowed only for user who has uploaded the file Active first?

Still needs more access based testing.

removed invalidated test case and added new test case to cover different mime type upload.

The Needs Review Queue Bot → tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

Rebased.

The Needs Review Queue Bot → tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

Note: The query string that's added is an interesting approach, but fundamentally not a solution. I mentioned before that it's similar to what crop module does, but that's not entirely true. It still results in a different URL, that difference just happens to be in the query string and not the filename. The result is the same. Depending on your caching situation/infrastructure, original links are _not_ guaranteed to fetch the most recent version of a file.

Those limitations might be OK as a contrib module, where you can document that and users will be aware of that. It's however IMHO challenging to add this as a default feature to core, knowing that it will not work reliably in all cases.

If that's your requirement, then I recommend, as mentioned by #32 and #38, to try https://www.drupal.org/project/media_entity_download → , which I maintain. It doesn't expose your file names and is instead only tied to the media entity, resulting in non-cacheable or only short-term caches responses. There are downsides like the mentioned performance issues and you will need to integrate using those links into your processes (there are formatters as well as a linkit integration) and instruct your editors to use it.

As a starting point, I'd suggest updating the issue summary to explain what workarounds are being implemented here, limitations of that and links to alternatives like media_entity_download.