Dealing with unexpected file deletion due to incorrect file usage

Created on 22 October 2016, over 7 years ago
Updated 26 January 2024, 5 months ago

Problem/Motivation

Recently @catch, @webchick, @xjm, @cottser, @cilefen and I discussed #2801777: Give users the option to prevent drupal from automatically marking unused files as temporary β†’ , πŸ› $entity->isDefaultTranslation() behaves incorrectly when changing default translation, causing file/image field usage to be set to zero, causing files to be deleted Fixed and #2708411: [PP-1] editor.module's editor_file_reference filter not tracking file usage correctly for translations β†’ . We all agreed that the current file usage behaviour that is resulting in files being deleted when they are being used by a site is unacceptable and a critical issue.

This was also discussed at Drupal Dev Days Seville β€” see the meeting notes.

This issue is exist to propose a series of steps to mitigate and fix these related issues.

The current behaviour

As files are "used" in different situations we add to the counter in the file_usage table. As the usages are removed we subtract from the counter. Once the counter gets to 0 the file is made temporary in \Drupal\file\FileUsage\FileUsageBase::delete(). Then a cron process (file_cron()) comes along and deletes temporary files after system.file:temporary_maximum_age seconds.

There are several cases where that counter is not incremented or decremented correctly.

Regardless of all the sub-issues we have the known issue that this does not track usages added by content editors if they are able to add links to file but editing the source HTML. Also the system of maintaining a counter seems very fragile and evidence supports that :).

Which cases/issues are there?

These fall into clear buckets, along three axes:

  1. Field (image/file/entity reference) versus editor.module's editor_file_reference filter
  2. Entity revisions versus entity translations
  3. Data loss: yes or no

Proposed resolution

TBD

Remaining tasks

TBD

User interface changes

TBD

API changes

TBD

Data model changes

TBD

🌱 Plan
Status

Active

Version

11.0 πŸ”₯

Component
File systemΒ  β†’

Last updated 1 minute ago

Created by

πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ

Live updates comments and jobs are added and updated live.
  • Triaged core critical

    There is consensus among core committers that this is a critical issue. Only core committers should add this tag.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ΊπŸ‡ΈUnited States maskedjellybean Portland, OR

    I think @Paul Dudink had a good suggestion in #87 to look into the entity_usage β†’ module. If all this is caused by file_usage counts being wrong, shouldn't we look at how a contrib module is doing the same thing and see if it's better?

    I'm also wondering has anyone has tried media_files_handler β†’ ?

    For those of us who need deleted files to actually be deleted, what is the best option? Just enable $config['file.settings']['make_unused_managed_files_temporary'] = TRUE; and cross our fingers?

  • πŸ‡¨πŸ‡­Switzerland Berdir Switzerland

    For those of us who need deleted files to actually be deleted, what is the best option? Just enable $config['file.settings']['make_unused_managed_files_temporary'] = TRUE; and cross our fingers?

    Not cross your fingers, but actively verify that it works for your site and your use cases.

    That this setting even exists is a stopgap workaround, to remove it again, we need to fix the bugs around file usage and we need people to test it and report issues they find, if any. FWIW, πŸ› $entity->isDefaultTranslation() behaves incorrectly when changing default translation, causing file/image field usage to be set to zero, causing files to be deleted Fixed is the only one that I'm aware of, we have that setting enabled on quite a lot of production sites and it's working well for us.

    If you have a multilingual site, you definitely want to use that.

  • πŸ‡¦πŸ‡·Argentina hanoii πŸ‡¦πŸ‡·UTC-3

    I started researching what's going on around this issue. I will likely use 'make_unused_managed_files_temporary' for now, but I was wondering if a module that (as opposed to most of the modules out there that tries to add some smarter capability) that adds a setting to each file field so that you can manually set which file file flags a file for deletion when the entity is removed.

  • πŸ‡¬πŸ‡§United Kingdom catch

    Adding 🌱 Add a reliable entity-usage system to core Active to related issues.

Production build 0.69.0 2024