Temporary folder full with temporary files

Created on 24 June 2024, 5 months ago

Problem/Motivation

Currently in our sites we're having issues with a lot os temporary files on temporary folder. Examples:

253M /tmp/magick-65335shsBIw99yoal
253M /tmp/magick-65335NRGCQ7oluwF9
253M /tmp/magick-53473muHYCH46ZTmV
253M /tmp/magick-53473HtZ7Skg6ch5q

The disks was full and we had other issues.
As worked around we removed them from filesystem

Proposed resolution

Garbage collector to run a cleanup on cron

I've been talking to @aspilicious about:

  1. On admin area will have a checkbox: β€œRun garbage collector on cron to clean temporary files”
  2. Unchecked by default
  3. If checked the hook_cron will remove the temporary files
  4. Remove files older than 1h

Remaining tasks

  1. Create admin area
  2. Implements hook_cron

User interface changes

N/A

API changes

N/A

Data model changes

N/A

✨ Feature request
Status

Active

Version

1.0

Component

Code

Created by

πŸ‡§πŸ‡·Brazil renatog Campinas

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @renatog
  • Status changed to Needs review 5 months ago
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    This is an initial patch, but is already working

  • πŸ‡§πŸ‡·Brazil renatog Campinas
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    Did few updates on patch #2

    • Added / Improved comments
    • Fixed coding standards
    • Minor improvements for code reading and future maintainability

    To be easier to review I created a Merge Request: https://git.drupalcode.org/project/imagick/-/merge_requests/10

  • πŸ‡§πŸ‡·Brazil renatog Campinas
  • πŸ‡§πŸ‡·Brazil renatog Campinas
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    MR updated pointing to the dev branch

    If someone needs, Just attached a patch with target for version 1.6 stable so it can be applied though composer

  • Status changed to Needs work 5 months ago
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    Necessary to update the service to get dynamically from temp folder

  • Status changed to Needs review 5 months ago
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    MR updated pointing to the dev branch

    If someone needs, Just attached a patch with target for version 1.6 stable so it can be applied though composer

  • Status changed to Needs work 5 months ago
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    Moving to NR to apply the suggestion above

  • πŸ‡§πŸ‡ͺBelgium flyke

    Thank you for your great work, this really should be included, and needs info on the module readme.
    We have developed a DAM system for our clients based on Drupal. Imagick is used for lots of things, for example to convert selected assets to lower-res versions or to other image formats.
    Since one of our DAM systems was reaching the hosting storage limit, I started investigating by getting the top 5 biggest folders via following ssh command:
    du -Sh | sort -rh | head -5

    Then I immediatly saw that the tmp folder was 96GB, which is very large. It contained lots and lots of /magick-... files.
    I patched the imagick module with this MR, enabled the garbage collector, deployed these changes and ran cron.
    Result is that our tmp folder has been reduced from 96GB to 2GB.
    Needless to say, I am very happy with this functionality.

  • πŸ‡§πŸ‡·Brazil renatog Campinas

    Thanks for sharing your feedback @flyke

  • First commit to issue fork.
  • πŸ‡¬πŸ‡§United Kingdom rossb89 Bristol

    I've update the MR with the following:

    • * merged in the latest changes from 8.x-1.x so there are no merge conflicts at time of writing
    • * fixed the config type values in the imagick.config.yml
    • * added the missing properties to the imagick.schema.yml
    • * made the suggested change of moving (and modifying) the check to skip processing if garage collector is disabled, into hook_cron instead of being within the service itself.
  • πŸ‡§πŸ‡·Brazil renatog Campinas

    made the suggested change of moving (and modifying) the check to skip processing if garage collector is disabled, into hook_cron instead of being within the service itself

    Appreciated

    Seems really good, but as suggested on #12 I think would be nice if we update the README.md with this functionality

  • πŸ‡§πŸ‡·Brazil renatog Campinas

    Added the README.md as suggested on #12

Production build 0.71.5 2024