Millions of cache files create issues

I can see from earlier issues that there is already focus on the potentially huge number of files that File Cache can generate.

For for now I solved my problem with a script that simply deletes the oldest files. This seems to have very little influence on the performence of the site, but certainly keeps the number of files in a manageable range.

For whatever it's worth to others, here's my script. I called it delete_oldest.sh:

#!/bin/bash

# Constants
NUM_FILES_TO_DELETE=1000

# Usage function
usage() {
  echo "Usage: $0 [-y] /path/to/directory"
  echo "  -y    Skip confirmation prompt"
  exit 1
}

# Check for arguments
CONFIRM=false
while getopts ":y" opt; do
  case ${opt} in
    y )
      CONFIRM=true
      ;;
    \? )
      usage
      ;;
  esac
done

# Shift out processed options
shift $((OPTIND -1))

# Get directory argument
TARGET_DIR="$1"

if [ -z "$TARGET_DIR" ]; then
  usage
fi

# Make sure the directory exists
if [ ! -d "$TARGET_DIR" ]; then
  echo "Error: Directory '$TARGET_DIR' does not exist."
  exit 1
fi

# Confirm if not using -y
if [ "$CONFIRM" = false ]; then
  read -p "Are you sure you want to delete the $NUM_FILES_TO_DELETE oldest files in '$TARGET_DIR'? [y/N]: " answer
  case "$answer" in
    [yY][eE][sS]|[yY])
      ;;
    *)
      echo "Operation cancelled."
      exit 0
      ;;
  esac
fi

# Delete oldest files
find "$TARGET_DIR" -type f -printf '%T@ %p\n' | \
  sort -n | \
  head -n "$NUM_FILES_TO_DELETE" | \
  cut -d' ' -f2- | \
  xargs -d '\n' rm -f

echo "Deleted the $NUM_FILES_TO_DELETE oldest files in '$TARGET_DIR'"

It's run by cron every 30 minutes for the directories that seems to grow fastest and largest.

*/30 * * * * /usr/bin/env delete_oldest.sh -y /var/filecache/page > /dev/null
*/30 * * * * /usr/bin/env delete_oldest.sh -y /var/filecache/render > /dev/null
*/30 * * * * /usr/bin/env delete_oldest.sh -y /var/filecache/dynamic_page_cache > /dev/null
*/30 * * * * /usr/bin/env delete_oldest.sh -y /var/filecache/data > /dev/null

/var/filecache/ is of course the location of the cache flies, and needs to be adjusted according to your setup.

The script can also run from the command line as
delete_oldest.sh /var/filecache/render

My version deletes the oldest 1000 files, and for now that seems to tame the file numbers. I started out with 10000 to get the number of files reduced, but after a while it seems that 1000 is an OK number to cull.

Martin

This appears to be a duplicate of https://www.drupal.org/project/filecache/issues/3001324 ✨ Limit number of cache files in cache bin directories Needs work . Martin, would you be interested in testing my patch in https://www.drupal.org/project/filecache/issues/3001324#comment-16058304 ✨ Limit number of cache files in cache bin directories Needs work to see if that helps? It splits the cache items into sub-directories within each cache bin directory. We've patched our sites with it, and it has made a dramatic difference. But I don't think any of the sites are as large as the one you describe.