Aggregated files can be created during a cache rebuild, which can lead to an inability of cache rebuild to delete directories which now have files in them

Created on 10 November 2023, about 1 year ago
Updated 18 June 2024, 6 months ago

Problem/Motivation

If the drush cr command takes a bit longer it seems like new files are already written into the css / js folder while the files get deleted recursively. Than the deletion of the folder gives a warning that the directory is not empty.

> > [warning] rmdir(.....sites/default/files/css): Directory not empty FileSystem.php:267
> > [warning] rmdir(.......sites/default/files/js): Directory not empty FileSystem.php:267

Steps to reproduce:
For example make a test php script that writes data in the css folder:
<?php
$directory = './docroot/sites/default/files/css';
while (TRUE) {
$filename = $directory . '/' . uniqid('css', TRUE) . '.css';
file_put_contents($filename, 'test');
usleep(100000);
}

Turn aggregation on

execute drush cr

πŸ› Bug report
Status

Active

Version

11.0 πŸ”₯

Component
Asset libraryΒ  β†’

Last updated 1 day ago

No maintainer
Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Issue created by @shagel
  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    I get a similar message whenever I run a PHPUnit (functional) test locally:

    % ddev ssh
    $ vendor/bin/phpunit -c /var/www/html/docroot/core .../mymodule/tests/src/Functional/myTest.php
    PHPUnit 9.6.17 by Sebastian Bergmann and contributors.
    
    Testing Drupal\Tests\mymodule\Functional\myTest
    E                                                                   1 / 1 (100%)
    
    Time: 00:45.250, Memory: 6.00 MB
    
    There was 1 error:
    
    1) Drupal\Tests\mymodule\Functional\myTest::testSomething
    rmdir(sites/simpletest/66945416/files/php/twig): Directory not empty
    
    /var/www/html/docroot/core/lib/Drupal/Core/File/FileSystem.php:267
    /var/www/html/docroot/core/lib/Drupal/Core/File/FileSystem.php:372
    /var/www/html/docroot/core/lib/Drupal/Core/File/FileSystem.php:368
    /var/www/html/docroot/core/lib/Drupal/Core/File/FileSystem.php:368
    /var/www/html/docroot/core/lib/Drupal/Core/File/FileSystem.php:368
    /var/www/html/docroot/core/tests/Drupal/Tests/BrowserTestBase.php:440
    /var/www/html/docroot/core/tests/Drupal/Tests/BrowserTestBase.php:451
    /var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    

    I think the problem is that the directories have some read-only files:

    $ ls -la docroot/sites/simpletest/66945416/files/php/twig/6600805a5f742_form.html.twig__H1nbTzrPjfuHbUA8mlc314Nq/
    total 8
    drwxrwxrwx  4 benji dialout  128 Mar 24 19:34 .
    drwx------ 19 benji dialout  608 Mar 24 19:34 ..
    -r--r--r--  1 benji dialout  685 Mar 24 19:34 .htaccess
    -r--r--r--  1 benji dialout 3216 Mar 24 19:34 S0AF3M2xqzavbsX8kSimZT3HqUxKvNg8rT1PXMN6340.php
    

    I can remove the file, but only with a confirmation:

    $ rm docroot/sites/simpletest/66945416/files/php/twig/6600805a5f742_form.html.twig__H1nbTzrPjfuHbUA8mlc314Nq/.htaccess 
    rm: remove write-protected regular file 'docroot/sites/simpletest/66945416/files/php/twig/6600805a5f742_form.html.twig__H1nbTzrPjfuHbUA8mlc314Nq/.htaccess'? y
    $ ls -la docroot/sites/simpletest/66945416/files/php/twig/6600805a5f742_form.html.twig__H1nbTzrPjfuHbUA8mlc314Nq/
    total 4
    drwxrwxrwx  3 benji dialout   96 Mar 24 19:39 .
    drwx------ 19 benji dialout  608 Mar 24 19:34 ..
    -r--r--r--  1 benji dialout 3216 Mar 24 19:34 S0AF3M2xqzavbsX8kSimZT3HqUxKvNg8rT1PXMN6340.php
    
  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    I do not know whether my problem is the same as what @shagel sees, but I realize now that

    1. I should have mentioned that I see the same thing when running a core test, like core/modules/node/tests/src/Functional/NodeTitleTest.php.
    2. The test fails with the message in #6 when I use a new Mac.
    3. The test passes when I use an older Linux computer.

    My work computer is a shiny, new Mac with an M3 ARM processor. I use DDEV and Colima to run Drupal, including the PHPUnit tests. There must be some problem with how the Docker container works with the underlying filesystem.

    My personal computer is Ubuntu with an AMD chip.

  • πŸ‡ΊπŸ‡ΈUnited States mark_fullmer Tucson

    I can corroborate that our team has experienced this specifically on MacOS using Docker Desktop > v4.20. We were able to reliably confirm that previous versions of Docker Desktop did not result in this same problem.

  • πŸ‡«πŸ‡·France nod_ Lille

    So something like having a lock set on cache clear, and only serving the assets without writing to the folder when the lock is present?

  • πŸ‡§πŸ‡ͺBelgium stefvanlooveren

    We also experencied some deployments where not all aggregated files were deleted. I figure a lot of devOps systems might be having problems regarding this issue.

Production build 0.71.5 2024