copy() can fail in FunctionalTestSetupTrait::prepareSettings() because of a race condition

Created on 7 January 2021, over 3 years ago
Updated 8 March 2023, over 1 year ago

Problem/Motivation

Similarly to the other file operations that can fail under reasonable load caused by parallel requests (see referenced issues), it seems the the copy() operation can also fail in \Drupal\Core\Test\FunctionalTestSetupTrait::prepareSettings(). Even though the destination direction should exist and should be writable based on the call stack it seems at least one of these are not fulfilled when multiple tests are running parallel.

The error from copy() you can see below means the issue is with the destination and not with the source, I check it, if the issue is with the source, the error is different.

[2021-01-07T07:35:46.090Z] There was 1 error:

[2021-01-07T07:35:46.090Z] 

[2021-01-07T07:35:46.090Z] 1) Drupal\Tests\MY_MODULE\FunctionalJavascript\MyModuleTabBlockTest::testApiTabs

[2021-01-07T07:35:46.091Z] copy(/mnt/files/local_mount/build/web/sites/simpletest/50370776/settings.php): failed to open stream: No such file or directory

[2021-01-07T07:35:46.091Z] 

[2021-01-07T07:35:46.091Z] /mnt/files/local_mount/build/web/core/lib/Drupal/Core/Test/FunctionalTestSetupTrait.php:80

[2021-01-07T07:35:46.091Z] /mnt/files/local_mount/build/web/core/tests/Drupal/Tests/BrowserTestBase.php:572

[2021-01-07T07:35:46.091Z] /mnt/files/local_mount/build/web/core/tests/Drupal/Tests/BrowserTestBase.php:406

[2021-01-07T07:35:46.091Z] /mnt/files/local_mount/build/web/modules/contrib/MY_MODULE/tests/src/FunctionalJavascript/ApiRefTestBase.php:45

[2021-01-07T07:35:46.091Z] /mnt/files/local_mount/tests/src/FunctionalJavascript/MyModuleTestBase.php:93

[2021-01-07T07:35:46.091Z] /mnt/files/local_mount/tests/src/FunctionalJavascript/MyModuleTabBlockTest.php:124

[2021-01-07T07:35:46.091Z] 

[2021-01-07T07:35:46.091Z] ERRORS!

[2021-01-07T07:35:46.091Z] Tests: 1, Assertions: 0, Errors: 1.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Needs work

Version

10.1

Component
PHPUnit 

Last updated about 1 hour ago

Created by

🇭🇺Hungary mxr576 Hungary

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇺🇸United States smustgrave

    So if this were an issue wouldn't we see this random fail on other tickets?

    Can we update the MR for 10.1 please.

  • 🇭🇺Hungary mxr576 Hungary

    So if this were an issue wouldn't we see this random fail on other tickets?

    AFAIK tests on Drupal CI are not running in parallel... but we do execute them in parallel to save time with the help a tool like https://github.com/Pronovix/testrunner.

    If you check my core credits, you can see I have a long history with hunting down and killing these race conditions that are almost always around mkdir()...

  • Status changed to Needs review over 1 year ago
  • Status changed to RTBC over 1 year ago
  • 🇺🇸United States smustgrave

    Thank you for the explanation.

    Don't see any issue with adding a directory check in the test trait personally. Lets see what the committers say.

  • Status changed to Needs work over 1 year ago
  • 🇳🇿New Zealand quietone New Zealand

    Setting to NW for unresolved thread.

  • Status changed to Needs review over 1 year ago
  • Status changed to RTBC over 1 year ago
  • 🇺🇸United States smustgrave

    Threads appeared to be addressed. The open one about the comment is hard to tell but I think it has.

  • Status changed to Needs work over 1 year ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Re #11 - tests on DrupalCI are run in greater parallelisation than anywhere I've set up testing. I'm not convinced by the fix here. \Drupal\Core\File\FileSystem::prepareDirectory() already does an exists check if the mkdir fails. Also doing this so far from \Drupal\Core\Test\FunctionalTestSetupTrait::prepareEnvironment() (where the directory is supposed to be created) doesn't feel right. Furthermore each test has it's own folder so I'm not even sure how the race occurs - i.e each test gets a unique directory inside sites/simpletest.

    I think what might be good to see is the return value from \Drupal::service('file_system')->prepareDirectory($this->siteDirectory, FileSystemInterface::CREATE_DIRECTORY | FileSystemInterface::MODIFY_PERMISSIONS); checked and a more helpful error thrown if it is FALSE.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Also I don't think this is a major bug - if we had lots of reports of this occurring then I would agree but so far we have a single report.

Production build 0.69.0 2024