Private file system settings; absolute path not working

This has bugged me for years and I've finally discovered the issue.

In the FileSystem service the method prepareDirectory is called to ensure the directory and permissions are all set. Part of that method calls the File System mkdir method which splits up the full private path (i.e. /var/www/vhosts into ['var', 'www', 'vhosts']) and loops through from the top level all the way down through every required directory to ensure they exist and, if not, create them with permissions. Inside this method runs the nativefile_exists method which will ALWAYS returns false if the user running the method doesn't have permissions in the first place. The documentation states (taken from PhpStorm as php.net appears to have been simplified now):

This function returns false for files inaccessible due to safe mode restrictions. However these files still can be included if they are located in safe_mode_include_dir.
The check is done using the real UID/GID instead of the effective one.

In the instance of the file structure on most of my servers, the top level directory is /var which is owned by root and it is failing here (and most likely the next 2 directories after that which are also owned by root). To test the theory out I added /var to the open_basedir for my Drupal site and everything started working! Of course, I immediately removed this as that is a huge security hole granting this one site access to everything inside the /var directory.

This would be why relative directories can have better success even though Drupal state it should be an absolute path.

There must be a better work around that Drupal can do here to get around this security issue. Perhaps it doesn't start the checks from the top level but instead define a new setting for where the "root" of the site is which is typically one level lower than the installed directory.

Nice research! It looks like \Symfony\Component\Filesystem\Filesystem::mkdir also uses the native mkdir in the end so not sure its going to be any help.

Drupal 10

Thanks @nhwells for your comment #13, that's it.

Yes this is a Drupal bug in the way it checks if the directory exists.

I think the easy way to solve this if have the mkDir function of web/core/lib/Drupal/Core/File/FileSystem.php build the missing components of the parent directory in reverse order.

Instead of starting to check if /var exists in the below example start with checking if /var/www/webuser298/web/private_files/styles/thumbnails/large/private exists and work your way back until you get to a directory that does exist, and then you know what directories to create.

I havent needed a patch yet but that would be my approach.

thanks again!