This has bugged me for years and I've finally discovered the issue.
In the FileSystem service the method prepareDirectory is called to ensure the directory and permissions are all set. Part of that method calls the File System mkdir method which splits up the full private path (i.e. /var/www/vhosts into ['var', 'www', 'vhosts']) and loops through from the top level all the way down through every required directory to ensure they exist and, if not, create them with permissions. Inside this method runs the nativefile_exists method which will ALWAYS returns false if the user running the method doesn't have permissions in the first place. The documentation states (taken from PhpStorm as php.net appears to have been simplified now):
This function returns false for files inaccessible due to safe mode restrictions. However these files still can be included if they are located in safe_mode_include_dir.
The check is done using the real UID/GID instead of the effective one.
In the instance of the file structure on most of my servers, the top level directory is /var which is owned by root and it is failing here (and most likely the next 2 directories after that which are also owned by root). To test the theory out I added /var to the open_basedir for my Drupal site and everything started working! Of course, I immediately removed this as that is a huge security hole granting this one site access to everything inside the /var directory.
This would be why relative directories can have better success even though Drupal state it should be an absolute path.
There must be a better work around that Drupal can do here to get around this security issue. Perhaps it doesn't start the checks from the top level but instead define a new setting for where the "root" of the site is which is typically one level lower than the installed directory.