- Issue created by @megachriz
- 🇺🇸United States sk33lz
Re-rolling patch as it was not applying due to changes in the HTTP Fetcher file.
In 📌 Missing temporary files in load balanced environments Fixed a service was added to handle Feeds files that are in progress of being imported. Http downloaded sources files are saved in either private://feeds/in_progress/x or public://feeds/in_progress/x.
In the service the function \Drupal\Core\File\FileSystemInterface::realpath() is used to figure out the directory to save he downloaded file to. This works fine on local file systems, but it causes issues on remote file systems like s3fs.
TBD.
From #2912130-136: Missing temporary files in load balanced environments → :
In
FeedsFileSystemBase::tempnam(), instead of writing a temporary file with tempnam, I wrote a file w/\Drupal::service('file.repository')->writeData(). This method should support local and remote stream wrappers. Then I got additional error information that the getAsync call inHttpFetcher::get()appears to open a file with "w+" permissions by default which s3 (and perhaps other remote stream wrappers) do not seem to allow: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/s3-stream-wra...You can upload larger files by streaming data using fopen() and a “w”, “x”, or “a” stream access mode. The Amazon S3 stream wrapper does not support simultaneous read and write streams (e.g. “r+”, “w+”, etc). This is because the HTTP protocol doesn’t allow simultaneous reading and writing.
First opening the stream through
GuzzleHttp\Psr7\Utils::tryFopen()with "w" mode and passing that filestream as the sink to getAsync worked perfectly. Not sure if other use cases require "w+"
TBD.
TBD.
TBD.
Active
3.0
Code
Re-rolling patch as it was not applying due to changes in the HTTP Fetcher file.