Allow a remote file to be specified for the Source path parameter

Created on 22 August 2024, 5 months ago

Problem/Motivation

The current code only allows for local CSV files to be specified for the path parameter. This is due to the use of file_exists() function being used to check if the file exists.

Use case for this is uploading an updated file on a remote server, such as S3, on day of launch of a feature prior to running a migration. This will also allow for the CSV to be not committed into code and that all various servers have access to the CSV location that can be updated at any time without a code release.

Steps to reproduce

Update the migration file path with a remote file location that is not a local to the server running the migration. An exception is thrown of

source:
plugin: csv
path: https://domain.com/remotefile.csv

Proposed resolution

Update initializeIterator() to recognize a remote file is being used and download locally to the servers temp location so the rest of the CSV.php functionality functions as-is.

The file is downloaded to the servers temp directory as allows for minimal code changes to the overall logic. Due to use of League/CSV an exception is thrown when the file is located on S3 due to the Stream of the file ().

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

✨ Feature request
Status

Active

Version

3.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States webbywe

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024