Unable to Migrate files from s3fs on a D7 site to a D10 site

Created on 24 January 2024, over 1 year ago
Updated 14 February 2024, about 1 year ago

Problem/Motivation

I am trying to run a migration of my D7 site to a D10.2.2 site and am receiving an error stating that

Steps to reproduce

I am using the Migration UI tool to migrate my data and files.
I am enter the information on the Drupal upgrade page, click on the Review upgrade link, and receive an s3 access error.
I have tried using both the url and uri, but still receive the warnings, no matter what I enter.
--------------------------------
URL format:
https://.s3.amazonaws.com/s3fs-public/
https://.s3.amazonaws.com/s3fs-private/
URL error:
Failed to read from Document root for public files. The server reports the following message: Client error: `HEAD https://.s3.amazonaws.com/s3fs-public/` resulted in a `403 Forbidden` response.
Failed to read from Document root for private files. The server reports the following message: Client error: `HEAD https://.s3.amazonaws.com/s3fs-private/` resulted in a `403 Forbidden` response.
--------------------------------
URI format:
s3:///s3fs-public/
s3:///s3fs-private/
URI error:
Failed to read from Document root for public files.
Failed to read from Document root for private files.
--------------------------------
The instance has full access to S3 and I can open individual files via a browser

Please let me know what format is required to migrate the s3 files from the old site to the new site.

Thanks,

πŸ’¬ Support request
Status

Fixed

Version

3.0

Component
MigrationΒ  β†’

Last updated about 14 hours ago

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Status changed to Postponed: needs info over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States cmlara

    @cilefen Thanks for moving the issue into the correct queue.

    @john.malpede@threewiresys.com

    This sounds like you may have attempted to migrate with the d7_file(public://) and/or d7_file_private(private://) migrations instead of the d7_s3fs_public_migrate, and d7_s3fs_private_migrate migrations.

    If you have not already I would suggest reviewing the section on Upgrading from Drupal 7 in the README.txt ?

    Can you confirm you are using the s3fs provided migrations or are you attempting to migrate away from 3FS?

  • Our Drupal 7 site has files stored using s3fs public and private. (Version: 7.x-2.14)
    Out Drupal 10 site is configured to use s3fs public and private. (Version: 8.x-3.4)
    As test for the Drupal 10 site, I uploaded files to s3 and verified the files in the bucket and the verified that the site was working correctly.
    The issue I am running into, is we are using a new s3 bucket to store or files.
    From README.TXT:
    These scripts copy the managed file entries from D7 into D8 without
    copying the actual files because they are already stored in the s3
    bucket.

    So, it looks like for migrating to new buckets, I need to copy the files from the old bucket, then perform a config validation and a metadata cache refresh to import the current list of files stored in the S3 bucket.

    After preforming those steps, I can now see the files that were missing.

    Thanks for pointing me in the right direction.

  • Status changed to Fixed about 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States cmlara
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024