Migration throws warning when constants key is not defined in file migration source

Created on 9 March 2023, over 1 year ago
Updated 20 June 2023, over 1 year ago

Problem/Motivation

[warning] Trying to access array offset on value of type null File.php:105
[warning] Undefined array key "constants" File.php:105

Steps to reproduce

Create a files migration without constants defined in the source.
There are cases where you do not need source_base_path in the migration.
Like when you are migrating files from Drupal 7 to Drupal 9, You can use stage file proxy to refer files to the production server or you can have files already copied into your new sites where there is no need for re downloading of files.

uuid: fe9b8ff3-8713-4ded-9ccd-9644d540156b
langcode: en
status: true
dependencies: {  }
id: upgrade_d7_file
class: Drupal\migrate\Plugin\Migration
field_plugin_method: null
cck_plugin_method: null
migration_tags:
  - 'Drupal 7'
  - Content
migration_group: migrate_drupal_7
label: 'Public files'
source:
  plugin: d7_file
  scheme: public
  constants:
    source_base_path: 'https://example.com/'
process:
  fid:
    -
      plugin: get
      source: fid
  filename:
    -
      plugin: get
      source: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/source_base_path
        - filepath
    -
      plugin: urlencode
  uri:
    -
      plugin: file_copy
      source:
        - '@source_full_path'
        - uri
  filemime:
    -
      plugin: get
      source: filemime
  status:
    -
      plugin: get
      source: status
  created:
    -
      plugin: get
      source: timestamp
  changed:
    -
      plugin: get
      source: timestamp
  uid:
    -
      plugin: get
      source: uid
destination:
  plugin: 'entity:file'
migration_dependencies:
  required: {  }
  optional: {  }

Above migration doesnt throw warning. Where as below migration throws warning.

uuid: fe9b8ff3-8713-4ded-9ccd-9644d540156b
langcode: en
status: true
dependencies: {  }
id: upgrade_d7_file
class: Drupal\migrate\Plugin\Migration
field_plugin_method: null
cck_plugin_method: null
migration_tags:
  - 'Drupal 7'
  - Content
migration_group: migrate_drupal_7
label: 'Public files'
source:
  plugin: d7_file
  scheme: public
process:
  fid:
    -
      plugin: get
      source: fid
  filename:
    -
      plugin: get
      source: filename
  uri:
    -
      plugin: get
      source: uri
  filemime:
    -
      plugin: get
      source: filemime
  status:
    -
      plugin: get
      source: status
  created:
    -
      plugin: get
      source: timestamp
  changed:
    -
      plugin: get
      source: timestamp
  uid:
    -
      plugin: get
      source: uid
destination:
  plugin: 'entity:file'
migration_dependencies:
  required: {  }
  optional: {  }

Proposed resolution

Add check for the config defined or not in prepare row of file migration plugin.

  public function prepareRow(Row $row) {
    // Compute the filepath property, which is a physical representation of
    // the URI relative to the Drupal root.
    $path = str_replace(['public:/', 'private:/'], [$this->publicPath, $this->privatePath], $row->getSourceProperty('uri'));
    // At this point, $path could be an absolute path or a relative path,
    // depending on how the scheme's variable was set. So we need to shear out
    // the source_base_path in order to make them all relative.
    $path = preg_replace('#' . preg_quote($this->configuration['constants']['source_base_path']) . '#', '', $path, 1);
    $row->setSourceProperty('filepath', $path);
    return parent::prepareRow($row);
  }

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Closed: works as designed

Version

9.5

Component
Migration 

Last updated about 22 hours ago

Created by

🇮🇳India bhanu951

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @bhanu951
  • 🇳🇿New Zealand quietone

    Just removing tag, according to the issue tag guidelines .

  • First commit to issue fork.
  • @banoodle opened merge request.
  • Status changed to Closed: works as designed over 1 year ago
  • 🇺🇸United States mikelutz Michigan, USA

    the d7_file plugin requires a source base path for all core migrations that use it, and for most customized file migrations. defining source_base path is required for this plugin, and we do not want to add a quiet bypass when it isn't set because it's easy to forget, and we want to throw this error when you forget. If you do not need the source to supply the filepath to the original file, you can extend the plugin and override the prepare_row method to not supply it.

Production build 0.71.5 2024