Contrib.social

Merged file references aren't being migrated

Open on Drupal.org →
Created on 18 August 2023, over 1 year ago
Updated 14 September 2023, about 1 year ago

Problem/Motivation

Some file references aren't being migrated. Investigating a little bit I found that the missing files were merged with another file from other migrations.

Steps to reproduce

First, you need two different migrations that migrate the same file.
Run both migrations and then the node migration.

Looking at the code I found two things.

First, in the class MediaMigrateSubscriber the update command searches the row using the target_id column. That means that only the merged file media_id will be registered. 

    $this->connection->update('migrate_file_to_media_mapping')
      ->condition('target_fid', $fid)
      ->fields([
        'media_id' => $mid,
      ])
      ->execute();

Then, in the FileIdLookup class there is a code that queries the table using the fid column, and because it wasn't updated by the above code the row that it gets doesn't have the value media_id set.

Finally, as a fallback, it calls migration lookup using the target_fid column (merged file id), and since the file was migrated from another migration it also doesn't return any value.

Proposed resolution

There are two options here. Changing the update sentences to use the fid column so all records receive their respective media_id or using the fid instead of target_fid in the migration lookup.

🐛 Bug report
Status

Closed: cannot reproduce

Version

2.0

Component

Code

Created by

fabianfiorotto

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @fabianfiorotto
  • Assigned to HitchShock
  • Comment about 1 year ago →
    🇺🇦Ukraine HitchShock Ukraine
  • Status changed to Closed: cannot reproduce about 1 year ago
  • Comment about 1 year ago →
    🇺🇦Ukraine HitchShock Ukraine

    Hi @fabianfiorotto
    I wasn't able to reproduce the issue.
    Please make sure that you are following the documentation on how to use the module.
    1. Generate migration configs, they must match the original templates:
    - media-migration-step1.yml.twig
    - media-migration-step2.yml.twig
    2. Prepare duplicate file detection to fill in `migrate_file_to_media_mapping` table.

    The MediaMigrateSubscriber simply maps the new media file to the migrated file and all duplicates of this file. This means, that if a file is a duplicate of the migrated file, it will also be set to this value, and this is correct.
    Here is an example of how it must look for duplicates after step 1

    Afterward, in step 2 the file_id_lookup plugin will check the mapping first by fid column.

    Seems like you missed some steps from the documentation or modified some config/plugin.

    At the moment I'm making a conclusion that the module works as designed. you can reopen the ticket if you have more clear steps.

  • Issue was unassigned.
  • Comment about 1 year ago →
    🇺🇦Ukraine HitchShock Ukraine
contrib.social Blog FAQ Discussions
Production build 0.71.5 2024