MigMagLookup can unintentally strip out messages from non-SQL sources

Created on 11 August 2022, over 2 years ago
Updated 17 July 2023, over 1 year ago

Problem/Motivation

MigMagMigrateStub::createStub can unintentionally delete existing idMap messages when doing a loop on non-SQL sources.

Essentially whenever \Drupal\migrate\Plugin\migrate\source\SourcePluginBase::next() is called.

And since MigMagMigrateStub::createStub loops through the source iterator, any previous messages which were previously stored will be deleted when looping through a source iterator and next() is called.

Steps to reproduce

Given:

  1. migration_a is a non-SQL source migration which creates a message for each row
  2. migration_b is a migration which does a migmag_lookup against migration_a
  3. migration_a has completely ran before migration_b, and has a populated idmap table

When migration_b is ran, all the original messages created in the initial migration_a run will be deleted as it loops through.

Proposed resolution

  1. Override the source plugin definition so that it uses a NullIdMap so it doesn't interfere with the original migration idMap and highWaterStorage during iterations.

MigrationDeriverTrait::getSourcePlugin() seems like a good base for what we want.

Remaining tasks

Provide issue fork/patch with tests.

API changes

No changes.

πŸ› Bug report
Status

Active

Version

1.8

Component

MigMagLookup Process Plugin

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024