Contrib.social

Allow overriding id map migrate plugin

Open on Drupal.org →
Created on 14 February 2018, about 7 years ago
Updated 21 February 2023, about 2 years ago

Problem/Motivation

Many migrate plugins are directly exposed on the yml files, so they can be overridden.
Id map migrate plugins are not exposed from migrate plus.

Here is one use case. I have a D6 or D7 site that creates Article nodes from a feed, using the Feeds module. When I migrate the site to D8, I want to create Article nodes from two sources:

  1. Migrate Articles from the existing site.
  2. Continue to import Articles from the feed.

In order to make sure that I do not import the same article twice (once from each source), I would like to use the same mapping table for both sources. The mapping table is defined by the Id map plugin, so I would like to be able to specify that in my migration.

Proposed resolution

Expose the Id map in the migration config, i.e. change the schema.

Once the idMap key is added to a migration, it is used in the core Migration object: see core/modules/migrate/src/Plugin/Migration.php:

  public function __construct(/* ... */) {
    parent::__construct($configuration, $plugin_id, $plugin_definition);
    // ...

    foreach (NestedArray::mergeDeep($plugin_definition, $configuration) as $key => $value) {
      $this->$key = $value;
    }

  // ...

  public function getIdMap() {
    if (!isset($this->idMapPlugin)) {
      $configuration = $this->idMap;
      $plugin = isset($configuration['plugin']) ? $configuration['plugin'] : 'sql';
      $this->idMapPlugin = $this->idMapPluginManager->createInstance($plugin, $configuration, $this);
    }
    return $this->idMapPlugin;
  }

Completed tasks

  • Write a patch

Remaining tasks

  • Update the documentation.

User interface changes

N.A.

API changes

The idMap key is allowed on the root level of a migrate_plus.migration.* config object.

Data model changes

N.A.

✨ Feature request
Status

Needs review

Version

5.0

Component

API

Created by

🇵🇪Peru marvil07

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment about 2 years ago →
    🇫🇷France klelostec

    Rerolling patch to work with 6.0.1 release.

  • Status changed to RTBC 2 months ago
  • Comment 2 months ago →
    🇪🇸Spain nachosalvador Elx

    I have faced this issue when I need to use a custom idMap plugin and the last patch #35 works successfully to me.

  • Comment 2 months ago →
    🇺🇸United States benjifisher Boston area

    The patch from #35 does not apply to the 6.0.x branch.

    1. My version of git did not like the DOS-style line endings.
    2. My version of git did not like that there were some empty lines.
    3. There is a conflict from 🐛 Fix failing tests on HEAD Fixed .

    I fixed (1) and (2) by editing the patch file. (I added a single space to the empty lines.)

    I fixed (3) by checking out the commit before that issue, applying the patch, committing, and then rebasing on the 6.0.x branch.

    It looks as though I did some of the early work on this issue, but I have not looked at the code changes in years. I am setting the status back to NR. Someone who has worked with this issue more recently should review and re-test after my rebase.

    I created a MR for this issue, and I am attaching a patch for convenience.

  • Merge request !122Issue #2944627, Comment #35 → (Open) created by benjifisher
  • Pipeline finished with Success
    2 months ago
    Total: 291s
    #423454
  • Status changed to Needs review 2 days ago
  • Comment 2 days ago →
    🇺🇸United States ramzypro

    I'm successfully using the merge request in #38 on a sync from an external MySQL data source with no issues.

  • Comment 2 days ago →
    🇺🇸United States ramzypro

    Apologies; I had to read up more on Drupal etiquette about changing status - I've used the patch from #38 extensively without issue.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024