Add support for the Migrate API

Created on 23 February 2025, about 2 months ago

Problem/Motivation

The Migrate API already provides a framework for updating content and config entities (or creating new ones) on a Drupal site. It provides a lot of features that overlap with this module:

  • Track status of processed/unprocessed items and errors.
  • Log messages.
  • Update individual fields, with flexible transformation options.

As a first step to integration, it would be great if this module could provide a way for running migrations based defined in a module.

Proposed resolution

Provide a UI for running a selected migration, just as you can already run a selected script.

Remaining tasks

User interface changes

API changes

Data model changes

✨ Feature request
Status

Active

Version

1.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @benjifisher
  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    The most useful migration source plugins for same-site migrations are content_entity and d8_config. Currently, both are defined in the migrate_drupal module.

    πŸ“Œ Move content_entity source plugin to migrate module Active moved the content_entity plugin to the migrate module. Starting with Drupal 11.2, that plugin should be available without enabling migrate_drupal. We plan to do the same with the d8_config plugin, and probably rename it, in πŸ“Œ Move the d8_config source plugin to the migrate module Active .

  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida

    This idea is very interesting for me. Leveraging the flexibility of cron this could be very useful to run ongoing migrations. I can immediately see two useful variations of this:

    1. processOne() calls a full migration directly - so it becomes a way to fire a migration, however if run through the UI would not actually behave as a batch and would be at risk for timeouts.
    2. processOne() calls a migration but passes it the item as an id so it just runs a migration for that one item. This would be less prone to timeouts if run in the UI but would be much less performant as the it still does all the initial bootstrapping of the migration which for every item would look for all the items covered by the migration. This could be useful though to process items that were done as kind of an incremental migration (example: migrating any nodes that have a workflow state of "ready for migration"
  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    The drush migrate:import command accepts a --limit option.

    The implementation, unfortunately, is rather complicated. I think the key parts are in https://github.com/drush-ops/drush/blob/13.x/src/Drupal/Migrate/MigrateE..., which extends Drupal\migrate\MigrateExecutable.

    I think an MVP that does not include the limit option would still be useful. Give some sort of warning that large migrations can cause problems when run from the UI.

Production build 0.71.5 2024