[meta] Migrate <> Modeler API

Created on 11 August 2025, 23 days ago

Problem/Motivation

We should be able to use the Modeler API β†’ to create and manage migrations in the Drupal Admin UI.

Proposed resolution

A good first step (proof of concept) would be a contrib module (maybe a sandbox project) that uses modeler_api to edit migrate_plus.migration.* configuration. Maybe limit it to the csv source plugin and the entity destination plugin and one or two process plugins.

In fact, that might already provide value (not just proof of concept).

The problem is that the Migrate API does not provide enough information to tell the modeler how to build the config form for each (source, process, destination) plugin. That is a long-term project.

- benjifisher

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

✨ Feature request
Status

Active

Version

11.0 πŸ”₯

Component

migration system

Created by

πŸ‡ΊπŸ‡ΈUnited States thejimbirch Cape Cod, Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @thejimbirch
  • πŸ‡©πŸ‡ͺGermany jurgenhaas Gottmadingen

    This sounds great. And while the plugins don't come with configuration forms, we should be able to derive them from the config schema, where available.

  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    @thejimbirch: Thanks for opening this issue.

    @jurgenhaas: There are only a few source and destination plugins, but there are a lot of process plugins. These are PHP classes, generally extending Drupal\migrate\ProcessPluginBase. So I am not sure what config schema you mean.

    When I called it "a long-term project", I was thinking of the old issue πŸ“Œ Create a trait and base class to implement \Drupal\Component\Plugin\ConfigurableInterface Needs work , which was Fixed 2 or 3 months ago (2025-06-14). I think the next step is to expand Drupal\Component\Plugin\ConfigurableInterface, or add a related interface, with some sort of form-builder method. Then configurable plugins would have a consistent way to provide their configuration forms.

    What that new, public method would be, and how it is implemented, is up for discussion. Maybe it would involve a config schema, at least as one option. Or maybe it would involve a config entity type, and the config schema would declare that it handles that type.

    That is why I describe this issue as providing a proof of concept. For now, then suggested module is responsible for providing config forms for a limited number of plugins.

    The suggested module would not be a candidate for Drupal core. Since it relies on migrations defined by config entities, it has to depend on the migrate_plus module. I suggested using the csv source plugin for the proof of concept (and for making something that is actually useful). That comes from another contrib module, migrate_source_csv.

  • πŸ‡©πŸ‡ͺGermany jurgenhaas Gottmadingen

    There are only a few source and destination plugins, but there are a lot of process plugins. These are PHP classes, generally extending Drupal\migrate\ProcessPluginBase. So I am not sure what config schema you mean.

    I was thinking in the context of migrate_plus where migrations are stored in config entities. That should normally come with config schemas, which may not be the case yet, but that would have to be defined anyway. Then, once they (the schema definitions) are available, instead of also building configuration forms, it should be possible to derive config forms from the schema.

    That thinking could be applied to all 3 types of migrate plugins.

  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    My understanding is that config schemas are for config entity types. In theory, we could have a config schema for migrate_plus.migration.*, and in theory we could generate a config form from that schema. In practice, I do not see how we can get that to work unless we have some way for process plugins to specify their config forms.

    Background: the main part of any migration (core plugin or migrate_plus config entity) is the process section. This consists of several process pipelines. Many pipelines are single steps (like assign a source value to a destination value) but some pipelines are long chains of process plugins, each with its own configuration.

    Source, process, and destination plugins are not config entities, so config schemas do not apply directly. In Comment #3, I suggested an approach to getting config schemas to apply. But that is still in the ideation phase, a few steps before vaporware.

  • πŸ‡©πŸ‡ͺGermany jurgenhaas Gottmadingen

    I'm not sure there is really a difference between process plugins and the source/destination plugins. Aren't the latter also be part of the migrate_plu.migration.* config entities?

    For process plugins, either defining a single process plugin or a sequence of such plugins can be a challenge, you're right. However, on the granularity of a process plugin, the schema should be possible and helpful. Allowing a config entity to reference to single or a sequence of components, if not yet possible, should probably be discussed upstream.

    There might be other ways to get something defined to build any UI around migration, but each of them will at least print dozens and dozens of warnings on screen, if such config entities get saved through the UI. Those warnings are about missing config schemas, and we don't want that to happen, as it is really confusing from a user perspective. In other words, if we build a UI for config, the underlying config requires a config schema.

Production build 0.71.5 2024