UI will not display migrations that are not in configuration.

I'm facing a similar problem as well. My migrations are all defined in a module like "foo/migrations", however, I want the site's administrators to be able to see and run them in the UI. Thinking aloud, maybe this could be a setting? Show non-config migrations in the UI?

@jigarius when you are done with developing your migration, assuming it will not change, you can move it to `config/install` and enabled the module that provides it. This way it will be visible in the UI.

It will become configuration, so any updates on the file itself will not be reflected, but you can re-install the module or write an update hook to force it (I think that's possible).

I also want my migrations, located in the current correct place (which is 'modulefolder/migrations') → , to be listed in the UI.

Moving it to config/install is not best practice any longer. Re-installing modules or writing update hooks is messy, laborious and unnecessary.

If there really is some kind of issue about about having too many migrations shown, then just have tabs or another UI mechanism to filter them.

Adding my use case for this ..

We are in active development of a new Drupal application. Migration source is an Oracle database (non Drupal). We do not have any migrations created by Drupal itself. All of our migrations are being created like foo_migrate/migrations/*.yml. These migration configs are very rapidly changing and will go through many rounds of refinement before they can be considered ready for config/install or config/sync directories.

Our issue is that all our testing and staging environments are in Kubernetes clusters and ssh access is not readily available. So no easy drush execution.

It would be great to have a UI that can run theses migrations. I fully understand it is not a simple problem to solve. The current lists are based on displaying migrate_plus.migration configs while migrations that exist inside of a migrations/ folder are actually plugins. So maybe a separate UI that only lists plugins might be needed.

Indeed, bserem. That's how I had them, but then they need to change so config/install is not possible in my case. I have considered some other solutions which I'll share here.

After every time I do a drush config:import I can do a drush config:import --partial --source=modules/custom/foo_migrate/config/install to update my migration configuration.

I have also considered setting up symlinks to all my migration YML files inside Drupal's config sync directory. At the point of writing this comment, I think the config_ignore module can only ignore imports and not exports. Thus, a config:export would overwrite my symlinks. However, if it were to support "ignore during export only" some day then my symlinks approach might work.

IMHO, all migrations are migrations irrespective of how they're being created. So it'd be great to have a similar UI to manage them all. But maybe some people might want to hide certain "internal" migrations from the UI? In which case, a parameter like "hidden: true" in the migration YML might help.

That said, for now, I'm just going to live without these migrations in the UI. Sad, but true.

You can use hook_install, like this for example:

/**
 * Update config from config/install.
 */
function hook_update_8201() {
  $message = NULL;

  if (\Drupal::moduleHandler()->moduleExists('views')) {
    $config_path = \Drupal::service('extension.list.module')->getPath('EXAMPLE') . '/config/install/views.view.EXAMPLE.yml';
    $data = Yaml::parseFile($config_path);
    \Drupal::configFactory()->getEditable('views.view.EXAMPLE')->setData($data)->save(TRUE);
    $message = 'The EXAMPLE views have been updated.';
  }
  else {
    $message = 'Oh no! Something went wrong! Is views installed?';
  }
  return $message;
}

This is from a module of mine (admin_feedback) and worked fine. You'll have to update the paths to reflect migrations instead of views.

I closed 🐛 UI needs to handle all migration plugins Closed: duplicate as a dupe of this, which was already a dupe of #2873283: Add views_data and use views to list migrations and migration groups → . Adding the second as related.

I'm also going to up the Priority to "Major" since that's what it was on the issue @mikeryan put in.

Regarding some of the pain points mentioned above:

I'm not sure we want to web UI littered with hundreds of drupal provided migrations.

This could easily be solved by providing a configuration where the user can hide certain migration_tags, such as "Drupal 7" or "Drupal 6".

Though one could probably not get with the current approach of the Web UI and using EntityListBuilder for displaying the list.

You can in fact still use this setup, you'd just have to override ConfigEntityListBuilder::load, and create Migration entities from the plugin manager's definitions.

<?php
    $pluginDefinitions = $this->pluginManager->getDefinitions();
    foreach ($plugin_definitions as $plugin_id => $plugin_definition) {
      $entities[$plugin_id] = new Migration($pluginDefinition, 'migration');
    }
?>

In my case, I went a step further and replaced the handlers on the Migration entity type for storage and list_builder. I overwrite doLoadMultiple to create Migration entities from plugin definitions (caution: avoid the recursion on Drupal\migrate_plus\Plugin\MigrationConfigDeriver, which creates plugin definitions from Migration entities), and then change the getEntityIds method on my list_builder to return all plugin_ids from the plugin manager. The rest is magic - BUT this is a very hacky approach!

Stumbled over this too - for now I resorted to these drush commands to register migrations - might be handy for someone finding this issue too:
cat docroot/modules/custom/my_module/migrations/my_migration.yml | drush config:set --input-format=yaml migrate_plus.migration.my_migration ? -

And here the a way to import all migrations in a folder:
find docroot/modules/custom/my_module/migrations/ -type f -name "*.yml" -print0 | xargs -0 -I {} sh -c 'echo "Importing {}"; cat {} | drush config:set --input-format=yaml migrate_plus.migration.$(basename "{}" .yml;) ? -'