Add a continue on failure option

Oh, sorry I missed this, I'll check it out.

if running multiple migrations, when one migration reports any failure, the entire process stops with that migration and subsequent migrations are not run. That's not necessarily desirable, especially when developing a set of migrations where you'd like it to run the whole thing and gather all the errors from all the migrations rather than have to identify and fix them one at a time. So, I propose a --continue-on-failure option to the migrate-import command, which will run all selected migrations whether or not one in the middle fails.

https://www.drupal.org/project/migrate_tools/issues/3092369 →

I am a bit confused by this feature request, The example shown seems to apply for groups. i.e. if one item in the group fails, then the next one is not skipped.

How does this apply to migrate queue importer that does not support groups AFAICT?

This is all about migration_dependencies.

https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Plugi...

In my experience there are some issues when running dependencies using migrate tools UI or via drush/cron. From what I can tell the options are ignored for dependencies. e.g. if I specify to limit say 1 row, this appears to be ignored when running the dependencies as part of the main migration.

I think it might be better to run a migration dependencies separately i.e. select the option to ignore dependencies and run each one separately. This should make sure a dependency runs even if there is an error with a row in the parent migration.

You need the entire dependent migration to run if it exists. The row limit should only apply to the main migration you're controlling. The dependent migrations shouldn't be limited because their successful completion is crucial for ensuring the main migration can run properly.
Limiting dependencies could lead to incomplete data or other issues downstream.

Often you just want to update a single row for a migration and its dependencies. When specifying a limit, this ignored by dependencies. In fact, I cannot think of a single case where I want to limit the main migration but not one of its dependencies.

e.g. if I use one or more migration_lookup in my main migration, i should be able to specify an id and it will just import that one. However, even if i limit it by one, the dependencies will run in their entirety. This can be very tine consuming, and in my case unnecessary.

Actually --continue-on-fallurre does seem to do something. I am getting the following output if I do not specify it.

[notice] Processed 4 items (0 created, 0 updated, 4 failed, 0 ignored) - done with 'my_import_feed'

In MigrateToolsCommands.php line 1092:

The code for this is like so:

if ($error_message) {
      if ($options['continue-on-failure']) {
        $this->logger()->error($error_message);
      }
      else {
        // Nudge Drush to use a non-zero exit code.
        throw new \Exception($error_message);
      }
    }

If I understand this correctly this is useful to prevent drush outputting a non zero value

By convention, an exit code of zero indicates that the command completed successfully, and non-zero means that an error was encountered.

https://linuxize.com/post/bash-exit/