There's a possible edge case with long-running migrations when multiple Drush cron runners could be running at the same time (i.e. if migration takes more time than a default cron execution lock timeout).
In this case, the second drush process may start the same migration (and reset its status, mark all items for update etc). This could lead to application slowdown, overwriting the same data multiple times, possible conflicts with entities created with migration lookup etc.
1. Create a large migration with lots of data that takes e.g. 1 hour to complete.
2. Enable running this migration with Cron.
3. Run `drush cron`.
4. Wait for 15 minutes (this is a `cron` lock expiration time, ref `\Drupal\Console\Command\Cron\ExecuteCommand::execute`); otherwise, Drupal won't let you run another cron process..
5. Run `drush cron` again.
Expected: The second process does not start the migration since it's already running. A second migration may be executed though.
Actual: The second process resets the migration status and starts it over again.
The cron job should acquire a lock while starting the migration, then subscribe to Migrate row process event and renew the lock periodically (to make sure it's not expired by timeout).
Release the lock after migration ends.
The cron job should not start migrations if the lock could not be acquired.
Do it.
None.
None.
Nope.
Active
1.0
Code
Not all content is available!
It's likely this issue predates Contrib.social: some issue and comment data are missing.
Our solution to this was to use ultimate_cron and put a long lock on the migrate_cron job.
Another solution would be to use a tool like flock (https://manned.org/flock) to prevent multiple cron commands running at the same time.