Migrate D7 field collections to D9 paragraphs has Unprocessed items but will not complete

Created on 27 November 2023, about 1 year ago
Updated 28 November 2023, about 1 year ago

Problem/Motivation

When I run drush mim d7_field_collection:typename the job runs and ends. Then I try to run another migration which was dependent on the previous but it returns an error that it cannot run without the dependency which is the previous job. When I check the status with drush ms d7_field_collection:typename then I see the "Status: Idle" but "Unprocessed: 2". There were no messages in drush mmsg d7_field_collection:typename and nothing would happen if I tried to re-run the job.

Steps to reproduce

Note: This may be specific to my source field collection being nested in another field collection.

First, run the migration for a nested field collection. Then try to run the migration job for dependent field collection.

Proposed resolution

I found the query for source records was returning a different number of records for "Total" than items processed once the migration seemed to finish and the difference equaled the Unprocessed items. After some digging, I found the nested field collections could have the same entity id assigned to multiple revisions. This was causing the Total number to be higher than the number of entities migrated. The attached patch adds groupBy() to the query to prevent the extra rows for entities with different revision_id's.

πŸ› Bug report
Status

Needs review

Version

1.16

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States trrroy

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024