Field Collection Node content translation source plugin

Created on 22 July 2020, over 4 years ago
Updated 19 October 2023, about 1 year ago

A lot the work so far for Paragaphs to migrate from D7 field collections revolves around entity translation. But for sites using the old content translation method we still need a means to move those field collections to paragraphs.

We created a pair of source plugins based off of the πŸ“Œ Field collections deriver and base migration Fixed to bring in specifically field collections attached to content translation nodes. The first plugin is to create the source paragraph entity with the same langcode as the tnid node had. The second is to bring in the other Field collections as translations to that source paragraph using more SQL joins than I'd care to admit.

But the result is a set of translated paragraphs that work.

The source paragraph plugin is a replacement for the d7_field_collection_item plugin. Just swap the name of the plugin to `d7_field_collection_item_content_translation_source` in the generated migration and add the langcode mapping

Example:

source:
  constants:
    parent_field_name_value: field_country_idrc_support
  field_name: field_country_idrc_support
  plugin: d7_field_collection_item_content_translation_source
process:
process:
  id:
    -
      plugin: get
      source: item_id
  parent_id:
    -
      plugin: get
      source: parent_id
  langcode:
    -
      plugin: default_value
      source: language
      default_value: und
  parent_type:
    -
      plugin: get
      source: parent_type

The latter translations plugin works nearly the same. Copy the source migration yaml into a new migration and change the plugin to d7_field_collection_content_translation_translations, set the source and destination translation flags and make sure the langcode field is present. And don't forget the content_translation_source definition. That was annoying to debug.

Example:

source:
  constants:
    parent_field_name_value: field_country_idrc_support
  plugin: d7_field_collection_item_content_translation_translations
  field_name: field_country_idrc_support
  translations: true
process:
  id:
    -
      plugin: get
      source: source_item_id
  parent_id:
    -
      plugin: get
      source: parent_id
  langcode:
    -
      plugin: get
      source: language
  field_country_support_title:
    -
      plugin: get
      source: field_country_support_title
  content_translation_source:
    -
      plugin: get
      source: source_langcode
destination:
  plugin: 'entity_reference_revisions:paragraph'
  default_bundle: total_idrc_support
  translations: true

Looking for feedback or input on making this compatible with the built in migration code. I'm not too familiar with the deriver classes so not sure how to make this automatically generate.

✨ Feature request
Status

Needs review

Version

1.0

Component

Code

Created by

πŸ‡¨πŸ‡¦Canada minoroffense Ottawa, Canada

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    180 pass
  • πŸ‡ΊπŸ‡¦Ukraine vlad.dancer Kyiv

    Added small fix for undefined $field variable.

  • πŸ‡ΊπŸ‡¦Ukraine vlad.dancer Kyiv

    Thank you @minorOffense for saving my time and giving base point for adjustemnts.
    Out of the box the @minorOffense's code doen't work (at least in 2023 and for migrating from d7 -> d8).
    So below is how we managed it.

    But before a quick tip to @Ludo.R.

    id:
        -
          plugin: get
          source: source_item_id
    

    This will set id of the paragraph's source translation.
    And later node translation migration (that includes this paragraph)
    uses plugin: paragraphs_lookup that in case non-existing id will create stub for paragraph translation.
    So just change the above code to:

      id:
        - plugin: paragraphs_lookup
          tags: 'Field Collection Content'
          source: source_item_id
          no_stub: true
        - plugin: extract
          index:
            - id
    

    Ok.
    First we need to patches:
    - paragraph #3160679_6 ✨ Field Collection Node content translation source plugin Needs review
    - entity_reference_revisions #2903495_6 β†’

    Then adjust paragragh source migration (example is for d7 data):

    source:
      # This is a new thing
      plugin: d7_field_collection_item_content_translation_source
      field_name: field_contact_fc
    
    process:
      type:
        -
          plugin: migration_lookup
          migration: upgrade_d7_field_collection_type
          no_stub: true
          source: field_name
      parent_id:
        -
          plugin: get
          source: parent_id
      parent_type:
        -
          plugin: get
          source: parent_type
      parent_field_name:
        -
          plugin: get
          source: field_name
      
      # This is a new thing    
      langcode:
        -
          plugin: default_value
          source: language
          default_value: und
    ...
    destination:
      plugin: 'entity_reference_revisions:paragraph'
    
    

    Then create a copy of this migration and modify it to make it for paragraph translations only:

    ...
    source:
      # This is a new thing
      plugin: d7_field_collection_item_content_translation_translations
      field_name: field_contact_fc
      # This is a new thing
      translations: true
    
    process:
      id:
        # This is a new thing, use correct destination id
        - plugin: paragraphs_lookup
          tags: 'Field Collection Content'
          source: source_item_id
          # Let's prevent stubs (for out case)
          no_stub: true
        - plugin: extract
          index:
            - id
      type:
        -
          plugin: migration_lookup
          migration: upgrade_d7_field_collection_type
          no_stub: true
          source: field_name
      parent_id:
        -
          plugin: get
          source: parent_id
      parent_type:
        -
          plugin: get
          source: parent_type
      parent_field_name:
        -
          plugin: get
          source: field_name
      # This is a new thing
      langcode:
        -
          plugin: get
          source: language
    
      # This is a new thing
      content_translation_source:
        - plugin: get
          source: source_langcode
    
      ....
    destination:
      plugin: 'entity_reference_revisions:paragraph'
      # This is a new thing
      translations: true
    ...
    
    

    Just as example of complete process I'll include node import config here too (for d7):

    ...
    source:
      plugin: d7_node
      node_type: your_type
    process:
      nid:
        -
          plugin: get
          source: tnid
      langcode:
        -
          plugin: default_value
          source: language
          default_value: und
      field_contact_fc:
        -
          plugin: sub_process
          source: field_contact_fc
          process:
            target_id:
              -
                plugin: paragraphs_lookup
                tags: 'Field Collection Content'
                source: value
                no_stub: true
              -
                plugin: extract
                index:
                  - id
            target_revision_id:
              -
                plugin: paragraphs_lookup
                no_stub: true
                tags:
                  - 'Field Collection Revisions Content'
                  - 'Field Collection Content'
                tag_ids:
                  'Field Collection Revisions Content':
                    - revision_id
                  'Field Collection Content':
                    - value
              -
                plugin: extract
                index:
                  - revision_id
    ...
    destination:
      plugin: 'entity:node'
      translations: true
      default_bundle: your_type
    ...
    

    Mostly, I followed steps described by @minorOffense with minor adjustments.
    As result we are able to import nodes with translated paragraphs.

    We skipped automated solution for creating migrate translation configuration.
    But if someone interested in it then I think this is a good example for start docroot/core/modules/migrate_drupal/src/Plugin/migrate/EntityReferenceTranslationDeriver.php

    BTW: don't forget to check if migration tables were created after importing your new configuration for paragrapg translation.
    If there are no neede tables use this script

    It is scary how data migration became so complex.
    What we will do in 2026 with migrating layout_builder and inline blocks?..

    I hope it save time for someone too.

  • πŸ‡©πŸ‡ͺGermany vistree

    Hi @vlad.dancer,
    thanx for your description!!! One question: in

    process:
      type:
        -
          plugin: migration_lookup
          migration: upgrade_d7_field_collection_type
          no_stub: true
          source: field_name

    where does the source: field_name come from? What to use? Is it the same fieldname defined in the source section? Or should it always be static "field_name"?

  • πŸ‡©πŸ‡ͺGermany vistree

    And a second question: do we need to also touch the revisions migration files?
    E.g. I have
    - migrate_plus.migration.upgrade_d7_field_collection_education.yml
    - migrate_plus.migration.upgrade_d7_field_collection_revisions_education.yml

    + for my migrations, the new paragraph field will have translation turned on (true). Do we need to change this or will this automatically be correctly set by the new plugins?

  • πŸ‡ΊπŸ‡¦Ukraine vlad.dancer Kyiv

    Hi @vistree.

    > where does the source: field_name come from?

    The field_name will be picked up from:

    source:
      field_name: field_contact_fc
    

    and set as a row property in \Drupal\paragraphs\Plugin\migrate\source\d7\FieldCollectionItem::query().
    So field_name in type mapping is dynamic variable set by mentioned method which was configured by source field_name configuration property.

    > do we need to also touch the revisions migration files?

    If you want to import translations, then answer is yes.
    Check my previous answer. Yo need to change the source plugin:

    source:
      # This is a new thing
      plugin: d7_field_collection_item_content_translation_source
  • πŸ‡©πŸ‡ͺGermany vistree

    Hi Vlad,
    thanx again ;-) My question was more about existing migration ymls after creating the initial D7 to D10 migration files.
    E.g. for my D7 field-collection field_collection_education I find
    1. migrate_plus.migration.upgrade_d7_field_collection_education.yml
    2. migrate_plus.migration.upgrade_d7_field_collection_revisions_education.yml

    From your information in #7 I now edited file from 1. -> migrate_plus.migration.upgrade_d7_field_collection_education.yml

    I did all the modifications and AFTERWARDS I did a copy of that file - renamed it to
    3. migrate_plus.migration.upgrade_d7_field_collection_education_trans.yml

    and did the modifications described in #7

    I now have 3 files!!!! And run the migration for all 3 of them.
    The result seems not to be OK ;-(

    Am I doing something wrong? Do I need to exclude file 2. (migrate_plus.migration.upgrade_d7_field_collection_revisions_education.yml) ?

  • πŸ‡©πŸ‡ͺGermany vistree

    I am still struggling with the migration of the multilingual field collections. Does anyone here have any advice for me?

Production build 0.71.5 2024