Contrib.social

Node complete creates new nodes when used for incremental migrations

Open on Drupal.org →
Created on 17 August 2020, over 4 years ago
Updated 6 June 2023, over 1 year ago

When using the node_complete i get new nodes in stead of revisions

I exposed my d7 db and created the migrations using the command drush migrate-upgrade --configure-only, then i started to change the generated code to my wish so for example in the process part of my migrate_plus.migration.upgrade_d7_node_complete_page.yml i removed nid and vid part as i need nids and vids to be created from drupal 8.

When i run a query like:

SELECT COUNT(DISTINCT uuid ) FROM node WHERE type = 'page'; on my d7 database, i get 268 nodes. So i would expect 268 nodes to be created. Instead, i get 864 nodes being created. Looking at the mapping tables, it seems to me that Drupal is generating one node per revision:
SELECT sourceid1,sourceid2,sourceid3,destid1,destid2,destid3 FROM `migrate_map_upgrade_d7_node_complete_page` WHERE sourceid1=2511417;

"sourceid1","sourceid2","sourceid3","destid1","destid2","destid3"
"2511417","2511449","en","5547493","5786980","en"
"2511417","2543841","en","5547494","5786981","en"
"2511417","2543849","en","5547495","5786982","en"
"2511417","2543857","en","5547496","5786983","en"
"2511417","2552945","en","5547498","5786985","en"
"2511417","2847097","en","5547524","5787011","en"
"2511417","2847121","en","5547525","5787012","en"
"2511417","2847177","en","5547526","5787013","en"
"2511417","2847201","en","5547527","5787014","en"
"2511417","2847217","en","5547528","5787015","en"
"2511417","2847241","en","5547529","5787016","en"
"2511417","3060609","en","5547530","5787017","en"
"2511417","3060801","en","5547531","5787018","en"
"2511417","3061881","en","5547533","5787020","en"
"2511417","3061913","en","5547534","5787021","en"
"2511417","3072113","en","5547536","5787023","en"
"2511417","3072129","en","5547537","5787024","en"
"2511417","3079553","en","5547589","5787076","en"
"2511417","3079801","en","5547590","5787077","en"
"2511417","3079809","en","5547591","5787078","en"
"2511417","3088649","en","5547612","5787099","en"
"2511417","3088665","en","5547613","5787100","en"
"2511417","3088729","en","5547614","5787101","en"
"2511417","3092793","en","5547615","5787102","en"
"2511417","3097073","en","5547621","5787108","en"

my migration yml is: 

uuid: 0d6830e2-5fc0-4b28-93af-ca37f69c18d3
langcode: en
status: true
dependencies:
  enforced:
    module:
      - migrate_media_handler
id: upgrade_d7_node_complete_page
class: Drupal\node\Plugin\migrate\D7NodeTranslation
field_plugin_method: null
cck_plugin_method: null
migration_tags:
  - 'Drupal 7'
  - Content
migration_group: migrate_drupal_7
label: 'Node complete (Basic pages)'
source:
  plugin: d7_node_complete
  node_type: page
  ids:
    uuid:
      type: string
    nid:
      type: integer
process:
  nid:
     -
       plugin: migration_lookup
       migration: d7_node_complete
       source: tnid
     -
       plugin: node_complete_node_lookup
  langcode:
    -
      plugin: default_value
      source: language
      default_value: und
  title:
    -
      plugin: get
      source: title
  uid:
    -
      plugin: migration_lookup
      migration: upgrade_d7_user
      source: node_uid
      source_ids:
        upgrade_d7_user:
          - node_uid
  status:
    -
      plugin: get
      source: status
  created:
    -
      plugin: get
      source: created
  changed:
    -
      plugin: get
      source: timestamp
  promote:
    -
      plugin: get
      source: promote
  sticky:
    -
      plugin: get
      source: sticky
  revision_uid:
    -
      plugin: get
      source: revision_uid
  revision_log:
    -
      plugin: get
      source: log
  revision_timestamp:
    -
      plugin: get
      source: timestamp
  content_translation_source:
    -
      plugin: get
      source: source_langcode
  comment_node_page/0/status:
    -
      plugin: get
      source: comment
  body:
    -
      plugin: iterator
      source: body
      process:
        value: value
        format:
          plugin: default_value
          default_value: full_html
  field_attachment:
    -
      plugin: sub_process
      source: field_document_attachment_2
      process:
        target_id:
          -
            plugin: migration_lookup
            migration: upgrade_d7_file_private
            no_stub: true
            source: fid
          -
            plugin: update_file_to_document
  field_environment:
    -
      plugin: sub_process
      source: field_environment
      process:
        target_id:
          plugin: migration_lookup
          migration: upgrade_d7_taxonomy_term_environment
          no_stub: true
          source: tid
  field_published:
    -
      plugin: get
      source: field_published
  field_software_version:
    -
      plugin: migration_lookup
      migration: upgrade_d7_taxonomy_term_rev
      no_stub: true
      source: field_rev
  field_published_on:
    -
      plugin: get
      source: field_published_on
  field_product:
    -
      plugin: default_value
      default_value: 1
destination:
  plugin: 'entity_complete:node'
  translations: true
  default_bundle: page
migration_dependencies:
  required:
    - upgrade_d7_taxonomy_term_rev
    - upgrade_d7_attachment_to_document
    - upgrade_d7_user

thanks for helping me out on this.

Steps to reproduce

  1. Remove the following lines from d6 or d7 node_complete.yml and to run an incremental migration.
    2.   nid: tnid
 vid: vid
  2. Run a node migration.
  3. Check the results via the UI.
  4. There should be extra nodes on the destination site.

With patch and NOT entity translation type.

  1. Apply the patch in #2
  2. Repeat the steps above for a content type that does not use entity translation.
  3. Check the results via the UI
  4. There should be no extra node.

With patch and entity translation type.

  1. Apply the patch in #2
  2. Repeat the steps above for a content type that does use entity translation.
  3. Check the results via the UI
  4. There should extra nodes.

Proposed resolution

That patch in #3 works, except for entity translation.

Remaining tasks

Find out what why incremental with entity translation fails and fix it.

🐛 Bug report
Status

Closed: works as designed

Version

11.0 🔥

Component
Migration  →

Last updated 1 day ago

Created by

🇫🇮Finland joey-santiago

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment over 1 year ago →
    🇫🇷France damien laguerre

    I think the error here is the migration ID

    d7_node_complete must be replaced by the migration ID upgrade_d7_node_complete_page

    id: upgrade_d7_node_complete_page
migration_group: migrate_drupal_7
label: 'Node complete (Basic pages)'
source:
  plugin: d7_node_complete
  node_type: page
process:
  nid:
     -
       plugin: migration_lookup
       migration: upgrade_d7_node_complete_page <<-- Must match the migration ID
       source: tnid
     -
       plugin: node_complete_node_lookup
  • Status changed to Closed: works as designed over 1 year ago
  • Comment over 1 year ago →
    🇺🇸United States mikelutz Michigan, USA

    Closing this, as the Issue summary steps to reproduce require running custom migrations using contrib modules that don't populate required values for node_complete to work (i.e. unsetting the required nid and vid keys)

    The node_complete plugin is designed for use in core migrations and works only when preserving node and revision ids.

    Regarding the issue in #11, that error message occurs when changing a destination plugin or it's configuration such that the id count changes after a migration has been run and the map tables already created. This creates a mismatch between what the map plugin knows about past runs and what the destination plugin is telling it about the current run, and it can't do anything except refuse to save to the map. if you change your destination plugin or configuration you need to delete your map tables so they can be recreated for your new configuration.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024