Allow entity_generate to run process pipelines on values

Created on 22 August 2025, about 1 month ago

Problem/Motivation

I am using Entity Generate to generate taxonomy terms on the fly, which seems like a common use case. My terms also have an image attached, which I also need to import.

The usual way of doing this is something like

process:
  _location_image:
    plugin: image_import
    source: imageUrl
    destination: constants/image_destination
    alt: alternateText

  field_location:
    plugin: entity_generate_process
    entity_type: taxonomy_term
    bundle_key: vid
    bundle: location
    value_key: name
    values:
      field_image: '@_location_image'

However this imports the image for every row, even if the term already exists.

Steps to reproduce

Proposed resolution

I propose extending values so if the value is an array it is treated as a process plugin; this allows additional processing on other fields, but that is only run when the entity is created. For example:

process:
  field_location:
    plugin: entity_generate_process
    entity_type: taxonomy_term
    bundle_key: vid
    bundle: location
    value_key: name
    values:
      description/value:
        plugin: snippet
        module: custom_migrate
        path: clean_html
        source: locationDescription
      description/format:
        plugin: default_value
        default_value: basic_html
      path/alias:
        plugin: str_replace
        regex: true
        source: locationUrl
        search: '/^https?:\/\/[^\/]+/'
        replace: ''
      path/pathauto:
        plugin: default_value
        default_value: false
      field_image:
        plugin: image_import
        source: imageUrl
        destination: constants/image_destination
        alt: alternateText

Remaining tasks

User interface changes

API changes

Data model changes

✨ Feature request

Status

Active

Version

6.0

Component

Plugins

Created by

🇬🇧United Kingdom longwave UK

Live updates comments and jobs are added and updated live.

Comments & Activities

Issue created by @longwave
@longwave opened merge request.
Comment about 1 month ago →
🇬🇧United Kingdom longwave UK
This needs work on docs (and tests) but a custom plugin with this code that extends entity_generate works for me.
Comment about 1 month ago →
🇬🇧United Kingdom longwave UK
Comment about 1 month ago →
🇬🇧United Kingdom longwave UK
It just struck me that values here is just an entire process pipeline, and get is the default, so perhaps we can do away with the loop entirely!
Comment about 1 month ago →
heddn Nicaragua
This is some of the oldest code in migrate_plus. Can we add tests to make sure this works how we expect and regressions don't break this? Also, we should add docs for how to make this super cool feature work.
First commit to issue fork.

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024