Changes to field mappers in config are never imported

Created on 2 October 2024, about 2 months ago

Configuration changes to field mappers are never imported so it seems.

For example, I created a new (fictive) external entity type through the UI and exported it to config:

uuid: 92a92745-279b-4029-94b1-15c185c02948
langcode: en
status: true
dependencies: {  }
id: external_item
label: 'External item'
label_plural: 'External items'
description: ''
content_class: Drupal\external_entities\Entity\ExternalEntity
read_only: false
debug_level: 0
generate_aliases: false
field_mappers:
  id:
    id: generic
    config:
      property_mappings:
        value:
          id: simple
          config:
            mapping: id
            required_field: true
            main_property: true
            data_processors: {  }
            description: ''
  uuid:
    id: generic
    config:
      property_mappings:
        value:
          id: simple
          config:
            mapping: id
            required_field: false
            main_property: true
            data_processors: {  }
            description: ''
  title:
    id: generic
    config:
      property_mappings:
        value:
          id: simple
          config:
            mapping: title
            required_field: true
            main_property: true
            data_processors: {  }
            description: ''
  langcode:
    id: generic
    config:
      property_mappings:
        value:
          id: simple
          config:
            mapping: language
            required_field: false
            main_property: true
            data_processors: {  }
            description: ''
field_mapping_notes: ''
data_aggregator:
  id: single
  config:
    storage_clients:
      -
        id: rest
        config:
          endpoint: 'https://www.externalentities.com'
          endpoint_options:
            single: ''
            count: ''
            count_mode: entities
            cache: false
            limit_qcount: 0
            limit_qtime: 0
            requests_by_user: false
          response_format: json
          data_path:
            list: ''
            single: ''
            keyed_by_id: false
            count: ''
          pager:
            default_limit: 0
            type: pagination
            page_parameter: ''
            page_parameter_type: pagenum
            page_start_one: false
            always_query: false
            page_size_parameter: ''
            page_size_parameter_type: pagesize
          api_key:
            type: none
            header_name: ''
            key: ''
          http:
            headers: ''
          parameters:
            list: {  }
            list_param_mode: query
            single: {  }
            single_param_mode: query
          filtering:
            drupal: false
            basic: false
            basic_fields: {  }
            list_support: none
            list_join: ''
data_aggregator_notes: ''
persistent_cache_max_age: 0
annotation_entity_type_id: null
annotation_bundle_id: null
annotation_field_name: null
inherits_annotation_fields: false

If you then, for example, change the mapping (in config) for the id field to:

  id:
    id: generic
    config:
      property_mappings:
        value:
          id: simple
          config:
            mapping: identifier // This is what I changed
            required_field: true
            main_property: true
            data_processors: {  }
            description: ''

... and then execute a config import, the change is never imported. Executing a second, third, ... time doesn't help either.

Changes to other parts of the external entity type (such as the label, for example) works as expected.

🐛 Bug report
Status

Active

Version

3.0

Component

Code

Created by

🇧🇪Belgium rp7

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @rp7
  • 🇧🇪Belgium rp7

    I've found the code that's causing this, although it's not clear to me why this code is there in the first place. Removing it fixes the issue & I don't immediately see something that breaks without this code.

    Patch attached removes the code. Not a real solution, more of a (temporary) quick-fix to work around this issue.

  • 🇫🇷France guignonv

    Yep, that code is needed. I'll try to remind why, but it is needed... I'll see what's going on.

  • 🇫🇷France guignonv

    On my site running Drupal 10.3.5 with PHP 8.1.14 (and PostgreSQL 12 db) and external entities dev 3ac5d32c4b09dd7beefeb42552013c6dbf921838, I was not able to reproduce your problem. Note: that xntt dev version should not differ from the beta 1 in the code used for config.

    What I did try:
    1) (single) export an existing external entity type of mine
    2) make changes in the 'id' field mapping by changing the target source field name, changed the prefix of an aggregation group, changed the REST URL of a REST client
    3) then imported that modified yml config
    4) re-opened the external entity type of mine and the changes made were there. Loading the modified config did modify my current external entity type in database.

    Then:
    5) Loaded your example yml config still with the Drupal config importer (/admin/config/development/configuration/single/import)
    It created a new external entity type as expected with the corresponding field mapping and REST client settings.
    6) I modified you yml:

    uuid: 92a92745-279b-4029-94b1-15c185c02948
    langcode: en
    status: true
    dependencies: {  }
    id: external_item
    label: 'External item'
    label_plural: 'External items'
    description: ''
    content_class: Drupal\external_entities\Entity\ExternalEntity
    read_only: false
    debug_level: 0
    generate_aliases: false
    field_mappers:
      id:
        id: generic
        config:
          property_mappings:
            value:
              id: direct
              config:
                mapping: id
                required_field: true
                main_property: true
                data_processors: {  }
                description: ''
      uuid:
        id: generic
        config:
          property_mappings:
            value:
              id: simple
              config:
                mapping: blabla
                required_field: false
                main_property: true
                data_processors: {  }
                description: ''
      title:
        id: generic
        config:
          property_mappings:
            value:
              id: simple
              config:
                mapping: title
                required_field: true
                main_property: true
                data_processors: {  }
                description: ''
      langcode:
        id: generic
        config:
          property_mappings:
            value:
              id: simple
              config:
                mapping: language
                required_field: false
                main_property: true
                data_processors: {  }
                description: ''
    field_mapping_notes: ''
    data_aggregator:
      id: single
      config:
        storage_clients:
          -
            id: rest
            config:
              endpoint: 'https://www.toto.com'
              endpoint_options:
                single: ''
                count: ''
                count_mode: entities
                cache: false
                limit_qcount: 0
                limit_qtime: 0
                requests_by_user: false
              response_format: json
              data_path:
                list: ''
                single: ''
                keyed_by_id: false
                count: ''
              pager:
                default_limit: 0
                type: pagination
                page_parameter: ''
                page_parameter_type: pagenum
                page_start_one: false
                always_query: false
                page_size_parameter: ''
                page_size_parameter_type: pagesize
              api_key:
                type: none
                header_name: ''
                key: ''
              http:
                headers: ''
              parameters:
                list: {  }
                list_param_mode: query
                single: {  }
                single_param_mode: query
              filtering:
                drupal: false
                basic: false
                basic_fields: {  }
                list_support: none
                list_join: ''
    data_aggregator_notes: ''
    persistent_cache_max_age: 0
    annotation_entity_type_id: null
    annotation_bundle_id: null
    annotation_field_name: null
    inherits_annotation_fields: false
    
    If you then, for example, change the mapping (in config) for the id field to:
    
      id:
        id: generic
        config:
          property_mappings:
            value:
              id: simple
              config:
                mapping: identifier // This is what I changed
                required_field: true
                main_property: true
                data_processors: {  }
                description: ''
    

    (basically, the 'id' field mapper type, the 'uuid' mapping, and the endpoint of the REST client)
    6) I re-imported the config with the changes.
    ...the changes were there on the UI when I edited "external_item".

    By the way, I also have "Configuration Inspector" 2.1.9" recently update that "now" displays config schema problems that it did not before. That might help me investigate the other issue #3476772 you had I did not have before...

    Could it be a Drupal version issue or some other changes you made in the source code of external entities?
    If it is confirmed on your side, it is a major problem.

    The code part you disabled is required as we have multiple configs living in parallel that need to be synchronized at some point. The external entity type itself contains the configs of its field mappers and its storage aggregator. But those configs also include the configs of sub-plugins (ie. property mappers, data processors, storage clients). We can't separate those configs as they define the whole external entity type. But when we instantiate each plugin, it needs its own part of the config. So, later, if when change the config of a plugin using its own methods to do so, its own part of the config will be modified but the whole external entity config hold by the external entity itself won't be modified accordingly at the same time. Therefore, when an external process will need to access to the external entity type config, it will require a synchronization. That's the reason why this "get" method has been overridden. I hope it clarify things but don't hesitate to ask for more details if it not clear (I'm also available on drupalchat.me from times to times or on demand).
    I also thought about using PHP variable references to have "sub-configs" linked to each plugin config which would solve the need of any synchronization process but I forgot why but it was not possible for some reasons... and anyway, the config stored in memory contains extra-elements that must not be exported so the "get" method also clear those.

  • 🇫🇷France guignonv

    I was able to reproduce a similar problem. When I have field ("language" field for instance) that is using a generic field mapper and I want to set it to "not mapped", the change is not saved. However, changes to other fields are saved.

    So I will investigate that case and see what's going on.

  • 🇫🇷France guignonv

    Raf, could you give a try to this issue fork? I added a setter that might do the job for your issue case (since I cannot reproduce it exactly on my side).

  • Merge request !60Added property setter for external entity type → (Merged) created by guignonv
  • 🇫🇷France guignonv

    Please re-open if not fixed.

  • 🇧🇪Belgium rp7

    Sorry for the late reply, but it looks like it works. Thanks!

  • 🇫🇷France guignonv

    Awesome! :D

  • 🇧🇪Belgium rp7

    Argh, sorry. But I think I cheered a little too early.
    It looks like updates are now correctly imported, but a deleted mapping is not.
    So if I delete a mapping for a field from the config file, import it & perform drush cex -y - it's added again.
    Any idea?

  • 🇫🇷France guignonv

    I'll have a look in the coming days...

  • 🇫🇷France guignonv

    I can reproduce the problem. I'll fix it.

    • 38e352db committed on 3.0.x
      Issue #3478219: Changes to field mappers in config are never imported (#...
  • 🇫🇷France guignonv

    It should have been fixed by commit #38e352db. Please test and confirm!... ;)
    It was a regression because of change made on how sub-plugins are managed (now using DefaultLazyPluginCollection class).

Production build 0.71.5 2024