Mismatched field definitions after address upgrade

Created on 4 January 2024, 6 months ago
Updated 14 May 2024, about 1 month ago

Problem/Motivation

Address 2.0.0 added an address_line3 property. The address_update_9201() function attempts to upgrade all field definitions in order to make them compatible with the new module.

It looks like this is failing for the Order report bundle of the Commerce Reporting module.

Steps to reproduce

Upgrade the address module from 1.x to 2.x on any Drupal Commerce instance where the Commerce Reporting module is enabled.

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Needs review

Version

2.0

Component

Code

Created by

🇨🇭Switzerland znerol

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @znerol
  • 🇨🇭Switzerland znerol

    I think the following is happening:

    address_update_9201() tries to find all field instances using getFieldMapByFieldType().

      $entity_field_manager = \Drupal::service('entity_field.manager');
      $entity_field_map = $entity_field_manager->getFieldMapByFieldType('address');
    

    However, that method only reports fields on fieldable entity types. Regrettably, the commerce report entity type is not fieldable, and as a result address_update_9201() fails to update the field definitions on the Order Report bundle.

  • 🇬🇧United Kingdom nigelwhite Marsden

    Same problem here.
    Some more detail, in case it's helpful --

    Error
    Mismatched entity and/or field definitions
    The following changes were detected in the entity type and field definitions.
    Order report
    The Address field needs to be updated.

    Yesterday I used composer to update Drupal core-recommended from 10.1.6 to 10.2.2, and drupal/commerce from 2.11 to 2.37. This introduced a new dependency - commerceguys/addressing ^2.1.1.

    Does this bug belong in commerceguys/addressing or elsewhere?

  • 🇷🇸Serbia bojanz

    No, it's an Address bug for sure.

    Unfortunately, I have no time to chase it, someone from the Commerce (Reporting) side will have to take a look.

  • 🇮🇱Israel jsacksick

    However, that method only reports fields on fieldable entity types

    All content entity types are fieldable.
    The OrderReport entity type therefore is fieldable. I don't see where the address field is defined.

  • 🇮🇱Israel jsacksick

    Found it:

    $fields['billing_address'] = BundleFieldDefinition::create('address')
          ->setLabel(t('Address'))
          ->setDescription(t('The billing address.'))
          ->setCardinality(1)
          ->setDisplayConfigurable('view', TRUE);
    
  • 🇮🇱Israel jsacksick

    The Address update function looks fine and should have updated the billing_address field. Not really sure what happened, don't really have time to dig further right now... But at first glance, the code looks good. Can you check if the actual schema was updated (i.e: do you have the address_line3 column?)

  • 🇮🇱Israel jsacksick

    Attaching a screenshot of the result of the following code:
    $entity_field_map = $entity_field_manager->getFieldMapByFieldType('address');
    @znerol: As you can see, the order report field is found:

  • 🇨🇭Switzerland znerol

    I worked around this issue by deleting all report data, uninstalling commerce reports and reinstalling it again. After that I did regenerate all report data.

    Unfortunately I cannot pinpoint the exact circumstances which are triggering the problem. It was reproducible though (the status report was always there when I upgraded from the same database snapshot).

  • Status changed to Needs review 5 months ago
  • 🇮🇳India keshav.k

    Execute the following code in an update_hook or using drush ev. It will fix all the issues related to the Mismatched field definitions for all entities.

    $entity_type_manager = \Drupal::entityTypeManager();
    $entity_type_manager->clearCachedDefinitions();

    $entity_type_ids = [];
    $change_summary = \Drupal::service('entity.definition_update_manager')->getChangeSummary();
    foreach ($change_summary as $entity_type_id => $change_list) {
    $entity_type = $entity_type_manager->getDefinition($entity_type_id);
    \Drupal::entityDefinitionUpdateManager()->installEntityType($entity_type);
    $entity_type_ids[] = $entity_type_id;
    }

  • 🇺🇸United States SocialNicheGuru

    This also fails on group and profiles not just commerce.

  • Status changed to Needs work 5 months ago
  • Status changed to Active 5 months ago
  • 🇷🇸Serbia bojanz

    There is no patch here, changing status.

  • First commit to issue fork.
  • Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.2 & MySQL 8
    last update 4 months ago
    Waiting for branch to pass
  • Status changed to Needs review 4 months ago
  • 🇳🇱Netherlands Ruuds

    I also got this issue when upgrading from 1.x to 2.x. I found there was a field storage definition (field.storage.profile.address) which indeed didn't get the address_line3 field after the updated.

    I've created a MR which contains a new update hook which checks which field storage definitions are missing the address_line3 field, and adds it when needed.

  • Pipeline finished with Success
    4 months ago
    Total: 390s
    #88655
  • 🇷🇸Serbia bojanz

    Can we please try to debug the original update function, and fix the problem there, before we introduce update functions with workarounds?

    Our main problem here is that none of the maintainers have sites that are affected, so we need a db dump where the problem can be observed, or we need a developer with such a site to do the debugging themselves.

  • 🇳🇱Netherlands Ruuds

    That would be the best solution, but as there is already a released version which only updates the tables partially, an additional update hook would be the most appropriate in my opinion. If really needed, I can try to extract a minimal db dump which contains the problem.

  • 🇺🇸United States SocialNicheGuru

    I added the MR and installed address.
    That worked

    But I received the Mismatched error on the status report
    I enabled entity_update and ran drush upe --basic

    I received this error:

    Exception thrown while performing a schema update. SQLSTATE[42S22]: Column not found: 1054 Unknown column 'field_my_address_address_line3' in 'where clause': SELECT 1 AS "expression"
    FROM
    "group_revision__field_my_address" "t"
    WHERE ("field_my_address_langcode" IS NOT NULL) OR ("field_my_address_country_code" IS NOT NULL) OR ("field_my_address_administrative_area" IS NOT NULL) OR ("field_my_address_locality" IS NOT NULL) OR ("field_my_address_dependent_locality" IS NOT NULL) OR ("field_my_address_postal_code" IS NOT NULL) OR ("field_my_address_sorting_code" IS NOT NULL) OR ("field_my_address_address_line1" IS NOT NULL) OR ("field_my_address_address_line2" IS NOT NULL) OR ("field_my_address_address_line3" IS NOT NULL) OR ("field_my_address_organization" IS NOT NULL) OR ("field_my_address_given_name" IS NOT NULL) OR ("field_my_address_additional_name" IS NOT NULL) OR ("field_my_address_family_name" IS NOT NULL)
    LIMIT 1 OFFSET 0; Array
    (
    )

  • Status changed to Needs work 4 months ago
  • Status changed to Needs review 4 months ago
  • Status changed to Needs work 4 months ago
  • 🇺🇸United States rymcveigh

    We have encountered the same issue on a custom entity, a custom block_type and with a commerce order. The initial update hook (9201) will not successfully run because it "Cannot add field 'some_field_name.field_address_address_line3': field already exists."

    The MR/Patch does not work for us because the first update hook keeps failing. I agree with @bojanz that "we should try to debug the original update function and fix the problem there before we introduce update functions with workarounds".

  • 🇺🇸United States SocialNicheGuru

    I think @rymcveigh view that the original update hook does not work on custom entities is a good one.

    How can we improve the update hook to also work on custom entities.

  • 🇺🇸United States dww

    We might need both a fix for the original update function, and perhaps a 2nd update function for sites that are in a partially mangled state, TBD.

    I haven't yet seen this myself, so I have no insights to share from direct experience.

    This seems like a blocker to a 2.0.1 release, so I'd love to get this fixed ASAP.

    Thanks,
    -Derek

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.2 & MySQL 8
    last update 4 months ago
    43 pass
  • 🇳🇱Netherlands Ruuds

    I've added a check in address_update_9201 which does not try to add the field again if it already exists. @rymcveigh this will probably fix your error.

  • Pipeline finished with Success
    4 months ago
    Total: 364s
    #90196
  • 🇺🇸United States SocialNicheGuru

    I applied the new MR
    I used drush to reset the address module update
    drush ev "\Drupal::keyValue('system.schema')->set('address', (int) 9200)";
    I ran drush updatedb
    address_line3 was added to all address fields

  • 🇺🇸United States rymcveigh

    Unfortunately, the patch (in its current state) does not work for my custom entity. Here's what I tried:

    I tried adjusting the field definition for my entity prior to running a updb with these changes:

    $fields['location'] = BaseFieldDefinition::create('address')
          ->setLabel('Location')
          ->setRevisionable(TRUE)
          ->setCardinality(1)
          ->setDefaultValue([
            'county_code' => 'US',
          ])
          ->setRequired(TRUE)
          ->setSetting('available_countries', ['US'])
          ->setSetting('field_overrides', [
            AddressField::GIVEN_NAME => ['override' => FieldOverride::HIDDEN],
            AddressField::FAMILY_NAME => ['override' => FieldOverride::HIDDEN],
            AddressField::ADDITIONAL_NAME => ['override' => FieldOverride::HIDDEN],
            AddressField::ORGANIZATION => ['override' => FieldOverride::HIDDEN],
            AddressField::ADDRESS_LINE1 => ['override' => FieldOverride::HIDDEN],
            AddressField::ADDRESS_LINE2 => ['override' => FieldOverride::HIDDEN],
            AddressField::ADDRESS_LINE3 => ['override' => FieldOverride::HIDDEN],
            AddressField::LOCALITY => ['override' => FieldOverride::OPTIONAL],
            AddressField::ADMINISTRATIVE_AREA => ['override' => FieldOverride::OPTIONAL],
            AddressField::POSTAL_CODE => ['override' => FieldOverride::HIDDEN],
          ])
          ->setDisplayOptions('form', [
            'type' => 'address_default',
            'weight' => 4,
          ])
          ->setDisplayConfigurable('view', TRUE)
          ->setDisplayConfigurable('form', TRUE);
    

    That resulting in this error: [error] Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.

    I then imported a fresh DB and tried running the updb without altering my field definition and got the same error: [error] Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.

    I am going to try to trigger a field definition update before the address update and see if that helps. If it doesn't help, I will look at the changes in the MR today and see if I can figure out why this may be happening. Thanks for helping everyone.

  • 🇺🇸United States rymcveigh

    I tried again today using Address 2.0.1 as the base version of the module.

    This workflow worked for me:

    • Upgrade from version 1.12.0 to 2.0.1 of the address module: composer require 'drupal/address:^2.0' --with-all-dependencies
    • run drush updatedb
    • Watch address_update_9201 fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
    • Apply the patch from this MR
    • Rerun drush updatedb and watch it pass

    This workflow failed the first time I tried to do an updatedb but passed the second time I ran it:

    • Upgrade from version 1.12.0 to 2.0.1 of the address module: composer require 'drupal/address:^2.0' --with-all-dependencies
    • Apply the patch from this MR
    • Run drush updatedb and watch it fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
    • Rerun drush updatedb and watch it pass.

    That made me wonder what would happen if I ran drush updatedb twice using version 2.0.1 of the module without the patch. This did not work. Here is what I got:

    • Upgrade from version 1.12.0 to 2.0.1 of the address module: composer require 'drupal/address:^2.0' --with-all-dependencies
    • Run drush updatedb and watch it fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
    • Rerun drush updatedb and watch it fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
  • I tried to apply the patch but it throw an error while running the update.
    SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'DEFAULT NULL' at line 1: ALTER TABLE "profile__address" ADD "address_address_line3" DEFAULT NULL; Array

    It tries to access the key that doesn't exist yet. So I made a small update and it worked for me.

  • The previous patch is broken, here is the correct one.

  • Status changed to Needs review 3 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.2 & MySQL 8
    last update 3 months ago
    43 pass
  • 🇩🇪Germany ktpm

    I'm still getting

    commerce_order_report entity type :
    The Address field needs to be updated.

    with patch #30.

  • 🇺🇦Ukraine khiminrm

    Solution from the #10 helped to fix the error for the Order report.
    Used this code in the custom module's update function:

     $entity_type = \Drupal::entityTypeManager()->getDefinition('commerce_order_report');
      \Drupal::entityDefinitionUpdateManager()->installEntityType($entity_type);
    
Production build 0.69.0 2024