Mismatched field definitions after address upgrade

I think the following is happening:

address_update_9201() tries to find all field instances using getFieldMapByFieldType().

  $entity_field_manager = \Drupal::service('entity_field.manager');
  $entity_field_map = $entity_field_manager->getFieldMapByFieldType('address');

However, that method only reports fields on fieldable entity types. Regrettably, the commerce report entity type is not fieldable, and as a result address_update_9201() fails to update the field definitions on the Order Report bundle.

Same problem here.
Some more detail, in case it's helpful --

Error
Mismatched entity and/or field definitions
The following changes were detected in the entity type and field definitions.
Order report
The Address field needs to be updated.

Yesterday I used composer to update Drupal core-recommended from 10.1.6 to 10.2.2, and drupal/commerce from 2.11 to 2.37. This introduced a new dependency - commerceguys/addressing ^2.1.1.

Does this bug belong in commerceguys/addressing or elsewhere?

No, it's an Address bug for sure.

Unfortunately, I have no time to chase it, someone from the Commerce (Reporting) side will have to take a look.

However, that method only reports fields on fieldable entity types

All content entity types are fieldable.
The OrderReport entity type therefore is fieldable. I don't see where the address field is defined.

Found it:

$fields['billing_address'] = BundleFieldDefinition::create('address')
      ->setLabel(t('Address'))
      ->setDescription(t('The billing address.'))
      ->setCardinality(1)
      ->setDisplayConfigurable('view', TRUE);

The Address update function looks fine and should have updated the billing_address field. Not really sure what happened, don't really have time to dig further right now... But at first glance, the code looks good. Can you check if the actual schema was updated (i.e: do you have the address_line3 column?)

Attaching a screenshot of the result of the following code:
$entity_field_map = $entity_field_manager->getFieldMapByFieldType('address');
@znerol: As you can see, the order report field is found:

I worked around this issue by deleting all report data, uninstalling commerce reports and reinstalling it again. After that I did regenerate all report data.

Unfortunately I cannot pinpoint the exact circumstances which are triggering the problem. It was reproducible though (the status report was always there when I upgraded from the same database snapshot).

Execute the following code in an update_hook or using drush ev. It will fix all the issues related to the Mismatched field definitions for all entities.

$entity_type_manager = \Drupal::entityTypeManager();
$entity_type_manager->clearCachedDefinitions();

$entity_type_ids = [];
$change_summary = \Drupal::service('entity.definition_update_manager')->getChangeSummary();
foreach ($change_summary as $entity_type_id => $change_list) {
$entity_type = $entity_type_manager->getDefinition($entity_type_id);
\Drupal::entityDefinitionUpdateManager()->installEntityType($entity_type);
$entity_type_ids[] = $entity_type_id;
}

This also fails on group and profiles not just commerce.

There is no patch here, changing status.

I also got this issue when upgrading from 1.x to 2.x. I found there was a field storage definition (field.storage.profile.address) which indeed didn't get the address_line3 field after the updated.

I've created a MR which contains a new update hook which checks which field storage definitions are missing the address_line3 field, and adds it when needed.

Can we please try to debug the original update function, and fix the problem there, before we introduce update functions with workarounds?

Our main problem here is that none of the maintainers have sites that are affected, so we need a db dump where the problem can be observed, or we need a developer with such a site to do the debugging themselves.

That would be the best solution, but as there is already a released version which only updates the tables partially, an additional update hook would be the most appropriate in my opinion. If really needed, I can try to extract a minimal db dump which contains the problem.

I added the MR and installed address.
That worked

But I received the Mismatched error on the status report
I enabled entity_update and ran drush upe --basic

I received this error:

Exception thrown while performing a schema update. SQLSTATE[42S22]: Column not found: 1054 Unknown column 'field_my_address_address_line3' in 'where clause': SELECT 1 AS "expression"
FROM
"group_revision__field_my_address" "t"
WHERE ("field_my_address_langcode" IS NOT NULL) OR ("field_my_address_country_code" IS NOT NULL) OR ("field_my_address_administrative_area" IS NOT NULL) OR ("field_my_address_locality" IS NOT NULL) OR ("field_my_address_dependent_locality" IS NOT NULL) OR ("field_my_address_postal_code" IS NOT NULL) OR ("field_my_address_sorting_code" IS NOT NULL) OR ("field_my_address_address_line1" IS NOT NULL) OR ("field_my_address_address_line2" IS NOT NULL) OR ("field_my_address_address_line3" IS NOT NULL) OR ("field_my_address_organization" IS NOT NULL) OR ("field_my_address_given_name" IS NOT NULL) OR ("field_my_address_additional_name" IS NOT NULL) OR ("field_my_address_family_name" IS NOT NULL)
LIMIT 1 OFFSET 0; Array
(
)

We have encountered the same issue on a custom entity, a custom block_type and with a commerce order. The initial update hook (9201) will not successfully run because it "Cannot add field 'some_field_name.field_address_address_line3': field already exists."

The MR/Patch does not work for us because the first update hook keeps failing. I agree with @bojanz that "we should try to debug the original update function and fix the problem there before we introduce update functions with workarounds".

I think @rymcveigh view that the original update hook does not work on custom entities is a good one.

How can we improve the update hook to also work on custom entities.

We might need both a fix for the original update function, and perhaps a 2nd update function for sites that are in a partially mangled state, TBD.

I haven't yet seen this myself, so I have no insights to share from direct experience.

This seems like a blocker to a 2.0.1 release, so I'd love to get this fixed ASAP.

Thanks,
-Derek

I've added a check in address_update_9201 which does not try to add the field again if it already exists. @rymcveigh this will probably fix your error.

I applied the new MR
I used drush to reset the address module update
drush ev "\Drupal::keyValue('system.schema')->set('address', (int) 9200)";
I ran drush updatedb
address_line3 was added to all address fields

Unfortunately, the patch (in its current state) does not work for my custom entity. Here's what I tried:

I tried adjusting the field definition for my entity prior to running a updb with these changes:

$fields['location'] = BaseFieldDefinition::create('address')
      ->setLabel('Location')
      ->setRevisionable(TRUE)
      ->setCardinality(1)
      ->setDefaultValue([
        'county_code' => 'US',
      ])
      ->setRequired(TRUE)
      ->setSetting('available_countries', ['US'])
      ->setSetting('field_overrides', [
        AddressField::GIVEN_NAME => ['override' => FieldOverride::HIDDEN],
        AddressField::FAMILY_NAME => ['override' => FieldOverride::HIDDEN],
        AddressField::ADDITIONAL_NAME => ['override' => FieldOverride::HIDDEN],
        AddressField::ORGANIZATION => ['override' => FieldOverride::HIDDEN],
        AddressField::ADDRESS_LINE1 => ['override' => FieldOverride::HIDDEN],
        AddressField::ADDRESS_LINE2 => ['override' => FieldOverride::HIDDEN],
        AddressField::ADDRESS_LINE3 => ['override' => FieldOverride::HIDDEN],
        AddressField::LOCALITY => ['override' => FieldOverride::OPTIONAL],
        AddressField::ADMINISTRATIVE_AREA => ['override' => FieldOverride::OPTIONAL],
        AddressField::POSTAL_CODE => ['override' => FieldOverride::HIDDEN],
      ])
      ->setDisplayOptions('form', [
        'type' => 'address_default',
        'weight' => 4,
      ])
      ->setDisplayConfigurable('view', TRUE)
      ->setDisplayConfigurable('form', TRUE);

That resulting in this error: [error] Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.

I then imported a fresh DB and tried running the updb without altering my field definition and got the same error: [error] Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.

I am going to try to trigger a field definition update before the address update and see if that helps. If it doesn't help, I will look at the changes in the MR today and see if I can figure out why this may be happening. Thanks for helping everyone.

I tried again today using Address 2.0.1 as the base version of the module.

This workflow worked for me:

• Upgrade from version 1.12.0 to 2.0.1 of the address module: composer require 'drupal/address:^2.0' --with-all-dependencies
• run drush updatedb
• Watch address_update_9201 fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
• Apply the patch from this MR
• Rerun drush updatedb and watch it pass

This workflow failed the first time I tried to do an updatedb but passed the second time I ran it:

• Upgrade from version 1.12.0 to 2.0.1 of the address module: composer require 'drupal/address:^2.0' --with-all-dependencies
• Apply the patch from this MR
• Run drush updatedb and watch it fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
• Rerun drush updatedb and watch it pass.

That made me wonder what would happen if I ran drush updatedb twice using version 2.0.1 of the module without the patch. This did not work. Here is what I got:

• Upgrade from version 1.12.0 to 2.0.1 of the address module: composer require 'drupal/address:^2.0' --with-all-dependencies
• Run drush updatedb and watch it fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.
• Rerun drush updatedb and watch it fail with error: Cannot add field 'maf_memorial_field_revision.location__address_line3': table doesn't exist.

I tried to apply the patch but it throw an error while running the update.
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'DEFAULT NULL' at line 1: ALTER TABLE "profile__address" ADD "address_address_line3" DEFAULT NULL; Array

It tries to access the key that doesn't exist yet. So I made a small update and it worked for me.

The previous patch is broken, here is the correct one.

I'm still getting

commerce_order_report entity type :
The Address field needs to be updated.

with patch #30.

Solution from the #10 helped to fix the error for the Order report.
Used this code in the custom module's update function:

 $entity_type = \Drupal::entityTypeManager()->getDefinition('commerce_order_report');
  \Drupal::entityDefinitionUpdateManager()->installEntityType($entity_type);

#10 also fixed it for me, with the same code as #33. I seem to have missed that before!

I've updated the fork with the code of #30.