Update 8004 is failing

I have re-add the hook_entity_base_field_info to make the field authorization_drupal_roles_roles available again via Drupal::entityDefinitionUpdateManager.

Hey, i am a colleague of sunlix. After we fixed the missing authorization_drupal_roles_entity_base_field_info hook, there was a follow up issue in the new update hook authorization_drupal_roles_update_8004.

Because a do-while-loop is used, under some circumstances the loop is run through once too often. The finished variable is calculated like this: $finished = ($sandbox['current'] - 1) / $sandbox['total'];. This makes it appear as if the additional loop pass is intended. Unfortunately, the additional loop pass results in a user with the ID ‘NULL’ being loaded.

My commit converts the do-while-loop into a while-loop, fixes the calculation for the finished variable and adds an initialisation for the new variable. I think, now we are ready for a review.

The solution proposed in fork 3473259 solves the problem for me!

I applied the patch https://git.drupalcode.org/project/authorization/-/merge_requests/32.diff via composer without problems.

Grazie!

We are experiencing the same (or very similar) issue:

Do you wish to run the specified pending updates? (yes/no) [yes]:
>

> [notice] Update started: authorization_drupal_roles_update_8004
> [notice] No profiles have revoke_provider_provisioned enabled. No roles to move to user data.
> [warning] Undefined array key 1 Schema.php:580
> [warning] Undefined array key 1 Schema.php:580
> [warning] Undefined array key 1 Schema.php:580
> [error] Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR: relation "field_deleted_data_21874036cb____idx" already exists: ALTER INDEX "public"."user__authorization_drupal_roles_roles__revision_id__idx" RENAME TO field_deleted_data_21874036cb____idx; Array
> (
> )
>
> [error] Update failed: authorization_drupal_roles_update_8004
[error] Update aborted by: authorization_drupal_roles_update_8004
[error] Finished performing updates.

We are currently unable to login with any LDAP user account. When can we expect an official fix?

Not applied the patch yet, but now when I run drush updatedb again, I get the following:

> [notice] Update started: authorization_drupal_roles_update_8004
> [notice] No profiles have revoke_provider_provisioned enabled. No roles to move to user data.
> [warning] Undefined array key 1 Schema.php:587
> [error] Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR: relation "field_deleted_data_21874036cb____pkey" already exists: ALTER INDEX "public"."field_deleted_data_21874036cb____pkey" RENAME TO field_deleted_data_21874036cb____pkey; Array
> (
> )
>
> [error] Update failed: authorization_drupal_roles_update_8004
[error] Update aborted by: authorization_drupal_roles_update_8004
[error] Finished performing updates.

I have the same problem when executing the `drush updb` command. The problem occurs after upgrading drupal/authorization to version 1.4.0

...@...:/application$ drush updb -y
 ------------- ----------- --------------- ------------------------------------ 
  Module        Update ID   Type            Description                         
 ------------- ----------- --------------- ------------------------------------ 
  authorizati   8004        hook_update_n   8004 - Migrate roles from field     
  on_drupal_r                               'authorization_drupal_roles_roles'  
  oles                                      to user data.                       
 ------------- ----------- --------------- ------------------------------------ 


 // Do you wish to run the specified pending updates?: yes.                                                        

[2024-09-16T13:46:39.304957+02:00] system.NOTICE: Module node has an entry in the system.schema key/value storage, but is not installed. <a href="https://www.drupal.org/node/3137656">More information about this error</a>. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
>  [notice] Update started: authorization_drupal_roles_update_8004
> [2024-09-16T13:46:50.536441+02:00] authorization_drupal_roles.NOTICE: No profiles have revoke_provider_provisioned enabled. No roles to move to user data. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
>  [warning] Undefined array key 1 Schema.php:587
>  [error]  Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR:  relation "field_deleted_data_21874036cb____pkey" already exists: ALTER INDEX "public"."field_deleted_data_21874036cb____pkey" RENAME TO field_deleted_data_21874036cb____pkey; Array
> (
> )
>  
>  [error]  Update failed: authorization_drupal_roles_update_8004 

I have the same problem when executing the `drush updb` command. The problem occurs after upgrading drupal/authorization to version 1.4.0

...@...:/application$ drush updb -y
 ------------- ----------- --------------- ------------------------------------ 
  Module        Update ID   Type            Description                         
 ------------- ----------- --------------- ------------------------------------ 
  authorizati   8004        hook_update_n   8004 - Migrate roles from field     
  on_drupal_r                               'authorization_drupal_roles_roles'  
  oles                                      to user data.                       
 ------------- ----------- --------------- ------------------------------------ 


 // Do you wish to run the specified pending updates?: yes.                                                        

[2024-09-16T13:46:39.304957+02:00] system.NOTICE: Module node has an entry in the system.schema key/value storage, but is not installed. <a href="https://www.drupal.org/node/3137656">More information about this error</a>. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
>  [notice] Update started: authorization_drupal_roles_update_8004
> [2024-09-16T13:46:50.536441+02:00] authorization_drupal_roles.NOTICE: No profiles have revoke_provider_provisioned enabled. No roles to move to user data. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
>  [warning] Undefined array key 1 Schema.php:587
>  [error]  Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR:  relation "field_deleted_data_21874036cb____pkey" already exists: ALTER INDEX "public"."field_deleted_data_21874036cb____pkey" RENAME TO field_deleted_data_21874036cb____pkey; Array
> (
> )
>  
>  [error]  Update failed: authorization_drupal_roles_update_8004 

Can you fill out the issue summary?

authorization_drupal_roles_entity_base_field_infowas removed. Why are you adding it back?

We have re-add authorization_drupal_roles_entity_base_field_info for the migration update hook 8004.
In the update hook 8004 the base field is queried but it will not found due to missing metadata for the field API. (the database table is there, but not the field metadata).
So to make the migration the module need to know the old base field to make the query and fetch the data for the migration.
The module have to make the migration and in a later major version it can safely removed.

I'm now unable to login even as a local user. After what appears to be successful input of credentials on logging in, you are presented with a blank /user/login page. Navigating to another page shows that you are not logged in.

Not sure why with each day, the situation is getting worse for us. All we are doing is running "drush updatedb".

I have three findings that I would like to share that seem to work for me as a workaround. YMMV, please try this out first in a test environment.

These findings were made on a site that had unsuccesfully been updated to authorization 1.4.

1. After I cancel and delete all LDAP-provided user accounts from my site, the 'authorization' update 8004 works (drush updb -y). For me this is a valid workaround, as in my case all users are provided via LDAP, and are recreated the next time a valid user logs in.

2. If I manually set the 'authorisation' module version in composer.json back to to "drupal/authorization": "1.3", and run composer update, the login functionality is restored (for as far as I can see), without having to delete the user accounts as in step 1. Not sure what the effect of the failed update 8004 is in this case. This step requires you to manually change composer.json again when the 'authorization' module provides a valid 1.4 update path.

3. Overwriting the unsuccesful update 1.4 with the dev version of [16 Sep 2024 at 21:45 CEST] did not provide a working update 8004 for me.

Hoping for a resolution shortly,
Mark.

@ishore,

You are not using the revoke roles provisioned by authorization.

No profiles have revoke_provider_provisioned enabled. No roles to move to user data.

You could:

1. Revert to 8.x-1.3
2. Uninstall authorization_drupal_roles
3. Upgrade to 8.x-1.4
4. Re-install authorization_drupal_roles

Thanks @bluegeek. Actually, before you posted your suggestion, I had tried suggestion 1 from @cpdp. I was able do this because deleting all LDAP users wasn't a major headache because I am doing this a on a test instance.

After deleting all LDAP users, I was able to run "drush updatedb" cleanly. However, I'm still having major issues that I will try and explain.

I can only login once as the admin user. The login isn't clean - I am greeted by a blank page after entering the TFA code (we use the TFA module). However, going back to the front page shows that I am logged in. After that, when I try and login a second time, I just see a white screen, stuck on /user/login. However, clearing the cache on the browser allows me to login successfully again.

Logging in as an LDAP user, I am just greeted by a blank white page, stuck on /user/login. When I go to the front page, I see the front page but it also says I am not logged in. However if I view recent log messages, I see that the LDAP user has been created. All subsequent attempts to login as that user fail, although clearly the user and password are accepted and recent log messages says "session opened for ".

Managed to get ldap user login working again by uninstalling authorization_group. We had version 2.0.x-dev@dev installed.

Hi everyone,

The original problem is as follows:
The hook_entity_base_field_info which brings the metadata of the field authorization_drupal_roles_roles is deleted.
At the same time, the authorization_drupal_roles_update_8004() method uses the following code to migrate the data :

    $user = $user_storage->load($user_id);
    $drupal_roles = $user->get(‘authorization_drupal_roles_roles’);

The $user->get('authorization_drupal_roles_roles') cannot work properly because the metadata for the authorization_drupal_roles_roles field is no longer available due to the deletion of the hook_entity_base_field_info. This causes the update to fail with the error 'Field authorization_drupal_roles_roles is unknown.'

@sunlix suggests re-adding the hook_entity_base_field_info in order to play authorization_drupal_roles_update_8004().

This works for data migration. However, authorization_drupal_roles_update_8004() uninstalls the storage of the authorization_drupal_roles_roles field so, at the end of the process, we have an inconsistency authorization_drupal_roles_roles field defintion:
- the field's metadata are present because the hook_entity_base_field_info is present.
- the field storage no longer exists because the update authorization_drupal_roles_update_8004() has deleted it.

To ensure that data migration works properly and that the authorization_drupal_roles_roles field is completely deleted, we propose to modify the way to retrieved original data in authorization_drupal_roles_update_8004() : we use a "direct database query" instead of the user object loaded.

This isn't very conventional, but it seems to solve all the problems.

What do you think?

Aurélien.

@aurm

The update hook also uses to entityFieldUpdateManager to uninstall the field storage.
I think this part of the update hook have to altered to the database connection.

I think the current MR fixes the bug. I will merge and make a new release after it is confirmed to fix the bug.

https://git.drupalcode.org/project/authorization/-/merge_requests/32.diff

Hi bluegeek9,

I've put myself in a position to try the patch: Drupal with LDAP users in 1.3.0 which is then upgraded to 1.4.0

If I apply the patch to the module in version 1.4.0 composer refuses to apply it.

On version 1.3.0 the patch goes in, but I do not have the migration that is present in 1.4.0.

Installing drupal/authorization:1.x-dev@dev instead and applying the patch successfully migrates.

admin@5300ce0b0887:/var/www/html/xxx$ drush updb
 ------------- ----------- --------------- ------------------------------------
  Module        Update ID   Type            Description
 ------------- ----------- --------------- ------------------------------------
  authorizati   8004        hook_update_n   8004 - Migrate roles from field
  on_drupal_r                               'authorization_drupal_roles_roles'
  oles                                      to user data.
 ------------- ----------- --------------- ------------------------------------


 ┌ Do you wish to run the specified pending updates? ───────────┐
 │ Yes                                                          │
 └──────────────────────────────────────────────────────────────┘

>  [notice] Update started: authorization_drupal_roles_update_8004
>  [notice] Update completed: authorization_drupal_roles_update_8004
 [success] Finished performing updates.

I then did some login tests and there don't seem to be any problems.
I therefore believe that the patch solves the problem.

Hi uccio,

Thank you for testing.

I completed the MR. With the fix will be in dev more people might test.

I want to make a new release soon.

I tested the dev release with copies of existing sites and was able to log in after running the update. I can't speak for everyone, but it worked for me, so +1 vote for RTBC.

Thanks!

This was fixed in 8.x-1.5, released November 7th.