Update 8004 is failing

Created on 10 September 2024, 2 months ago
Updated 19 September 2024, 2 months ago

Problem/Motivation

We try to update our Drupal installations to Version 1.4 of authorization Module.
Currently we get this output from drush.

 ------------- ----------- --------------- ------------------------------------ 
  Module        Update ID   Type            Description                         
 ------------- ----------- --------------- ------------------------------------ 
  authorizati   8004        hook_update_n   8004 - Migrate roles from field     
  on_drupal_r                               'authorization_drupal_roles_roles'  
  oles                                      to user data.                       
 ------------- ----------- --------------- ------------------------------------ 


 // Do you wish to run the specified pending updates?: yes.                                                             

>  [notice] Update started: authorization_drupal_roles_update_8004
>  [error]  Field authorization_drupal_roles_roles is unknown. 
>  [error]  Update failed: authorization_drupal_roles_update_8004 
 [error]  Update aborted by: authorization_drupal_roles_update_8004 
 [error]  Finished performing updates. 
Our installation are running since a couple of years. So we don't know if the missing field should be present or not.
From our code base this module is a dependency by ldap.

Steps to reproduce

  1. Install authorization 1.3
  2. config LDAP connection
  3. config authorization profile
  4. drush cron
  5. update authorization to 1.4
  6. see update_hook failing
๐Ÿ› Bug report
Status

Active

Version

1.4

Component

Code

Created by

๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @sunlix
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel

    I have re-add the hook_entity_base_field_info to make the field authorization_drupal_roles_roles available again via Drupal::entityDefinitionUpdateManager.

  • Pipeline finished with Failed
    2 months ago
    Total: 208s
    #279158
  • Pipeline finished with Failed
    2 months ago
    #279162
  • First commit to issue fork.
  • Pipeline finished with Failed
    2 months ago
    Total: 149s
    #279239
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany JoCowood Kamp-Lintfort

    Hey, i am a colleague of sunlix. After we fixed the missing authorization_drupal_roles_entity_base_field_info hook, there was a follow up issue in the new update hook authorization_drupal_roles_update_8004.

    Because a do-while-loop is used, under some circumstances the loop is run through once too often. The finished variable is calculated like this: $finished = ($sandbox['current'] - 1) / $sandbox['total'];. This makes it appear as if the additional loop pass is intended. Unfortunately, the additional loop pass results in a user with the ID โ€˜NULLโ€™ being loaded.

    My commit converts the do-while-loop into a while-loop, fixes the calculation for the finished variable and adds an initialisation for the new variable. I think, now we are ready for a review.

  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel
  • ๐Ÿ‡ฎ๐Ÿ‡นItaly uccio Turin

    The solution proposed in fork 3473259 solves the problem for me!

    I applied the patch https://git.drupalcode.org/project/authorization/-/merge_requests/32.diff via composer without problems.

    Grazie!

  • ๐Ÿ‡ธ๐Ÿ‡ฆSaudi Arabia ishore

    We are experiencing the same (or very similar) issue:

    Do you wish to run the specified pending updates? (yes/no) [yes]:
    >

    > [notice] Update started: authorization_drupal_roles_update_8004
    > [notice] No profiles have revoke_provider_provisioned enabled. No roles to move to user data.
    > [warning] Undefined array key 1 Schema.php:580
    > [warning] Undefined array key 1 Schema.php:580
    > [warning] Undefined array key 1 Schema.php:580
    > [error] Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR: relation "field_deleted_data_21874036cb____idx" already exists: ALTER INDEX "public"."user__authorization_drupal_roles_roles__revision_id__idx" RENAME TO field_deleted_data_21874036cb____idx; Array
    > (
    > )
    >
    > [error] Update failed: authorization_drupal_roles_update_8004
    [error] Update aborted by: authorization_drupal_roles_update_8004
    [error] Finished performing updates.

    We are currently unable to login with any LDAP user account. When can we expect an official fix?

  • ๐Ÿ‡ธ๐Ÿ‡ฆSaudi Arabia ishore

    Not applied the patch yet, but now when I run drush updatedb again, I get the following:

    > [notice] Update started: authorization_drupal_roles_update_8004
    > [notice] No profiles have revoke_provider_provisioned enabled. No roles to move to user data.
    > [warning] Undefined array key 1 Schema.php:587
    > [error] Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR: relation "field_deleted_data_21874036cb____pkey" already exists: ALTER INDEX "public"."field_deleted_data_21874036cb____pkey" RENAME TO field_deleted_data_21874036cb____pkey; Array
    > (
    > )
    >
    > [error] Update failed: authorization_drupal_roles_update_8004
    [error] Update aborted by: authorization_drupal_roles_update_8004
    [error] Finished performing updates.

  • I have the same problem when executing the `drush updb` command. The problem occurs after upgrading drupal/authorization to version 1.4.0

    ...@...:/application$ drush updb -y
     ------------- ----------- --------------- ------------------------------------ 
      Module        Update ID   Type            Description                         
     ------------- ----------- --------------- ------------------------------------ 
      authorizati   8004        hook_update_n   8004 - Migrate roles from field     
      on_drupal_r                               'authorization_drupal_roles_roles'  
      oles                                      to user data.                       
     ------------- ----------- --------------- ------------------------------------ 
    
    
     // Do you wish to run the specified pending updates?: yes.                                                        
    
    [2024-09-16T13:46:39.304957+02:00] system.NOTICE: Module node has an entry in the system.schema key/value storage, but is not installed. <a href="https://www.drupal.org/node/3137656">More information about this error</a>. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
    >  [notice] Update started: authorization_drupal_roles_update_8004
    > [2024-09-16T13:46:50.536441+02:00] authorization_drupal_roles.NOTICE: No profiles have revoke_provider_provisioned enabled. No roles to move to user data. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
    >  [warning] Undefined array key 1 Schema.php:587
    >  [error]  Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR:  relation "field_deleted_data_21874036cb____pkey" already exists: ALTER INDEX "public"."field_deleted_data_21874036cb____pkey" RENAME TO field_deleted_data_21874036cb____pkey; Array
    > (
    > )
    >  
    >  [error]  Update failed: authorization_drupal_roles_update_8004 
    
  • I have the same problem when executing the `drush updb` command. The problem occurs after upgrading drupal/authorization to version 1.4.0

    ...@...:/application$ drush updb -y
     ------------- ----------- --------------- ------------------------------------ 
      Module        Update ID   Type            Description                         
     ------------- ----------- --------------- ------------------------------------ 
      authorizati   8004        hook_update_n   8004 - Migrate roles from field     
      on_drupal_r                               'authorization_drupal_roles_roles'  
      oles                                      to user data.                       
     ------------- ----------- --------------- ------------------------------------ 
    
    
     // Do you wish to run the specified pending updates?: yes.                                                        
    
    [2024-09-16T13:46:39.304957+02:00] system.NOTICE: Module node has an entry in the system.schema key/value storage, but is not installed. <a href="https://www.drupal.org/node/3137656">More information about this error</a>. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
    >  [notice] Update started: authorization_drupal_roles_update_8004
    > [2024-09-16T13:46:50.536441+02:00] authorization_drupal_roles.NOTICE: No profiles have revoke_provider_provisioned enabled. No roles to move to user data. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
    >  [warning] Undefined array key 1 Schema.php:587
    >  [error]  Exception thrown while performing a schema update. SQLSTATE[42P07]: Duplicate table: 7 ERROR:  relation "field_deleted_data_21874036cb____pkey" already exists: ALTER INDEX "public"."field_deleted_data_21874036cb____pkey" RENAME TO field_deleted_data_21874036cb____pkey; Array
    > (
    > )
    >  
    >  [error]  Update failed: authorization_drupal_roles_update_8004 
    
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bluegeek9
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bluegeek9

    Can you fill out the issue summary?

    authorization_drupal_roles_entity_base_field_infowas removed. Why are you adding it back?

  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel

    We have re-add authorization_drupal_roles_entity_base_field_info for the migration update hook 8004.
    In the update hook 8004 the base field is queried but it will not found due to missing metadata for the field API. (the database table is there, but not the field metadata).
    So to make the migration the module need to know the old base field to make the query and fetch the data for the migration.
    The module have to make the migration and in a later major version it can safely removed.

  • ๐Ÿ‡ธ๐Ÿ‡ฆSaudi Arabia ishore

    I'm now unable to login even as a local user. After what appears to be successful input of credentials on logging in, you are presented with a blank /user/login page. Navigating to another page shows that you are not logged in.

    Not sure why with each day, the situation is getting worse for us. All we are doing is running "drush updatedb".

  • ๐Ÿ‡ณ๐Ÿ‡ฑNetherlands cpdp

    I have three findings that I would like to share that seem to work for me as a workaround. YMMV, please try this out first in a test environment.

    These findings were made on a site that had unsuccesfully been updated to authorization 1.4.

    1. After I cancel and delete all LDAP-provided user accounts from my site, the 'authorization' update 8004 works (drush updb -y). For me this is a valid workaround, as in my case all users are provided via LDAP, and are recreated the next time a valid user logs in.

    2. If I manually set the 'authorisation' module version in composer.json back to to "drupal/authorization": "1.3", and run composer update, the login functionality is restored (for as far as I can see), without having to delete the user accounts as in step 1. Not sure what the effect of the failed update 8004 is in this case. This step requires you to manually change composer.json again when the 'authorization' module provides a valid 1.4 update path.

    3. Overwriting the unsuccesful update 1.4 with the dev version of [16 Sep 2024 at 21:45 CEST] did not provide a working update 8004 for me.

    Hoping for a resolution shortly,
    Mark.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bluegeek9

    @ishore,

    You are not using the revoke roles provisioned by authorization.

    No profiles have revoke_provider_provisioned enabled. No roles to move to user data.

    You could:

    1. Revert to 8.x-1.3
    2. Uninstall authorization_drupal_roles
    3. Upgrade to 8.x-1.4
    4. Re-install authorization_drupal_roles
  • ๐Ÿ‡ธ๐Ÿ‡ฆSaudi Arabia ishore

    Thanks @bluegeek. Actually, before you posted your suggestion, I had tried suggestion 1 from @cpdp. I was able do this because deleting all LDAP users wasn't a major headache because I am doing this a on a test instance.

    After deleting all LDAP users, I was able to run "drush updatedb" cleanly. However, I'm still having major issues that I will try and explain.

    I can only login once as the admin user. The login isn't clean - I am greeted by a blank page after entering the TFA code (we use the TFA module). However, going back to the front page shows that I am logged in. After that, when I try and login a second time, I just see a white screen, stuck on /user/login. However, clearing the cache on the browser allows me to login successfully again.

    Logging in as an LDAP user, I am just greeted by a blank white page, stuck on /user/login. When I go to the front page, I see the front page but it also says I am not logged in. However if I view recent log messages, I see that the LDAP user has been created. All subsequent attempts to login as that user fail, although clearly the user and password are accepted and recent log messages says "session opened for ".

  • ๐Ÿ‡ธ๐Ÿ‡ฆSaudi Arabia ishore

    Managed to get ldap user login working again by uninstalling authorization_group. We had version 2.0.x-dev@dev installed.

  • First commit to issue fork.
  • Pipeline finished with Failed
    about 1 month ago
    Total: 315s
    #303875
  • ๐Ÿ‡ซ๐Ÿ‡ทFrance aurm

    Hi everyone,

    The original problem is as follows:
    The hook_entity_base_field_info which brings the metadata of the field authorization_drupal_roles_roles is deleted.
    At the same time, the authorization_drupal_roles_update_8004() method uses the following code to migrate the data :

        $user = $user_storage->load($user_id);
        $drupal_roles = $user->get(โ€˜authorization_drupal_roles_rolesโ€™);
    

    The $user->get('authorization_drupal_roles_roles') cannot work properly because the metadata for the authorization_drupal_roles_roles field is no longer available due to the deletion of the hook_entity_base_field_info. This causes the update to fail with the error 'Field authorization_drupal_roles_roles is unknown.'

    @sunlix suggests re-adding the hook_entity_base_field_info in order to play authorization_drupal_roles_update_8004().

    This works for data migration. However, authorization_drupal_roles_update_8004() uninstalls the storage of the authorization_drupal_roles_roles field so, at the end of the process, we have an inconsistency authorization_drupal_roles_roles field defintion:
    - the field's metadata are present because the hook_entity_base_field_info is present.
    - the field storage no longer exists because the update authorization_drupal_roles_update_8004() has deleted it.

    To ensure that data migration works properly and that the authorization_drupal_roles_roles field is completely deleted, we propose to modify the way to retrieved original data in authorization_drupal_roles_update_8004() : we use a "direct database query" instead of the user object loaded.

    This isn't very conventional, but it seems to solve all the problems.

    What do you think?

    Aurรฉlien.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 403s
    #309944
  • Pipeline finished with Failed
    about 1 month ago
    Total: 519s
    #309970
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany sunlix Wesel

    @aurm

    The update hook also uses to entityFieldUpdateManager to uninstall the field storage.
    I think this part of the update hook have to altered to the database connection.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bluegeek9

    I think the current MR fixes the bug. I will merge and make a new release after it is confirmed to fix the bug.

    https://git.drupalcode.org/project/authorization/-/merge_requests/32.diff

  • ๐Ÿ‡ฎ๐Ÿ‡นItaly uccio Turin

    Hi bluegeek9,

    I've put myself in a position to try the patch: Drupal with LDAP users in 1.3.0 which is then upgraded to 1.4.0

    If I apply the patch to the module in version 1.4.0 composer refuses to apply it.

    On version 1.3.0 the patch goes in, but I do not have the migration that is present in 1.4.0.

    Installing drupal/authorization:1.x-dev@dev instead and applying the patch successfully migrates.

    admin@5300ce0b0887:/var/www/html/xxx$ drush updb
     ------------- ----------- --------------- ------------------------------------
      Module        Update ID   Type            Description
     ------------- ----------- --------------- ------------------------------------
      authorizati   8004        hook_update_n   8004 - Migrate roles from field
      on_drupal_r                               'authorization_drupal_roles_roles'
      oles                                      to user data.
     ------------- ----------- --------------- ------------------------------------
    
    
     โ”Œ Do you wish to run the specified pending updates? โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
     โ”‚ Yes                                                          โ”‚
     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
    
    >  [notice] Update started: authorization_drupal_roles_update_8004
    >  [notice] Update completed: authorization_drupal_roles_update_8004
     [success] Finished performing updates.
    
    

    I then did some login tests and there don't seem to be any problems.
    I therefore believe that the patch solves the problem.

  • Pipeline finished with Skipped
    16 days ago
    #331183
  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bluegeek9

    Hi uccio,

    Thank you for testing.

    I completed the MR. With the fix will be in dev more people might test.

    I want to make a new release soon.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States micahw156

    I tested the dev release with copies of existing sites and was able to log in after running the update. I can't speak for everyone, but it worked for me, so +1 vote for RTBC.

    Thanks!

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States bluegeek9
Production build 0.71.5 2024