Try to provide an upgrade path from 2.x.x to 3.x.x

Created on 11 April 2024, 9 months ago
Updated 19 September 2024, 4 months ago

Problem/Motivation

Because of the mismatch between entity type machine names in v2 and v3, there is no upgrade path. Let's see if there isn't a way to write an update hook that takes care of this after all.

Steps to reproduce

Proposed resolution

From a Slack conversation with @catch:

  1. Rename group_content__field_foo to group_relationship__field_foo , the column names seem unchanged
  2. Somehow get this info into the entity/field definition update manager, even if we bypassed it in step 1
  3. Update field storage entities that reflect the above, perhaps through config API rather than Entity API
  4. Loop over all ER fields and update their definition and tables to no longer point to group_content
  5. Loop over all Views?

We already renamed the base and data tables to be consistent across group 2 and 3. Although I don't think I ever updated the Views to account for the new table names.

Remaining tasks

  1. Create a fixture coming from v2 of a site with at least one custom field, one custom field with a really long machine name, a View, an entity reference to a membership.
  2. Write all of the necessary update hooks.

User interface changes

API changes

Could finally sunset Group 2 \o/

Data model changes

πŸ“Œ Task
Status

Fixed

Version

3.3

Component

Code

Created by

πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @kristiaanvandeneynde
  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Added a fixture, but could not get a garbled DB table name in it (longer than 48 characters becomes a hash), sadly enough. We'll have to make sure we call DefaultTableMapping::generateFieldTableName() with the old and new field storage to know which tables to rename to what.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    The MR only works in the browser when using the fix from πŸ› Allow update.php to load when entity type info needs to be updated RTBC .

    Still need to update all group content types to group relationship types now.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Starting to look really nice, think I just need to get the field instances correct now.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Looks like it's actually working :o

    It was failing on update.php because I forgot to drop tall tables before retrying. Gonna expand the tests now and see if I can make sure they work well too. Had some failures before.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    EntityFieldManager::getFieldMap() still doesn't know about the field_config entities we moved, so that needs fixing.

  • Is there any plan on when the upgrade path from 2 to 3 will be completed? Very much looking forward to this feat

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Waiting for 10.3 to come out with the fix to the update.php page, which is now RTBC here and should get committed next week. πŸ› Allow update.php to load when entity type info needs to be updated RTBC

    Having said that, still need confirmation from some people who have tried this on their sites.

  • πŸ‡ΊπŸ‡ΈUnited States shelane

    I would love to test this, but I want to make sure I'm doing to right path for testing. Let me know if I'm on track here.

    1. Update my local dev environment for a site using Group 2.2.2 with the latest dev version of 3.3.x.
    2. Download plain diff to apply patch with composer
    3. Run drush updb
    4. Test various users and group content actions
  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Re #10: Yup that's exactly it. You make sure your 2.x.x is on the latest version and then change your code to the latest 3.3.x with this MR applied. Also make sure you check out the 3.x.x-compatible code for any Group related modules you have installed.

    So far I've heard that this breaks with the Subgroup (Graph) β†’ module because they changed their plugin machine names between v2 and v3 and so that module would also need an upgrade path. Any other module such as Subgroup, Group Sites, etc. should theoretically be fine.

  • πŸ‡©πŸ‡ͺGermany dbielke1986

    I would also be interested in testing this upgrade path.
    We will do some tests as soon as Drupal 10.3 is released, and you give us the go to test.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    It's ready to test now. You don't need D10.3 if you use drush updb

  • πŸ‡ΊπŸ‡ΈUnited States shelane

    The patch does not apply against 3.2.2 (Could not apply patch! Skipping. The error was: Cannot apply patch ./patches/group.patch) and I'm unable to update to the dev of 3.3.x because of this:

    Your requirements could not be resolved to an installable set of packages.

    Problem 1
    - drupal/group dev-3.3.x requires drupal/core ^10.3 -> found drupal/core[10.3.0-beta1, 10.3.x-dev, 10.4.x-dev] but these were not loaded, likely because it conflicts with another require.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Could you try checking out the latest Group 3.3.x using git? If you use composer you'll have to specify the -W flag to also download all dependencies, including Drupal 10.3-rc1

  • πŸ‡§πŸ‡ͺBelgium gorkagr

    Hi!

    As mentioned in slack.
    Performed:

    ddev composer require 'drupal/group:3.3.x-dev@dev'
    ddev composer require 'drupal/subgroup:^3.0' --no-update
    added the patch https://git.drupalcode.org/project/group/-/merge_requests/151.diff
    ddev composer update
    drush updb
    

    then a fiend & replace in all my code from group_content to group_relationship

    all seems good.

    I have only experienced 2 issues I have noticed today.

    There was a pathauto alias that was broken: pathauto.pattern.group_content.yml
    and also a core.entity_view_mode.group_content.token.yml

    Then, i have no clue if it is just my DB, but when running cron() i have this log message:

    Drupal\Component\Plugin\Exception\PluginNotFoundException: The "group_content" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 142 of /var/www/html/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php).
    

    but there is no trace of such string in my database, so i have no clue from where the error is coming from

  • πŸ‡§πŸ‡ͺBelgium gorkagr

    I am not sure if it is on mine or if i did smth wrong (as i messed with my localhost a bit)

    but it seems i have some DB tables named after group_content (nstead of group_relationship) in config_export table
    and also a few entries in key_value with group_content_type keyword

    I will continue testing this

  • πŸ‡¬πŸ‡§United Kingdom joachim

    I can confirm #17.

    I ran the test src/Functional/Update/Group2to3UpdateTest.php --filter=testEntityReferenceFields (just because it seemed the simplest and quickest), with the patch from ✨ add a way to manually use the test site from a Functional test Needs review applied to core, so that the DB is preserved after the test.

    I found these rows in the {key_value} table of the test DB after the test:

    config.entity.key_store.group_content_type	uuid:000b9028-9efe-4a4f-9552-4ed2343481d5	a:1:{i:0;s:40:"group.content_type.class-group_node-page";}
    config.entity.key_store.group_content_type	uuid:1891ac47-eae0-4075-b85d-38c7ca8b5122	a:1:{i:0;s:41:"group.content_type.class-group_membership";}
    config.entity.key_store.group_relationship_type	uuid:000b9028-9efe-4a4f-9552-4ed2343481d5	a:1:{i:0;s:45:"group.relationship_type.class-group_node-page";}
    config.entity.key_store.group_relationship_type	uuid:1891ac47-eae0-4075-b85d-38c7ca8b5122	a:1:{i:0;s:46:"group.relationship_type.class-group_membership";}
    

    These values are used by Drupal\Core\Config\Entity\Query\QueryFactory::getConfigKeyStore() for fast lookups of config entities.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    This issue has been open for almost half a year and the little feedback we got has been addressed. Going to commit.

  • Pipeline finished with Skipped
    4 months ago
    #282193
  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium
  • Status changed to Fixed 4 months ago
  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium
  • heddn Nicaragua

    I never ran group 2.x and when updating to the latest 3.3.x today, I ran into issues with the three v2 => v3 upgrade hooks. The first one, group_update_10300 doesn't not seem to account for the possibility that no group_relationship config entities would exist (if no v2 was ever installed). Should we roll a new issue for this or call for a quick revert on this with more work to make this handle the non-upgrade path?

    >  [notice] Update started: group_update_10300
    >  [error]  Call to a member function set() on null 
    >  [error]  Update failed: group_update_10300 
     [error]  Update aborted by: group_update_10300 
    
  • Status changed to Needs work 4 months ago
  • πŸ‡¦πŸ‡ΊAustralia realityloop

    I've just hit the same errors as @heddn

  • Status changed to Fixed 4 months ago
  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Just pushed a simple fix, please verify.

  • heddn Nicaragua

    Fixes the issue. Thank you

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Thanks for verifying!

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024