Implement a migration path

Created on 15 August 2018, over 6 years ago
Updated 10 March 2023, over 1 year ago

From README.txt:
"It is reported that if you change your translation configuration on an existing site in order to start using this module, you might experience soft data loss.
More specifically, paragraph entities whose hosting fields get set from non translatable to translatable seem to get unlinked from their hosting entities.
This means that they are still in the database (since, until now, paragraphs module doesn't have garbage collection routines) but they are not rendered."
TODO: Implement a "migration path".

Feature request
Status

Needs review

Version

1.0

Component

Code

Created by

🇫🇮Finland merilainen

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Merge request !4Issue #2992777: Implement a migration path → (Open) created by phoang
  • 🇧🇪Belgium dieterholvoet Brussels

    Rebased the MR and processed the feedback from #51.

  • 🇳🇱Netherlands undersound3

    @DieterHolvoet thanks for this.

    I am testing this currently and I am noticing the following behaviour when applying https://git.drupalcode.org/project/paragraphs_asymmetric_translation_wid... to the 1.1 version of this module.

    I have a site with two languages NL (default) and EN.
    After running the migration via drush cim NL content is showing up in the the paragraphs on the english version of the node. I am expecting to see EN content here.

    When applying https://www.drupal.org/files/issues/2022-12-23/2992777-paragraphs_asymme... I am not experiencing this behaviour and things are working correctly. At least on the "frontend" part of things as I understand from your comment that new paragraphs/content was saved differently then it did until your fixes.

    It seems is has something to do with this elseif statement because bringing that back in shows english content correctly within paragraphs on an english node version.

            elseif ($paragraphs_entity->hasTranslation($langcode)) {
              // If host entity translation refers to same paragraph entity, then
              // fetch that instead. This is possible if Classic widget has been
              // changed to Asymmetric widget with existing content.
              $paragraphs_entity = $paragraphs_entity->getTranslation($langcode);
            }
    

    Also I am wondering how to verify if this new functionality https://git.drupalcode.org/project/paragraphs_asymmetric_translation_wid... is working as it should. Should the default_langcode of the, in my case english version of the paragraph, be set to 1?

  • 🇧🇪Belgium dieterholvoet Brussels
    I have a site with two languages NL (default) and EN.
    After running the migration via drush cim NL content is showing up in the the paragraphs on the english version of the node. I am expecting to see EN content here.

    I'll try to test this again on a project of mine in the coming weeks.

    Also I am wondering how to verify if this new functionality https://git.drupalcode.org/project/paragraphs_asymmetric_translation_wid... is working as it should. Should the default_langcode of the, in my case english version of the paragraph, be set to 1?

    The default_langcode on all paragraphs should be 1, since we're not actually translating paragraphs anymore. Every paragraph is a separate entity with a separate ID.

  • Status changed to Needs work over 1 year ago
  • 🇷🇺Russia qzmenko Novosibirsk

    I have a site with two languages NL (default) and EN.
    After running the migration via drush cim NL content is showing up in the the paragraphs on the english version of the node. I am expecting to see EN content here.

    Yes, I confirm that problem. The content of the paragraphs of the original node was copied, instead of the translated paragraphs.

  • 🇨🇦Canada geoanders Nova Scotia 🍁

    @qzmenko

    Experiencing the same problem. Looks like when it creates the duplicate paragraph, it's using the source paragraph and not the translated.

    Issue can be found in migrateEntity method.

  • 🇷🇺Russia qzmenko Novosibirsk

    Please test this patch.
    I created it March, but I forgot to post it here. Sorry I don't have possibility to add interdiff at the moment.

  • 🇨🇦Canada geoanders Nova Scotia 🍁

    @qzmenko

    Thanks for the patch.

    I think our solutions are basically the same, but seems to work much better now.

  • Status changed to Needs review over 1 year ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    2 fail
  • 🇷🇺Russia qzmenko Novosibirsk
  • The last submitted patch, 58: 2992777-paragraphs_asymmetric_translation_widgets-58.patch, failed testing. View results
    - codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

  • Status changed to Needs work over 1 year ago
  • 🇨🇦Canada geoanders Nova Scotia 🍁

    I wonder if this should be a sub-module, as I would imagine this would probably be a one time use anyways. Then the user could just disable the module after their done with it. Just some food for thought.

  • 🇧🇪Belgium dieterholvoet Brussels

    Please don't mix patches and MR's. An MR was already open, if you have anything to contribute, commit it there.

  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    2 fail
  • 🇧🇪Belgium dieterholvoet Brussels

    I added the fix as suggested in #57. This probably needs tests if we want to make sure it works consistently.

  • 🇫🇷France raphaeltbm

    Thanks for all the contributions! The MR looks already nice!
    But the drush command is missing some `bundle` context (VS the BO batch one which have one). Let said i have the same paragraph field across 2 node bundles, but only one is decided to be translatable, if you try to run the drush command `paragraphs:migrate-to-non-symmetrical node field_my_paragraphs`, things will be ugly for the content bundle with the not translatable one. Or, am i missing some info about the usage?

  • 🇫🇷France raphaeltbm

    During my tests with the last MR. Let said i have a page node with 2 languages FR (original language) & DE. existing before the migration path.
    If you delete manually the DE translation, the paragraph data is completely lost on the FR node too.

  • 🇳🇿New Zealand stewest Wellington

    Hi, I can confirm #66 too, all data lost on original node (when another translation is deleted)

  • 🇧🇪Belgium hoebekewim

    I do not have the issues described by raphaeltbm or stewest, are you sure you are changing all the applicable fields to translatable and then looking at the end result ? Also make sure you apply it to all nested paragraphs. It should work fine. If you export your config, you can import it on another environment and there it will apply the conversion again. You should not use manual drush commands on this patch.

  • 🇧🇪Belgium hoebekewim

    I encountered an issue in the conversion.
    I have a setup with a nested paragraph, so it contains a paragraph field inside a paragraph. When you want to edit the highest level paragraph, and the summary is visible, you see another language on the summary of the lower level paragraphs. But as soon as you expand the sub paragraph, you see the correct language.

    I was able to pin-point it to the way languages are handled for summary fields inside the paragraphs module itself, and I've created a patch for this. But this is however a workaround and not a real solution, the real solution should be provided here inside the migration script.
    I assume the issue is related to the duplicate content that exists using this specific scenario, which I described here, as a new ticket, as it is not directly part of this migration ticket. https://www.drupal.org/project/paragraphs_asymmetric_translation_widgets...

    The issue is related to the fact that this patch does not remove duplicates, nor does it correct all langcodes inside all paragraph and content tables, making the wrong content appear.

  • 🇳🇱Netherlands undersound3

    Does anybody know if there is a problem to be expected when switching from Paragraphs (legacy) to Paragraphs (stable) widget for now.
    We want to migrate later on by using the patch/functionality from this issue but for now sync all paragraph field widgets for consistency sake.

    Also I have read that support for the legacy widget will end eventually. See https://www.drupal.org/project/paragraphs/issues/3331203#comment-15212442 🐛 Core 9.5.0 causing broken-looking styling (Legacy widget, with Seven) Fixed

  • chinataun changed the visibility of the branch 2992777-implement-a-migration to hidden.

  • chinataun changed the visibility of the branch 2992777-implement-a-migration to active.

  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update 12 months ago
    2 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update 12 months ago
    2 fail
  • Regarding the previous commit...

    Using this functionality in our project we have encountered a problem that could appear in other developments.
    Our case:
    We have a type of content in which we have a paragraph with different fields. Some of them are translatable but some others were not marked as such, for example a field_image. Before applying paragraph async, both the original node and the translations have that field_image, obviously. The problem was that when we applied the patch the translations lost that image (it may seem logical since it is not marked as translatable) but we were interested in keeping it. We could have developed a hook that updated all the nodes by making those fields translatable, saving the node and then applying the patch.
    It may be interesting that in this type of cases and in the duplication process, said old content information is not lost and in the content migration those empty fields can be filled with the fields of the original node if it has them.

    It has been committed to the MR and if you find it interesting we will keep it. If not, and it seems to you that it may be a case specific to a specific project, we will remove it.

  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update 12 months ago
    2 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.4 & MySQL 5.7
    last update 12 months ago
    2 fail
  • Regarding the problem reported by @raphaeltbm and confirmed by @stewest on #66 we have also been able to replicate it.
    The problem is that in the delete hook the entity is being deleted and not the translation due, among other things, to the problem with data duplication exposed in https://www.drupal.org/project/paragraphs_asymmetric_translation_widgets... .

    With the following image that shows data from the paragraphs_item_field_data table, we thoroughly explain the possible cases:
    Yellow: Existing pre-migration paragraphs (original EN, DE and PT-PT translations)
    Red: Once the migration process has been executed.
    Green: Added a new translation of an existing node.
    Blue: Added a new paragraph

    The problem, as they indicated, is that if we delete a translation of the old ones (de, pt-pt) all its references are eliminated, including the original node.
    For example, if we want to delete DE, in the delete hook it collects the following nodes: 8,9,10,67,68,69, 73,74,75 and deletes the entire entity by doing paragraph->delete().

    As a first approximation and until the problem of duplicates is fixed, we can differentiate two cases: if it is a new translation where the language of the paragraph and the langcode of the entity match, we can eliminate the entire entity (red case). In any other case and whenever a translation exists, we will eliminate the translation of the paragraph (yellow, green, blue).

  • 🇩🇪Germany stillworks

    Can someone please explain how to run the migrate? With drush and MigrateParagraphsCommand.php I do get command not defined with the latest contrib on diff4.

    Reimporting the config doesn't work if it hasn't change, right.

    I could uncheck "Users may translate that field" and recheck it, the batch process starts but gives me a memory exhaust. But that is currently my problem to solve.

  • 🇨🇦Canada mastap

    I want to thank everyone working on this migration path. This is really needed.

    What is the maturity of this road and implementation?
    What is the current expectations - in Summary. What works and what don't.

    We're stuck with content in English, translated to French. But enabling Asymmetric loses the FR connection we had.

    Thanks again!

Production build 0.71.5 2024