Contrib.social

Exports / imports should only update new / modified content

Open on Drupal.org →
Created on 16 October 2023, about 1 year ago
Updated 8 July 2024, 5 months ago

Problem/Motivation

Currently, manual imports/exports, with the changes done in this issue → , works fine. However, there exists a point that can improve the performance: all content is imported/exported although there aren't changes. This means that the first process of import/export has a reasonable execution time, but the next ones could spend more time than needed. In some cases, it may save just seconds, but in large projects with a large amount of default content, it may save a big amount of time.

I've preferred opening a new issue instead of continuing in the original one → so that:

  • The original issue is finished without needing to check additional changes
  • Although the proposed changes will make the default content work faster, probably it exceeds the scope of the module is done for (something I don't know and I would like to confirm).

In case you see this feature useful I would propose to postpone this issue until https://www.drupal.org/node/2640734 → is solved, and setting it as a 'Feature request' instead of a support request.

Proposed resolution

1. With a table that registers a row for every exported/imported content it is possible to detect when a content is changed, without needing to load the entities. This table would contain:

- Module
- FOlder
- Entity ID
- Entity type
- Entity uuid
- Hash: this contains a hashed version of the imported/exported file and it is the most important table property.

2. DUring import, only import when the hash has changed, or it does not exist in the table. After importing an entity, a registry is added or updated in the table with the content hash.

3. When an entity is updated and it has a registry in the table with the same uuid, update the table with the new hash.

4. When an entity is deleted, delete the registry using the uuid if exists.

5. Create a new export command that receives the module name and updates the content with changes (checking the hash between the database and the exported YAML file is different). This command would iterate over the table rows matching the module name and exporting them to the module content folder.

💬 Support request
Status

Needs review

Version

2.0

Component

Code

Created by

🇪🇸Spain omarlopesino

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @omarlopesino
  • Status changed to Needs review about 1 year ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 8.1 & MySQL 5.7
    last update about 1 year ago
    Patch Failed to Apply
  • Comment about 1 year ago →
    🇪🇸Spain omarlopesino

    Attaching a patch, that must be added with the changes of this issue → , that applies the proposed resolution. Please review, thanks!

  • Comment 7 months ago →
    🇧🇯Benin delacosta456

    hi
    thanks for the logic of this patch. while this patch is correctly being applied on D10.2+php8.2
    the on on this issue → doesn't apply

    can someone please help for this

    Thanks

  • Comment 5 months ago →
    🇪🇸Spain Juanjol Navarra

    I'm getting deprecation warnings with patch from #1:

    PHP Deprecated:  Optional parameter $link_domain declared before required parameter $database is implicitly treated as a required parameter in /var/www/html/web/modules/contrib/default_content/src/Importer.php on line 169

    I have updated it moving optional Importer service arguments to the end.

  • Comment 5 months ago →
    🇪🇸Spain Juanjol Navarra

    The previous patch does not work properly, it is badly generated. I have generated a new corrected version and now it seems to work as it should, correcting the warnings mentioned above.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024