Improve the way batch is being implemented on import and export

Created on 25 June 2024, 5 months ago

Updated 15 July 2024, 4 months ago

Problem/Motivation

Batch is currently processing all the nodes on the finishCallback function.
It works fine when exporting a lot of nodes or a whole content type, but the problem is when we import all those contents.

Since the export creates a single file with all nodes, there is a heavy load on memory usage.

Steps to reproduce

Export all nodes from specific content type that have a lot of contents (I got my local environment frozen with 200 nodes)
Import the contents (as new nodes)

Proposed resolution

Batch job should have the operations for each chunk of nodes, maybe chunks of 5 or 10 nodes (without considering translations)
Multiple files should be exported, instead a single file, so there is no heavy load when loading the json file.

📌 Task

Status

Active

Version

1.0

Component

Node export

Created by

🇧🇷Brazil geovanni.conti

Live updates comments and jobs are added and updated live.

Comments & Activities

Issue created by @geovanni.conti
Comment 4 months ago →
🇧🇷Brazil geovanni.conti
I did some implementation here, but it ended up dependent on https://www.drupal.org/project/node_export/issues/3456750 ✨ Import/Export node and its translations Active issue.

This is a patch with both implementations

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024