Recursion takes too much memory compared to earlier versions (1.4.6 or earlier)

Digging further, the module collects same entity multiple times so looks like it is cyclic until it runs out of memory or times out.

 array:2 [▼
  0 => "taxonomy_term"
  1 => "1336"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "439"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "1336"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "439"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "1336"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "439"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "1336"
]
 array:2 [▼
  0 => "taxonomy_term"
  1 => "439"
]

After further investigation I found versions 1.4.7+ are unusable for complex websites with a lot of references.

The issues discovered:

• Even if taxonomy terms are UNselected in config, the module still exports taxonomies
• Taxonomies go into infinite loop, because of $this->exporter->doExportToArray($entity->get('parent')->entity) in Drupal\single_content_sync\Plugin\SingleContentSyncBaseFieldsProcessor\TaxonomyTerm::exportBaseValues()
• Instead of single content it is now exports all the content linked with no mechanism to stop it so it can result in hundreds of nodes instead of one.

that's not a critical issue, it's easy to fix, expect a new patch in a few minutes

well I don't know what is your exact structure, so it's hard to test it for me as I can't reproduce it. In any case, I provided a MR that might improve the export. Do you export node with a taxonomy reference or you export a taxonomy entity itself? Please provide mode details of your content structure, so I can replicate it at least

Nope, getting

The website encountered an unexpected error. Try again later.

Error: Xdebug has detected a possible infinite loop, and aborted your script with a stack depth of '512' frames in Drupal\Core\TypedData\DataDefinition->getDataType() (line 54 of core/lib/Drupal/Core/TypedData/DataDefinition.php).

I asked you a few questions and received 0 answers, can you please provide any information about your content structure. If the issue is not reproducible out of the box, this cannot be critical

Thanks @nginex. Production data is too complex to export.
Let me setup clear instance and try to replicate it.
From what I investigated, entities that are referenced in text fields are being exported as well (both taxonomies and nodes).

Correct, that feature did not exist in 1.4.6. Probably it makes sense to change the logic of export there. Let me update the patch and try to optimize that moment (instead of exporting full entity I will only export stub version of it)

I provided a few new options of exporting for menu links of content and embedded links from formatted text. By default it will be Stub mode instead of Full one. Also it's possible to select None.

Let me know if new patch works for you. Don't forget to run drush updb.

Was able to export the zip with the patch suggested. However, currently unable to import zip with all assets - Bad Gateway

An AJAX HTTP error occurred.
HTTP Result Code: 502
Debugging information follows.
Path: /batch?id=6746686&op=do_nojs&op=do
StatusText: error
ResponseText:
502 Bad Gateway
502 Bad Gateway
nginx

My observation is that the function validateYamlFileContent() is causing Bad Gateway when trying to decode the YML file

$content = Yaml::decode($content); in /single_content_sync/src/ContentSyncHelper.php

If anybody has faced similar issue, please advise

would it be possible to share the file you exported? I would like to review it and see what's wrong with it.

I believe this is related to commit 72ae530b821671faf1db0931fead72a564a27d27 which was made to address issue #3491573 ✨ Export entities fron link field with uri entity:node/nid Active .

Instead of just have "uri: 'entity:node/xxx'", now we also have the "linked_entity" property with all related content.

For example, i've a node type "page" with lots of paragraphs inside. Inside one of them i've a field "link" related to another node "page". Instead having only this relation with the id, now i've have all contents of the second page and.. if this second have another link, and maybe it's enabled to be inside a menu, i have also all menu parents content in the export.

However, it's not important have a link inside the first page, it occours also if the first page provide a menu item (menu_link property), so all parents are exported.

The same content exported with different version of single_content_sync module could have a very different yml file sizes:

• v1.4.6 -> 227KB
• v1.4.7 -> 36.529KB
• v1.4.9 -> 37.046KB
• v1.4.11 -> 41.669KB

In this scenario i tried to export a "page" and the export starting from 1.4.7 exports about 75 pages with all contents.
In the first level i have "base_fields" at row 5 of the .yml, inside it i have "menu_link" at row 14 and "moderation_state" at row 190888. So i have 190k rows of "other pages content". In the 1.4.6 version (exporting the same content) the "menu_link" property that starts at row 14, ends at row 3254 because there isn't "linked_entity" prop.

So it's difficult to import in some circumstances (or need long processing times triggering a bad gateway in many environments). Maybe adding "linked_entity" could be an optional option of the export.

Ps. I'm not telling that it's wrong, i'm telling that after "linked_entity" property maybe it could be difficult import contents

Hope it helps!

I believe this is related to commit 72ae530b821671faf1db0931fead72a564a27d27, which was intended to address issue #3491573 ✨ Export entities fron link field with uri entity:node/nid Active .

When a field linked to an entity uses a "linked_entity" structure instead of simply "uri": "entity:node/xxx", it pulls in all of the related content. If one of those related items happens to be a menu_link_content, the export will recursively include all of its parent menu links. As a result, the same content exported under different module versions can vary dramatically in size:

v1.4.6 → 227 KB

v1.4.7 → 36.500 KB

v1.4.9 → 37.000 KB

v1.4.11 → 41.700 KB

v1.4.11 patch 154 stubbed both options → 12.000 KB (same with no menu link or no embedded link, there are about 300KB of difference, it's minimal)

This excessive payload can make imports very slow or even trigger “502 Bad Gateway” errors in some environments.
Another issue is that after this update, i think that if the page provides a menu link, all menu parents content are exported. Maybe with the last patch in comment #13 it's better, but there is more data (v1.4.6 227KB -> v1.4.11 patched 12MB instead of v1.4.11 37MB for the same content).

I'll try a downgrade to v1.4.6 to mitigate our production issues.
Hope this information is helpful!