Taxonomy imports with relationships may hang

Adding likely-related issue report. Sorry - hadn't noticed that one before (wasn't searching for dead lock, doh!)

So - having been bitten by this again, I dug a little deeper. I can now confirm that this will happen on the "full" side as well and have created a patch that will reproduce the issue in your tests.

This can happen regardless of duplicates, specifically, when the order of parent term id's is not correctly sorted.

I did review the other referenced patch in 3378470 and we're seeing the same thing: the issue comes down cases such as this where you have to be careful not to add things to tidsLeft when they are already in tidsDone, which is exactly what happens with this test.

Adding patch that basically does the same as #3378470, but without causing as much diff noise to make the change more obvious.

Appears to be related to https://www.drupal.org/project/structure_sync/issues/3177106 ✨ Support terms with the same name in a tree structure and multi-parent terms Needs work .

Some investigation shows that the error reported in this ticket is the cause of the failing \Drupal\Tests\structure_sync\FunctionalJavascript\StructureSyncMenuLinksTest::testMenuLinksExportImportUsingAdmin() in the 2.x branch. The test times out, but looking at the the artifacts and manually testing shows the batch job seems to just hanging there and never completing on the front-end. Checking the database logs, there are errors.

Unfortunately, this patch doesn't fully fix the test, and I can confirm the batch job gets stuck when I perform the tests manually.

The errors I get in the log after applying this patch are...

1. • Channel: menu_link_content
   • Severity: Error
   • Location: /batch?_format=json&id=2&op=do
   • Referrer: /batch?id=2&op=start
   • Message: Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '48062c1c-132b-476e-bc3c-8511fb8896e4' for key 'menu_link_content_field__uuid__value': INSERT INTO "menu_link_content" ("revision_id", "bundle", "uuid", "langcode") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3); Array ( [:db_insert_placeholder_0] => [:db_insert_placeholder_1] => menu_link_content [:db_insert_placeholder_2] => 48062c1c-132b-476e-bc3c-8511fb8896e4 [:db_insert_placeholder_3] => en ) in Drupal\mysql\Driver\Database\mysql\ExceptionHandler->handleExecutionException() (line 43 of /app/web/core/modules/mysql/src/Driver/Database/mysql/ExceptionHandler.php)
2. • Channel: php
   • Severity: Error
   • Location: /batch?_format=json&id=2&op=do
   • Referrer: /batch?id=2&op=start
   • Message: Drupal\Core\Entity\EntityStorageException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '48062c1c-132b-476e-bc3c-8511fb8896e4' for key 'menu_link_content_field__uuid__value': INSERT INTO "menu_link_content" ("revision_id", "bundle", "uuid", "langcode") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3); Array ( [:db_insert_placeholder_0] => [:db_insert_placeholder_1] => menu_link_content [:db_insert_placeholder_2] => 48062c1c-132b-476e-bc3c-8511fb8896e4 [:db_insert_placeholder_3] => en ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->save() (line 817 of /app/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

I wasn't able to apply the patch in 🐛 Dead lock when importing terms. Active ; it has diverged too much from the latest changes to the 2.x branch.

I'm promoting this issue to "Major", as it:

1. Triggers a PHP error through [normal use of] the user interface
2. Renders one feature [menu link import] unusable with no workaround.
3. Breaks a regression test on 2.x HEAD

I'm going to mark the other issue(s) as a duplicate of this one: this patch is cleaner and easier to understand.

@wolcen I think a merge request would show test results better, but I'm happy to collaborate with patches as well. Do you have a preference? If not, or if you'd prefer a merge request, I can open one.

I seem to be running across this in a client project as well, where importing a vocabulary that has multiple "Other" terms across a handful of top-level topics. The import taxonomies process (we usually run via drush) seems to all work, but just hangs at the end. We also import a small handful of simple menu links into the "main" menu via Structure Sync.

Applying this patch appears to resolve the hanging at the end of import, and I *don't* see the errors @mparker17 notes from the DrupalCI. This would seem to indicate there's something distinct about the menu fixtures created in the test scenario, but I'm not sure what that would be.

@mparker17 - sorry for the lag in response here. By all means, a merge request or whatever you prefer would be great - feel free to take over on this!

Let me know if I can possibly clarify things from my perspective/if you have any questions for me.

I've not had the time recently to dedicate to it - the patch has also been working for our production use case where it had been hanging, but I'll try to get some time free to bring this up and re-run the tests and see if I can reproduce your results.

Awesome! I've created a merge request!

Seeing that fix on the AJAX CI failure (for which I am grateful - I'd had NO clue how to fix that!) I rebased to re-run CI here.

Confirmed that the merge request fixes this! Taxonomy import was running, i presume in circles, for more than an hour before i ended drush, added this merge request, and then it imported fine.

I was having the 'hang' issue too.
It ended up being missing parents.

The same condition needed to be fixed for more than one round, to avoid infinite loops because the terms left array was always having values.
Added a patch to review https://www.drupal.org/project/structure_sync/issues/3464919#comment-157... 🐛 Taxonomy imports with missing parent creates an infinite loop (hang) Needs review , It takes into consideration one than more level of nesting.