- Issue created by @fishfree
- Status changed to Postponed: needs info
4 months ago 11:00am 10 September 2024 - 🇷🇴Romania claudiu.cristea Arad 🇷🇴
Also, could you, please, enter a proper title for this issue and not paste the error message. Thanks
- Status changed to Fixed
4 months ago 7:13am 12 September 2024 - 🇷🇴Romania claudiu.cristea Arad 🇷🇴
I tried manually batch write all my mapped nodes into Virtuoso as below:
This happens when one ore more nodes don't have a corresponding record in the
rdf_sync_uri
table (or for some reason the uri column is empty). Check and fix those cases - Status changed to Needs work
4 months ago 7:16am 13 September 2024 - 🇷🇴Romania claudiu.cristea Arad 🇷🇴
I have decided to give a 2nd look at this. I think it would be correct to handle the case when, accidentally, some entities URIs were lost and prevent any ugly error.
-
claudiu.cristea →
committed bbfae774 on 1.x
Issue #3472816 by claudiu.cristea, fishfree: "Return value must be of...
-
claudiu.cristea →
committed bbfae774 on 1.x
- Status changed to Fixed
4 months ago 7:23am 13 September 2024 - 🇨🇳China fishfree
@claudiu Thank you! After apply your patch, my error turned into:
In ProcessBase.php line 171: Unable to decode output into JSON: Syntax error Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or empty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Re source.php:69 Stack trace: #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(61): EasyRdf\Resource->__construct() #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(133): Drupal\rdf_sync\Normalizer\RdfSyncNormalizer->normalize() #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(104): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize() #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct() #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate() #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func() #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch() #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate() #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate() #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate() #10 [internal function]: Drush\Boot\DrupalBoot8->terminate() #11 {main} thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69
- Status changed to Needs work
4 months ago 6:59am 19 September 2024 - Status changed to Needs review
4 months ago 10:43am 19 September 2024 - 🇷🇴Romania claudiu.cristea Arad 🇷🇴
@fishfree, could you, please, test the latest MR?
- 🇨🇳China fishfree
@claudiucristea Thank you very much! Would you pls share a single patch file against the version https://www.drupal.org/project/rdf_sync/releases/1.0.0-alpha12 → ? Sorry I'm not able to manually merge your multiple MRs.
- 🇷🇴Romania claudiu.cristea Arad 🇷🇴
You should just go to MR and append .diff to URL
- 🇨🇳China fishfree
@claudiucristea Thank you very much! After applying https://git.drupalcode.org/project/rdf_sync/-/merge_requests/25.diff based on https://www.drupal.org/project/rdf_sync/releases/1.0.0-alpha12 → , I re-run
vendor/bin/drush rdf_sync:synchronize node
It showed as below:> ...... > [notice] Synchronized 1200 out of 1965 entities > [notice] Synchronized 1250 out of 1965 entities > [notice] Synchronized 1300 out of 1965 entities > [notice] Synchronized 1350 out of 1965 entities > > In Resource.php line 69: > > $uri should be a string and cannot be null or empty > > > PHP Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or empty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php:69 > Stack trace: > #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(108): EasyRdf\Resource->__construct() > #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(122): Drupal\rdf_sync\Normalizer\RdfSyncNormalizer->normalize() > #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(88): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize() > #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct() > #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate() > #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func() > #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch() > #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate() > #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate() > #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate() > #10 [internal function]: Drush\Boot\DrupalBoot8->terminate() > #11 {main} > thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69 In ProcessBase.php line 171: Unable to decode output into JSON: Syntax error Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or em pty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php:69 Stack trace: #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(108) : EasyRdf\Resource->__construct() #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(122): Drupal\ rdf_sync\Normalizer\RdfSyncNormalizer->normalize() #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(88): Drupal\r df_sync\RdfSyncSynchronizer->doSynchronize() #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.p hp(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct() #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTermina te() #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispa tcher.php(111): call_user_func() #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\Event Dispatcher\ContainerAwareEventDispatcher->dispatch() #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Sy mfony\Component\HttpKernel\HttpKernel->terminate() #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddl eware\StackedHttpKernel->terminate() #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKe rnel->terminate() #10 [internal function]: Drush\Boot\DrupalBoot8->terminate() #11 {main} thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69
I checked in my Virtuoso, it seems all the 1965 entities had been synchonized. Why still the error occured?
- 🇬🇷Greece idimopoulos
Hello @fishfree
From what I see, it seems that you might not have a URI for all entities. Probably you can check directly by running a quick query like
select * from node as n left join rdf_sync_uri as r on n.nid = r.entity_id where r.uri is NULL and n.type = 'my_bundle'
if we are talking about nodes and re run this for every bundle to see which IDs are empty.
Furthermore, I created the following script (works for me with nodes) to iterate through all entity types in order to check all of them for empty URIs. Sorry for all the container calls, it was created in haste. After running it with
drush php:script ./my-script.php
and you verify if I am correct, you can also uncomment the commented section designated, to also create URIs for the IDs that are missing them and update the entities. Please, note, that this will actually update the entities. If you do not wish them updated, adjust accordingly or manually force the values in therdf_sync_uri
table.<?php use Drupal\Core\Entity\ContentEntityInterface; $rdfMapper = \Drupal::getContainer()->get('rdf_sync.mapper'); $entityTypeManager = \Drupal::entityTypeManager(); $entityFieldManager = \Drupal::service('entity_field.manager'); $messages = []; foreach ($entityTypeManager->getDefinitions() as $entityTypeId => $entityType) { if (!$entityType->entityClassImplements(ContentEntityInterface::class) || !($bundleEntityTypeId = $entityType->getBundleEntityType())) { continue; } $bundleStorage = $entityTypeManager->getStorage($bundleEntityTypeId); foreach ($bundleStorage->loadMultiple() as $bundleId => $bundle) { if ($rdfSyncSettings = $bundle->getThirdPartySettings('rdf_sync')) { // Get the base table of the entity. $baseTable = $entityType->getBaseTable(); // Get the entity key for the ID. $idKey = $entityType->getKey('id'); // Get the bundle key. $bundleIdKey = $entityType->getKey('bundle'); // Load from the database IDs of the entity type that do not have do not // have a URI stored. Select IDs that in the `rdf_sync_uri` table, they // do not have a mapping to the columns `entity_type`, `entity_id`, and // `bundle`, or do, but the `uri` is empty. $query = \Drupal::database()->select($baseTable, 'e'); $query->leftJoin('rdf_sync_uri', 'u', "e.{$idKey} = u.entity_id"); $query->fields('e', [$idKey]); $query->condition("e.{$bundleIdKey}", $bundleId); $query->isNull('u.uri'); $entityIds = $query->execute()->fetchCol(); if ($entityIds) { $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, IDs with missing URIs: " . implode(', ', $entityIds); } // Uncomment below to also appoint new URIs. // foreach ($entityTypeManager->getStorage($entityTypeId)->loadMultiple($entityIds) as $entity) { // $pluginId = $rdfMapper->getRdfUriPluginId($entityTypeId, $bundleId); // $plugin = \Drupal::getContainer()->get('plugin.manager.rdf_sync.uri_generator')->createInstance($pluginId); // $uri = $plugin->setEntity($entity)->generate(); // $fieldName = \Drupal::getContainer()->get('rdf_sync.mapper')->getRdfUriFieldName(entity: $entity); // $entity->get($fieldName)->setValue($uri); // $entity->save(); // $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, ID: {$entity->id()}, New URI appointed: $uri"; // } } } } print implode("\n", $messages);
Further more, in https://www.drupal.org/project/rdf_sync/issues/3478179 ✨ Allow to check for the publication status before syncing Active , we are working on allowing a subscriber to tamper with the entities before syncing them. That means, that using this subscriber, and a simple check whether the entity does have a URI, you can either identify or block the entity from being synced.
Please, consider that, the error you are getting is based on the idea that having a bundle synced, all entities should have a URI. That is why if a bundle is mapped, the
getUri
should always return a value and we do not allow it to return null. That is because a URI is like a persistent identifier in RDF data and thus we thought we should be strict with it.I checked in my Virtuoso, it seems all the 1965 entities had been synchonized. Why still the error occured?
This might be that the URI was removed somewhere along the way or that Virtuoso already had the data before you sync. If you are using docker, and the container crashed, it doesn't mean data were deleted if the volume remained intact. I mean, we would need more information to identify further issues.
In terms of this ticket, I will review the second MR because in the first one, the deletion of triples was removed and now data cannot be removed when the entity is deleted.
- 🇬🇷Greece idimopoulos
We have discussed with Claudiu, I will merge MR 25 and will continue to https://www.drupal.org/project/rdf_sync/issues/3478179 ✨ Allow to check for the publication status before syncing Active to conclude the tests.
-
idimopoulos →
committed d5c94d7d on 1.x authored by
claudiu.cristea →
Issue #3472816 by claudiu.cristea, fishfree: "Return value must be of...
-
idimopoulos →
committed d5c94d7d on 1.x authored by
claudiu.cristea →
- 🇬🇷Greece idimopoulos
Sorry, here is an updated script that works. I tested it in my case (I managed to replicate it by tampering with the data in the table)
<?php use Drupal\Core\Entity\ContentEntityInterface; $rdfMapper = \Drupal::getContainer()->get('rdf_sync.mapper'); $entityTypeManager = \Drupal::entityTypeManager(); $entityFieldManager = \Drupal::service('entity_field.manager'); $messages = []; foreach ($entityTypeManager->getDefinitions() as $entityTypeId => $entityType) { if (!$entityType->entityClassImplements(ContentEntityInterface::class) || !($bundleEntityTypeId = $entityType->getBundleEntityType())) { continue; } $bundleStorage = $entityTypeManager->getStorage($bundleEntityTypeId); foreach ($bundleStorage->loadMultiple() as $bundleId => $bundle) { if ($rdfSyncSettings = $bundle->getThirdPartySettings('rdf_sync')) { // Get the base table of the entity. $baseTable = $entityType->getBaseTable(); // Get the entity key for the ID. $idKey = $entityType->getKey('id'); // Get the bundle key. $bundleIdKey = $entityType->getKey('bundle'); // Load from the database IDs of the entity type that do not have do not // have a URI stored. Select IDs that in the `rdf_sync_uri` table, they // do not have a mapping to the columns `entity_type`, `entity_id`, and // `bundle`, or do, but the `uri` is empty. $query = \Drupal::database()->select($baseTable, 'e'); $query->leftJoin('rdf_sync_uri', 'u', "e.{$idKey} = u.entity_id"); $query->fields('e', [$idKey]); $query->condition("e.{$bundleIdKey}", $bundleId); $query->isNull('u.uri'); $entityIds = $query->execute()->fetchCol(); if ($entityIds) { $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, IDs with missing URIs: " . implode(', ', $entityIds); } // Uncomment below to also appoint new URIs. foreach ($entityTypeManager->getStorage($entityTypeId)->loadMultiple($entityIds) as $entity) { $pluginId = $rdfMapper->getRdfUriPluginId($entityTypeId, $bundleId); $plugin = \Drupal::getContainer()->get('plugin.manager.rdf_sync.uri_generator')->createInstance($pluginId); $uri = $plugin->setEntity($entity)->generate(); \Drupal::database() ->insert('rdf_sync_uri') ->fields(['entity_type', 'entity_id', 'uri', 'bundle']) ->values([ 'entity_type' => $entity->getEntityTypeId(), 'entity_id' => $entity->id(), 'uri' => $uri, 'bundle' => $entity->bundle(), ])->execute(); $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, ID: {$entity->id()}, New URI appointed: $uri"; } } } } print implode("\n", $messages);
- 🇷🇴Romania claudiu.cristea Arad 🇷🇴
@fishfree, did you try to test with suggestions from @idimopoulos? I want to close this issue.
- 🇨🇳China fishfree
For me, the problem still exists after upgrading to the latest dev version of rdf_sync and running
vendor/bin/drush php:script ./my-script.php
. - Status changed to Postponed: needs info
about 1 month ago 6:25pm 20 November 2024 - 🇷🇴Romania claudiu.cristea Arad 🇷🇴
We cannot do anything else here. If you can give us more details, let us know