TypeError: Drupal\rdf_sync\RdfSyncSynchronizer::Drupal\rdf_sync\{closure}(): Return value must be of type string, null returned in Drupal\rdf_sync\RdfSyncSynchronizer->Drupal\rdf_sync\{closure}() (line 122 of /var/www/html/drupal/web/modules/con

Created on 7 September 2024, 4 months ago

Background: RDF_sync module worked well before I accidentally removed the Virtuoso container and lost all the synchronized RDF data. I re-run a new Virtuoso container and connected it with Drupal. I tried manually batch write all my mapped nodes into Virtuoso as below:
vendor/bin/drush rdf_sync:synchronize node
It showed error as below:

In ProcessBase.php line 171:
                                                                                                                                                                                                                                
  Unable to decode output into JSON: Syntax error                                                                                                                                                                               
                                                                                                                                                                                                                                
  TypeError: Drupal\rdf_sync\RdfSyncSynchronizer::Drupal\rdf_sync\{closure}(): Return value must be of type string, null returned in Drupal\rdf_sync\RdfSyncSynchronizer->Drupal\rdf_sync\{closure}() (line 122 of /var/www/ht  
  ml/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php).                                                                                                                                                        
                                                                                                                                                                                                                                
  Fatal error: Uncaught TypeError: Drupal\rdf_sync\RdfSyncSynchronizer::Drupal\rdf_sync\{closure}(): Return value must be of type string, null returned in /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSync  
  hronizer.php:122                                                                                                                                                                                                              
  Stack trace:                                                                                                                                                                                                                  
  #0 [internal function]: Drupal\rdf_sync\RdfSyncSynchronizer->Drupal\rdf_sync\{closure}()                                                                                                                                      
  #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(119): array_map()                                                                                                                          
  #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(104): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize()                                                                                 
  #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()                                                                       
  #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate()                                                                                                                          
  #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func()                                                                                              
  #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()                                                                          
  #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate()                                                                            
  #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate()                                                                                          
  #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate()                                                                                                             
  #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()                                                                                                                                                                  
  #11 {main}                                                                                                                                                                                                                    
    thrown in /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php on line 122 

When I created a node in Drupal Web UI, it was synchronized into Virtuoso.
Is it because there are many entity_reference fields between bundles?

🐛 Bug report
Status

Active

Version

1.0

Component

Code

Created by

🇨🇳China fishfree

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @fishfree
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    I need more context to understand your case

  • Status changed to Postponed: needs info 4 months ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    Also, could you, please, enter a proper title for this issue and not paste the error message. Thanks

  • Status changed to Fixed 4 months ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    I tried manually batch write all my mapped nodes into Virtuoso as below:

    This happens when one ore more nodes don't have a corresponding record in the rdf_sync_uri table (or for some reason the uri column is empty). Check and fix those cases

  • Status changed to Needs work 4 months ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    I have decided to give a 2nd look at this. I think it would be correct to handle the case when, accidentally, some entities URIs were lost and prevent any ugly error.

  • Merge request !24Protection against missing URIs → (Merged) created by claudiu.cristea
  • Pipeline finished with Skipped
    4 months ago
    #281924
  • Status changed to Fixed 4 months ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    Thank you for reporting

  • 🇨🇳China fishfree

    @claudiu Thank you! After apply your patch, my error turned into:

    In ProcessBase.php line 171:
                                                                                                                                                                    
      Unable to decode output into JSON: Syntax error                                                                                                               
                                                                                                                                                                    
      Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or empty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Re  
      source.php:69                                                                                                                                                 
      Stack trace:                                                                                                                                                  
      #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(61): EasyRdf\Resource->__construct()                              
      #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(133): Drupal\rdf_sync\Normalizer\RdfSyncNormalizer->normalize()            
      #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(104): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize()                 
      #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()       
      #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate()                                                          
      #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func()                              
      #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()          
      #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate()            
      #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate()                          
      #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate()                                             
      #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()                                                                                                  
      #11 {main}                                                                                                                                                    
        thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69  
  • Status changed to Needs work 4 months ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    I see the same bug in my project :(

  • Merge request !25Resolve #3472816 "Followup" → (Merged) created by claudiu.cristea
  • Status changed to Needs review 4 months ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    @fishfree, could you, please, test the latest MR?

  • 🇨🇳China fishfree

    @claudiucristea Thank you very much! Would you pls share a single patch file against the version https://www.drupal.org/project/rdf_sync/releases/1.0.0-alpha12 ? Sorry I'm not able to manually merge your multiple MRs.

  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    You should just go to MR and append .diff to URL

  • 🇨🇳China fishfree

    @claudiucristea Thank you very much! After applying https://git.drupalcode.org/project/rdf_sync/-/merge_requests/25.diff based on https://www.drupal.org/project/rdf_sync/releases/1.0.0-alpha12 , I re-run vendor/bin/drush rdf_sync:synchronize node
    It showed as below:

    > ......
     > [notice] Synchronized 1200 out of 1965 entities
    >  [notice] Synchronized 1250 out of 1965 entities
    >  [notice] Synchronized 1300 out of 1965 entities
    >  [notice] Synchronized 1350 out of 1965 entities
    > 
    > In Resource.php line 69:
    >                                                        
    >   $uri should be a string and cannot be null or empty  
    >                                                        
    > 
    > PHP Fatal error:  Uncaught InvalidArgumentException: $uri should be a string and cannot be null or empty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php:69
    > Stack trace:
    > #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(108): EasyRdf\Resource->__construct()
    > #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(122): Drupal\rdf_sync\Normalizer\RdfSyncNormalizer->normalize()
    > #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(88): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize()
    > #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()
    > #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate()
    > #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func()
    > #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()
    > #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate()
    > #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate()
    > #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate()
    > #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()
    > #11 {main}
    >   thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69
    
    In ProcessBase.php line 171:
                                                                                                        
      Unable to decode output into JSON: Syntax error                                                   
                                                                                                        
      Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or em  
      pty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php:69                         
      Stack trace:                                                                                      
      #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(108)  
      : EasyRdf\Resource->__construct()                                                                 
      #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(122): Drupal\  
      rdf_sync\Normalizer\RdfSyncNormalizer->normalize()                                                
      #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(88): Drupal\r  
      df_sync\RdfSyncSynchronizer->doSynchronize()                                                      
      #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.p  
      hp(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()                                           
      #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTermina  
      te()                                                                                              
      #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispa  
      tcher.php(111): call_user_func()                                                                  
      #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\Event  
      Dispatcher\ContainerAwareEventDispatcher->dispatch()                                              
      #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Sy  
      mfony\Component\HttpKernel\HttpKernel->terminate()                                                
      #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddl  
      eware\StackedHttpKernel->terminate()                                                              
      #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKe  
      rnel->terminate()                                                                                 
      #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()                                      
      #11 {main}                                                                                        
        thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69            
    

    I checked in my Virtuoso, it seems all the 1965 entities had been synchonized. Why still the error occured?

  • 🇬🇷Greece idimopoulos

    Hello @fishfree

    From what I see, it seems that you might not have a URI for all entities. Probably you can check directly by running a quick query like

    select * from node as n
    left join rdf_sync_uri as r on n.nid = r.entity_id
    where r.uri is NULL and n.type = 'my_bundle'
    

    if we are talking about nodes and re run this for every bundle to see which IDs are empty.

    Furthermore, I created the following script (works for me with nodes) to iterate through all entity types in order to check all of them for empty URIs. Sorry for all the container calls, it was created in haste. After running it with drush php:script ./my-script.php and you verify if I am correct, you can also uncomment the commented section designated, to also create URIs for the IDs that are missing them and update the entities. Please, note, that this will actually update the entities. If you do not wish them updated, adjust accordingly or manually force the values in the rdf_sync_uri table.

    <?php
    
    use Drupal\Core\Entity\ContentEntityInterface;
    
    $rdfMapper = \Drupal::getContainer()->get('rdf_sync.mapper');
    $entityTypeManager = \Drupal::entityTypeManager();
    $entityFieldManager = \Drupal::service('entity_field.manager');
    $messages = [];
    foreach ($entityTypeManager->getDefinitions() as $entityTypeId => $entityType) {
      if (!$entityType->entityClassImplements(ContentEntityInterface::class) || !($bundleEntityTypeId = $entityType->getBundleEntityType())) {
        continue;
      }
    
      $bundleStorage = $entityTypeManager->getStorage($bundleEntityTypeId);
      foreach ($bundleStorage->loadMultiple() as $bundleId => $bundle) {
        if ($rdfSyncSettings = $bundle->getThirdPartySettings('rdf_sync')) {
          // Get the base table of the entity.
          $baseTable = $entityType->getBaseTable();
          // Get the entity key for the ID.
          $idKey = $entityType->getKey('id');
          // Get the bundle key.
          $bundleIdKey = $entityType->getKey('bundle');
    
          // Load from the database IDs of the entity type that do not have do not
          // have a URI stored. Select IDs that in the `rdf_sync_uri` table, they
          // do not have a mapping to the columns `entity_type`, `entity_id`, and
          // `bundle`, or do, but the `uri` is empty.
          $query = \Drupal::database()->select($baseTable, 'e');
          $query->leftJoin('rdf_sync_uri', 'u', "e.{$idKey} = u.entity_id");
          $query->fields('e', [$idKey]);
          $query->condition("e.{$bundleIdKey}", $bundleId);
          $query->isNull('u.uri');
          $entityIds = $query->execute()->fetchCol();
          if ($entityIds) {
            $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, IDs with missing URIs: " . implode(', ', $entityIds);
          }
    
          // Uncomment below to also appoint new URIs.
    //      foreach ($entityTypeManager->getStorage($entityTypeId)->loadMultiple($entityIds) as $entity) {
    //        $pluginId = $rdfMapper->getRdfUriPluginId($entityTypeId, $bundleId);
    //        $plugin = \Drupal::getContainer()->get('plugin.manager.rdf_sync.uri_generator')->createInstance($pluginId);
    //        $uri = $plugin->setEntity($entity)->generate();
    //        $fieldName = \Drupal::getContainer()->get('rdf_sync.mapper')->getRdfUriFieldName(entity: $entity);
    //        $entity->get($fieldName)->setValue($uri);
    //        $entity->save();
    //        $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, ID: {$entity->id()}, New URI appointed: $uri";
    //      }
        }
      }
    }
    
    print implode("\n", $messages);
    

    Further more, in https://www.drupal.org/project/rdf_sync/issues/3478179 Allow to check for the publication status before syncing Active , we are working on allowing a subscriber to tamper with the entities before syncing them. That means, that using this subscriber, and a simple check whether the entity does have a URI, you can either identify or block the entity from being synced.

    Please, consider that, the error you are getting is based on the idea that having a bundle synced, all entities should have a URI. That is why if a bundle is mapped, the getUri should always return a value and we do not allow it to return null. That is because a URI is like a persistent identifier in RDF data and thus we thought we should be strict with it.

    I checked in my Virtuoso, it seems all the 1965 entities had been synchonized. Why still the error occured?

    This might be that the URI was removed somewhere along the way or that Virtuoso already had the data before you sync. If you are using docker, and the container crashed, it doesn't mean data were deleted if the volume remained intact. I mean, we would need more information to identify further issues.

    In terms of this ticket, I will review the second MR because in the first one, the deletion of triples was removed and now data cannot be removed when the entity is deleted.

  • 🇬🇷Greece idimopoulos

    We have discussed with Claudiu, I will merge MR 25 and will continue to https://www.drupal.org/project/rdf_sync/issues/3478179 Allow to check for the publication status before syncing Active to conclude the tests.

  • Pipeline finished with Skipped
    3 months ago
    #299766
  • 🇬🇷Greece idimopoulos

    Sorry, here is an updated script that works. I tested it in my case (I managed to replicate it by tampering with the data in the table)

    <?php
    
    use Drupal\Core\Entity\ContentEntityInterface;
    
    $rdfMapper = \Drupal::getContainer()->get('rdf_sync.mapper');
    $entityTypeManager = \Drupal::entityTypeManager();
    $entityFieldManager = \Drupal::service('entity_field.manager');
    $messages = [];
    foreach ($entityTypeManager->getDefinitions() as $entityTypeId => $entityType) {
      if (!$entityType->entityClassImplements(ContentEntityInterface::class) || !($bundleEntityTypeId = $entityType->getBundleEntityType())) {
        continue;
      }
    
      $bundleStorage = $entityTypeManager->getStorage($bundleEntityTypeId);
      foreach ($bundleStorage->loadMultiple() as $bundleId => $bundle) {
        if ($rdfSyncSettings = $bundle->getThirdPartySettings('rdf_sync')) {
          // Get the base table of the entity.
          $baseTable = $entityType->getBaseTable();
          // Get the entity key for the ID.
          $idKey = $entityType->getKey('id');
          // Get the bundle key.
          $bundleIdKey = $entityType->getKey('bundle');
    
          // Load from the database IDs of the entity type that do not have do not
          // have a URI stored. Select IDs that in the `rdf_sync_uri` table, they
          // do not have a mapping to the columns `entity_type`, `entity_id`, and
          // `bundle`, or do, but the `uri` is empty.
          $query = \Drupal::database()->select($baseTable, 'e');
          $query->leftJoin('rdf_sync_uri', 'u', "e.{$idKey} = u.entity_id");
          $query->fields('e', [$idKey]);
          $query->condition("e.{$bundleIdKey}", $bundleId);
          $query->isNull('u.uri');
          $entityIds = $query->execute()->fetchCol();
          if ($entityIds) {
            $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, IDs with missing URIs: " . implode(', ', $entityIds);
          }
    
          // Uncomment below to also appoint new URIs.
          foreach ($entityTypeManager->getStorage($entityTypeId)->loadMultiple($entityIds) as $entity) {
            $pluginId = $rdfMapper->getRdfUriPluginId($entityTypeId, $bundleId);
            $plugin = \Drupal::getContainer()->get('plugin.manager.rdf_sync.uri_generator')->createInstance($pluginId);
            $uri = $plugin->setEntity($entity)->generate();
            \Drupal::database()
              ->insert('rdf_sync_uri')
              ->fields(['entity_type', 'entity_id', 'uri', 'bundle'])
              ->values([
                'entity_type' => $entity->getEntityTypeId(),
                'entity_id' => $entity->id(),
                'uri' => $uri,
                'bundle' => $entity->bundle(),
              ])->execute();
            $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, ID: {$entity->id()}, New URI appointed: $uri";
          }
        }
      }
    }
    
    print implode("\n", $messages);
    
    
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    @fishfree, did you try to test with suggestions from @idimopoulos? I want to close this issue.

  • 🇨🇳China fishfree

    For me, the problem still exists after upgrading to the latest dev version of rdf_sync and running vendor/bin/drush php:script ./my-script.php.

  • Status changed to Postponed: needs info about 1 month ago
  • 🇷🇴Romania claudiu.cristea Arad 🇷🇴

    We cannot do anything else here. If you can give us more details, let us know

Production build 0.71.5 2024