Contrib.social

Migrate Google Sheets Example

Open on Drupal.org →
Created on 7 June 2023, over 1 year ago
Updated 5 April 2024, 9 months ago

Problem/Motivation

Attempting to test #3236814 using the existing example migration spreadsheets and config in Drupal 9.2 with PHP 8.1 has exposed a couple of issues.

First, extracting the migrate_google_sheets_example submodule into its own module directory fails to install out of the box, because Drupal complains it's missing a storage config file for the game node's image field. Dropping this into a file called field.storage.node.field_image.yml resolves that issue:

langcode: en
status: true
dependencies:
  module:
    - media
    - node
id: node.field_image
field_name: field_image
entity_type: node
type: entity_reference
settings:
  target_type: media
module: core
locked: false
cardinality: 1
translatable: true
indexes: {  }
persist_with_no_fields: false
custom_storage: false

Once that file is added, the example module can be enabled and its install file creates the expected entities. More challenging is what happens when attempting to run the example migration processes, even after changing the URL format to Google's v4 API per the patch provided by KarlShea. There seem to be two errors that pop up for all the migrations after the block one, which all rest on either ltrim or strlen being passed an array (specifically, a 1-length array with a string in the first element).

TypeError: ltrim(): Argument #1 ($string) must be of type string, array given in ltrim() (line 92 of /Users/brendanadkins/Sites/test/web/core/modules/menu_link_content/src/Plugin/migrate/process/LinkUri.php)
#0 /Users/brendanadkins/Sites/test/web/core/modules/menu_link_content/src/Plugin/migrate/process/LinkUri.php(92): ltrim(Array, '/')
#1 /Users/brendanadkins/Sites/test/web/core/modules/migrate/src/MigrateExecutable.php(448): Drupal\menu_link_content\Plugin\migrate\process\LinkUri->transform(Array, Object(Drupal\migrate_tools\MigrateBatchExecutable), Object(Drupal\migrate\Row), 'link/uri')
#2 /Users/brendanadkins/Sites/test/web/core/modules/migrate/src/MigrateExecutable.php(221): Drupal\migrate\MigrateExecutable->processPipeline(Object(Drupal\migrate\Row), 'link/uri', Array, Array)
#3 /Users/brendanadkins/Sites/test/web/modules/contrib/migrate_tools/src/MigrateBatchExecutable.php(215): Drupal\migrate\MigrateExecutable->import()
#4 /Users/brendanadkins/Sites/test/web/core/includes/batch.inc(295): Drupal\migrate_tools\MigrateBatchExecutable::batchProcessImport('menu_links', Array, Array)
#5 /Users/brendanadkins/Sites/test/web/core/includes/batch.inc(137): _batch_process()
#6 /Users/brendanadkins/Sites/test/web/core/includes/batch.inc(93): _batch_do()
#7 /Users/brendanadkins/Sites/test/web/core/modules/system/src/Controller/BatchController.php(55): _batch_page(Object(Symfony\Component\HttpFoundation\Request))
#8 [internal function]: Drupal\system\Controller\BatchController->batchPage(Object(Symfony\Component\HttpFoundation\Request))
#9 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)
#10 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/Render/Renderer.php(580): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#11 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#12 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)
#13 /Users/brendanadkins/Sites/test/vendor/symfony/http-kernel/HttpKernel.php(169): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#14 /Users/brendanadkins/Sites/test/vendor/symfony/http-kernel/HttpKernel.php(81): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#15 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#16 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#17 /Users/brendanadkins/Sites/test/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#18 /Users/brendanadkins/Sites/test/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#19 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#20 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#21 /Users/brendanadkins/Sites/test/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#22 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/DrupalKernel.php(718): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#23 /Users/brendanadkins/Sites/test/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#24 {main}
.
TypeError: strlen(): Argument #1 ($string) must be of type string, array given in strlen() (line 477 of /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Component/Utility/Unicode.php)
#0 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Component/Utility/Unicode.php(477): strlen(Array)
#1 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Component/Utility/Xss.php(65): Drupal\Component\Utility\Unicode::validateUtf8(Array)
#2 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Component/Utility/Xss.php(123): Drupal\Component\Utility\Xss::filter(Array, Array)
#3 /Users/brendanadkins/Sites/test/web/modules/contrib/migrate_tools/src/Controller/MigrationController.php(228): Drupal\Component\Utility\Xss::filterAdmin(Array)
#4 [internal function]: Drupal\migrate_tools\Controller\MigrationController->process(Object(Drupal\migrate_plus\Entity\MigrationGroup), Object(Drupal\migrate_plus\Entity\Migration))
#5 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)
#6 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/Render/Renderer.php(580): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#7 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#8 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)
#9 /Users/brendanadkins/Sites/test/vendor/symfony/http-kernel/HttpKernel.php(169): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#10 /Users/brendanadkins/Sites/test/vendor/symfony/http-kernel/HttpKernel.php(81): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#11 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#12 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#13 /Users/brendanadkins/Sites/test/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#14 /Users/brendanadkins/Sites/test/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#15 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#16 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#17 /Users/brendanadkins/Sites/test/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#18 /Users/brendanadkins/Sites/test/web/core/lib/Drupal/Core/DrupalKernel.php(718): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#19 /Users/brendanadkins/Sites/test/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#20 {main}
.

In both instances, the data coming through is just of the form [0 => 'foo'], but the code sees only that it's an array, not that it can be reduced to a string.

Steps to reproduce

  1. Install and patch the 2.x version of this module
  2. Extract the migrate_google_sheets_example as recommended on the ThinkShout blog, but in addition, extract the migrate_google_sheets_example_setup module as its own module and enable both
  3. Even after adding the storage file for the image field, attempting to run the migrations after the block one produces errors in the XSS filtering that is part of the migrate core module

Proposed resolution

🐛 Bug report
Status

Fixed

Version

2.0

Component

Code

Created by

🇺🇸United States brendanthinkshout

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @brendanthinkshout
  • Status changed to Needs review 10 months ago
  • Comment 10 months ago →
    🇺🇸United States karlshea Minneapolis 🇺🇸

    For at least the menu_links migration that seems to be due to the process settings here:

      link/uri:
    plugin: link_uri
    source:
      - path

    Passing path like that will create an array, the correct settings would be:

      link/uri:
    plugin: link_uri
    source: path

    (Search web/core for plugin: link_uri to see other examples)

    The same also seems to be true for the other migrations, however since they seem to be meant more of an example they can't actually be run (or the setup module even enabled—it depends on non-exportable preexisting content e.g. a content block uuid in the database).

    With the v4 API patch, enough changes to the example modules to get them enabled, and changing the process setting above I was able to get the menu_links to run without any errors.

  • Merge request !4Fix link_uri source as arrays → (Merged) created by karlshea
  • Status changed to Fixed 9 months ago
  • Comment 9 months ago →
    🇺🇸United States karlshea Minneapolis 🇺🇸
  • Comment 9 months ago →
    System Message

    Automatically closed - issue fixed for 2 weeks with no activity.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024