Multilingual Migrations for Non-Drupal Sources

Created on 25 May 2016, over 8 years ago
Updated 26 June 2023, about 1 year ago

Motivation

So there's a flurry of activity concerning multilingual Drupal content from D6 & D7 being properly migrated ( #2208401: [META] Remaining multilingual migration paths β†’ ), but I'm not seeing much if anything concerning non-Drupal imports for multilingual content. Point of fact, I'm having trouble coming up with good examples of how multilingual migrations are intended to work, let alone actual working examples using Drupal or other sources.

Proposed resolution

I'd like to use this thread to compile migration solutions or examples that successfully implement multilingual content. Ultimately, that would become a documentation page, or depending on how things go a feature request or module.

Groundwork

So since I had an immediate need and was looking for solutions I ended up finding #2313265: How import Multilingual Content with Migrate Module β†’ . While it originated Pre-D8, @digitaldonkey had a good solution for a POST_ROW_SAVE event in D8 β†’ . I liked it, but it wasn't flexible enough for me to I took it and made it a little more broadly based. I updated the 'updateTranslations' function of the original:

/**
   * MigrateEvents::POST_ROW_SAVE event handler.
   *
   * @param MigratePostRowSaveEvent $event
   *   Instance of Drupal\migrate\Event\MigratePostRowSaveEvent.
   */
  public function updateTranslations(MigratePostRowSaveEvent $event) {

    $row =  $event->getRow();

    if ($row->hasSourceProperty('constants/available_languages')) {

      // These are defined in migration.yml.
      $available_languages = $row->getSource()['constants']['available_languages'];
      $default_language = $row->getDestination()['langcode'];

      // Unset default language from available languages.
      unset($available_languages[$default_language]);

      $migrated_entity = $event->destinationIdValues[0];
      $dest_config = $event->getMigration()->getDestinationConfiguration();
      $dest_plugin = explode(':', $dest_config['plugin']);

      if ($dest_plugin[0] == 'entity') {
        $entity = \Drupal::entityTypeManager()
          ->getStorage($dest_plugin[1])
          ->load($migrated_entity);

        foreach ($available_languages as $key => $lang_map) {

          $translated_entity = $entity->addTranslation($key);
          foreach ($lang_map as $field => $source) {
            $translated_entity->$field = $row->getSourceProperty($source);
          }
          $translated_entity->save();

        }

        $map = $event->getMigration()->getIdMap();
        $map->saveIdMapping($event->getRow(), array($migrated_entity));
      }
    }
  }

What this allows you to do is declare translations in your YAML file by adding available translations as constants. So your source would look something like

source:
  ...
  constants:
    available_languages:
      en:
      es:
        title: title_spanish

Where title is the name of the field you're entering and title_spanish would be the field coming in from the source. This is perfect for a lightweight translation of content from the same source. Note that it only works on entities and that the main langcode MUST be set in the process.

For bulkier translations that need processing or those coming from a different source, I put together a custom destination. This takes a separate migration and appends identically ID'd translations to it.


namespace Drupal\custom_migrate\Plugin\migrate\destination;

use Drupal\Component\Plugin\DependentPluginInterface;
use Drupal\Core\Entity\DependencyTrait;
use Drupal\migrate\Plugin\MigratePluginManager;
use Drupal\migrate\Plugin\MigrationPluginManagerInterface;
use Drupal\Core\Language\LanguageManagerInterface;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\migrate\Plugin\MigrationInterface;
use Drupal\migrate\Row;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\migrate\Plugin\migrate\destination\DestinationBase;

/**
 * Provides Configuration Management destination plugin.
 *
 * Persist data to the config system.
 *
 * When a property is NULL, the default is used unless the configuration option
 * 'store null' is set to TRUE.
 *
 * @MigrateDestination(
 *   id = "entity_translation"
 * )
 */
class EntityTranslation extends DestinationBase implements ContainerFactoryPluginInterface, DependentPluginInterface {

  use DependencyTrait;

  /**
   * The config object.
   *
   * @var \Drupal\migrate\Plugin\MigrationInterface
   */
  protected $old_migration;

  /**
   * The language manager.
   *
   * @var \Drupal\Core\Language\LanguageManagerInterface
   */
  protected $language_manager;

  /**
   * The process plugin manager.
   *
   * @var \Drupal\migrate\Plugin\MigratePluginManager
   */
  protected $processPluginManager;

  /**
   * The migration plugin manager.
   *
   * @var \Drupal\migrate\Plugin\MigrationPluginManagerInterface
   */
  protected $migrationPluginManager;

  /**
   * Constructs a Config destination object.
   *
   * @param array $configuration
   *   A configuration array containing information about the plugin instance.
   * @param string $plugin_id
   *   The plugin ID for the plugin instance.
   * @param mixed $plugin_definition
   *   The plugin implementation definition.
   * @param \Drupal\migrate\Plugin\MigrationInterface $migration
   *   The migration entity.
   * @param \Drupal\Core\Language\ConfigurableLanguageManagerInterface $language_manager
   *   The language manager.
   */
  public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, MigrationPluginManagerInterface $migration_plugin_manager, MigratePluginManager $process_plugin_manager, LanguageManagerInterface $language_manager) {
    parent::__construct($configuration, $plugin_id, $plugin_definition, $migration);
    $this->migrationPluginManager = $migration_plugin_manager;
    $this->migration = $migration;
    $this->processPluginManager = $process_plugin_manager;
    $this->language_manager = $language_manager;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $migration,
      $container->get('plugin.manager.migration'),
      $container->get('plugin.manager.migrate.process'),
      $container->get('language_manager')
    );
  }

  /**
   * {@inheritdoc}
   */
  public function import(Row $row, array $old_destination_id_values = array()) {
    
    $migration_id = $this->configuration['migration'];
    $migration = $this->migrationPluginManager->createInstance($migration_id);

    // TODO: I think this only works right now if the keys and key labels match, so I'll need to expand later
    $source_id_values = $row->getSourceIdValues();

    $destination_ids = $migration->getIdMap()->lookupDestinationID($source_id_values);

    $dest_config = $migration->getDestinationConfiguration();
    $dest_plugin = explode(':', $dest_config['plugin']);

    $entity = \Drupal::entityTypeManager()
      ->getStorage($dest_plugin[1])
      ->load($destination_ids[0]);

    $lang = $row->getDestinationProperty('langcode');

    // TODO: Validate langcode against list of site languages

    $translated_entity = $entity->addTranslation($lang, $row->getDestination());
    $translated_entity->save();

    return $destination_ids;

  }

  /**
   * {@inheritdoc}
   */
  public function fields(MigrationInterface $migration = NULL) {
    // @todo Dynamically fetch fields using Config Schema API.
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    $ids['config_name']['type'] = 'string';
    return $ids;
  }

  /**
   * {@inheritdoc}
   */
  public function calculateDependencies() {
    $provider = explode('.', $this->config->getName(), 2)[0];
    $this->addDependency('module', $provider);
    return $this->dependencies;
  }

}

It's still very rough, but once it's in place you can use it to translate entities by adding this destination:

destination:
  plugin: entity_translation
  migration: original_content

Where original_content is the targeted migration. Note that it's still very rough, but is certainly a step in the right direction. Once again, it will only work for entities and the langcode needs to be set in the process.

πŸ’¬ Support request
Status

Closed: works as designed

Version

11.0 πŸ”₯

Component
MigrationΒ  β†’

Last updated 1 day ago

Created by

πŸ‡ΊπŸ‡ΈUnited States dhansen

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

  • Needs issue summary update

    Issue summaries save everyone time if they are kept up-to-date. See Update issue summary task instructions.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA

    Closing this, as it's ultimately not something we want to do to in the core plugins.

    I'm still not quite sure the point of the patch, as the parent getEntityId is expected to always return a string, around it's technically returning whatever is in the destination array under the entity types id key (like nid for nodes), so I suppose you could write a migration with a process that set nid to an array and the system would just return that, but that's not how the cure destination plugins are designed to be used. The cute destination plugins can import multilingual data when used properly, and they are indifferent to the source and processes used provided the end result is a destination array in the format the destination plugins are expecting.

    So in general, I would say if you are using the core plugins with custom sources and migrations, you are responsible for confirming the ultimate output of that into the format needed for the core destinations. If your source data isn't conducive to doing that easily and requires custom destination processing as well, by all means do so in a custom destination as well.

Production build 0.71.5 2024