End this module in style? Migration scenario to Drupal 10?

Created on 18 April 2020, over 4 years ago
Updated 18 April 2023, over 1 year ago

Problem/Motivation

An upgrade solution is needed to migrate data from the Weblinks structures on D7.

Proposed resolution

Build the necessary migration pieces to migrate the node data to D9/10 and replace the URL value with a Link field. This should work with Migrate Drupal UI and Migrate Tools.
The module is only used for migrating the data, after that it can be uninstalled.

Remaining tasks

Build a test fixture that adds the necessary D7 source data; this should
Build the necessary logic to add a Link field to the destination content type.
Build migration logic to ensure the data is migrated.
Add test coverage to confirm the data is migrated; see \Drupal\Tests\migrate_drupal\Kernel\d7\MigrateDrupal7TestBase.

User interface changes

API changes

TBD

Data model changes

TBD

Original report by Summit

Hi,

Wouldn't it not be great if this module would end in style moving to D8?
There is now the Link module, which is in core in D8.
The Link Module together with the core Node module contain 90% of the functionality of the weblinks module.

Wouldn't it be great to end this module with a nice migration .yaml file, so weblinks will smoothly convert to Link/Nodes?
What would be necessary to get this rocking?

I have more than 1000 weblinks nodes on D7, and would love to be able to get those to D10.
From D6 - D7
https://www.drupal.org/project/weblinks/issues/2210683 β†’ and https://www.drupal.org/project/migrate_d2d/issues/2467371 β†’
I had great help from Gerhard Stegemann ( https://www.drupal.org/u/gstegemann β†’ ), rsbecker and mikeryan ( https://www.drupal.org/u/mikeryan β†’ )

Now is the time to get from D7 - Drupal 10.
Could some of those great guys assist me with this? I think this is starting become a greater problem, while D7 module is also not maintained anymore. Thanks for who wants to participate to get this rocking!

Greetings, Martijn

πŸ“Œ Task
Status

Closed: duplicate

Component

Miscellaneous

Created by

πŸ‡³πŸ‡±Netherlands Summit

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,
    I start up the migration scenario again! Now from Drupal 7 -> Drupal 10.
    First is it all a lot of efford to get all modules in place.
    Then get the content over. Now over 1000 nodes :(

    Anyone has experience with the Web links module now, to get it in Link?

    Looking forward to replies:)
    Greetings, Martijn

  • πŸ‡©πŸ‡ͺGermany GStegemann

    Hi Martijn,

    Seems to be a good idea.

    Anyone has experience with the Web links module now, to get it in Link?

    No, not at all. I stopped all my Drupal activities since the beginning of 2020.

    Good luck and best regards,
    Gerhard

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,

    May be this is the best scenario?
    First converting the Weblinks to Link, Second migrate Link D7 to Link D10?
    Issue here: https://www.drupal.org/project/weblinks/issues/3347473 πŸ’¬ Convert Weblinks to Link module for preparation Drupal 10 migration Active
    Greetings,

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,
    Will this may be work for Weblinks: https://www.drupal.org/project/url/issues/1778224 β†’

    <?php
    namespace Drupal\custom_url_migrate\plugin\migrate\field;
    
    use Drupal\migrate\Plugin\MigrationInterface;
    use Drupal\migrate_drupal\Plugin\migrate\field\FieldPluginBase;
    /**
     * @MigrateField(
     * id = "url",
     * core = {7},
     * type_map = {
     * "url" = "link"
     * },
     * source_module = "weblinks",
     * destination_module = "link"
     * )
     */
    class url extends FieldPluginBase {
    
      /**
       * {@inheritdoc}
       */
      public function getFieldWidgetMap() {
        return [
          'url' => 'link',
        ];
      }
    
      /**
       * {@inheritdoc}
       */
      public function getFieldFormatterMap() {
        return [
         //'default' => 'link',
          //'url' => 'link',
          'url_external' => 'link',
        ];
      }
    
      /**
       * {@inheritdoc}
       */
      public function processFieldValues(MigrationInterface $migration, $field_name, $data) {
        $process = [
          'plugin' => 'sub_process',
          'source' => $field_name,
          'process' => [
            'uri' => 'value',
          ],
        ];
        $migration->setProcessOfProperty($field_name, $process);
      }
    
    }
    

    Didn't test it yet...Any one success with this?

  • πŸ‡ΊπŸ‡ΈUnited States DamienMcKenna NH, USA

    Does the Weblinks content type get picked up during a normal migration with Migrate Tools? It do, other thank the extra values from the "weblinks" table does everything else come across correctly?

    I think it could be worth collaborating on the necessary plumbing to make this work as expected on a new 1.0.x branch, to have both migration plugins for converting the "url" field from the custom table, and otherwise make sure that the content type is migrated with the rest of the normal migration.

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi Damien, because there is now Weblinks module in D8-10, it is not picked up I assume? Shouldn't there be a contenttype as Weblinks in D8-10 then?

    It would be great to get this to work to migrate to D8-10 to native Link module.

    Greetings and thanks for quick reply Martijn

  • πŸ‡ΊπŸ‡ΈUnited States DamienMcKenna NH, USA

    Here's what I propose:
    1. Build a barebones D7 site with the Weblinks module installed and see what migrates to D9/10 using just Migrate Drupal UI; review the migrated data, work out a list of what is migrated and what is not. Document the setup so tests can be built.
    2. Probably needed: a source plugin that extends the core D7 node source plugin and adds the fields from the "weblinks" table so they can be made available for migrations.
    3. Based on #1, build additional plugins as necessary.
    3. Add the necessary logic so that the new plugin(s) are used when migrating with Migrate Drupal.
    4. Document how to migrate the data for sites that are not using Migrate Drupal.

    It looks like parts of this have been built, it just will take someone to start compiling the pieces in one place.

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,
    I will start with one! Next week. Thanks!
    greetings,

  • πŸ‡³πŸ‡±Netherlands Summit
  • πŸ‡³πŸ‡±Netherlands Summit

    Hi, I did the following

    1) Build a latest Drupal 7 site from scratch; version 7.95.
    2) Add Weblinks module 7.x-1.1
    3) Add one Weblinks node as content. No other content in the site.

    4) Build the latest Drupal 10 site from scratch, version 10.0.5
    5) Enabled the three Migrate modules from core.

    6) Run the upgrading process. No content is being transferred unfortunately.

    I got this screen from modules which are not updated. Off course Weblinks, because there is no Weblinks in D10.
    How to proceed please? Do you want to see the logs of upgrading?

    Greetings, and thanks for your help in advance,

  • πŸ‡³πŸ‡±Netherlands Summit

    This screen

  • πŸ‡ΊπŸ‡ΈUnited States DamienMcKenna NH, USA

    Out of interest did you check the node tables in the database?

  • πŸ‡ΊπŸ‡ΈUnited States DamienMcKenna NH, USA

    I added a plan to the issue summary.

    The first thing would be to build a fixture that shows what the data looks like in D7.

  • πŸ‡ΊπŸ‡ΈUnited States DamienMcKenna NH, USA
  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,
    Yes I checked the node table: " MySQL returned an empty result set (i.e. zero rows). (Query took 0.0002 seconds.)"
    Sorry to report this.
    Greetings,

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,
    I tried to follow these instructions: https://www.drupal.org/docs/drupal-apis/migrate-api/generating-database-... β†’
    Make the changes and export to the changes to the fixture file
    1. Make the desired changes on your Drupal 6 / Drupal 7 test site that are needed for your additional test coverage.
    ==> I installed the weblinks 1.1 module on drupal 7 testsite.
    Then I run:

    php core/scripts/db-tools.php dump --database fixture_connection > core/modules/migrate_drupal/tests/fixtures/drupal7.php
    

    2. Export your changes to the fixture file. Make sure that you are now working in the git branch of your patch.
    ==> How to build and export the fixture file for weblinks?
    greetings,

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi Damian,

    Is this what you are looking for as Fixture, the test file?
    It is after a long search, standard in the weblinks module!
    See attached file (weblinks_test). I found it by enabling SImpletest_fixture module
    Greetings,

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi, can anyone help with this?
    Greetings and thanks in advance!

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi, I think I build fixture with
    Then one is completed right: 1) Build a test fixture that adds the necessary D7 source data;
    With https://www.drupal.org/files/issues/2023-04-01/weblinks.test β†’ ?
    Now I think next step is to do 2) Probably needed: a source plugin that extends the core D7 node source plugin and adds the fields from the "weblinks" table so they can be made available for migrations.

    How to build the source plugin for this please? I see this source, but I am not a good enough programmer to get this working: https://www.drupal.org/docs/8/api/migrate-api/migrate-source-plugins/ove... β†’ How to begin please? Weblinks module is a Drupal 7 module with links embedded, but not as a field like linkfield?

    greetings,

  • πŸ‡³πŸ‡±Netherlands Summit

    first step on Migration script of whole nodetype weblinks here: https://www.drupal.org/files/issues/2023-04-11/weblinks-listing-D7-D10-m... β†’
    first step on Migration script, just standard nodetype weblinks here: weblinks-clean-D7-D10-migratescript.txt

    I think most important is this peace:

     url:
        - 
        plugin: field_link
        uri_scheme: 'https://'
        source: url
        process: field_link: url
    

    Will this work? based on https://api.drupal.org/api/drupal/core%21modules%21link%21src%21Plugin%2...

    Greetings,

  • πŸ‡³πŸ‡±Netherlands Summit

    Hi,
    I made a specific issue for the Migration scenario using Drupal 10 Migrate modules based on https://evolvingweb.com/blog/drupal-8-migration-migrating-basic-data-part-1 https://www.drupal.org/project/weblinks/issues/3353910 πŸ’¬ An upgrade solution is needed to migrate data from the Weblinks structures on D7 to D10. Active
    So I end this issue for now to try to get Weblinks nodes migrated through Migration Modules support with you all.
    First shot of Weblinks-migrate module attached to that issue.
    Greetings,

  • Status changed to Closed: duplicate over 1 year ago
  • πŸ‡³πŸ‡±Netherlands Summit
Production build 0.71.5 2024