- First commit to issue fork.
- 🇺🇸United States hongpong Philadelphia
re upping this.
On YoastPluginExtension file_create_url is deprecated now.public function isActive(array $config) { $moduleHandler = \Drupal::service('module_handler'); if ($moduleHandler->moduleExists('metatag')) { // Look if the WordPress XML contains yoast plugin data. $file_url = file_create_url($config['file_uri']); $content = file_get_contents($file_url); return strpos($content, '_yoast_wpseo_title') !== FALSE; } return FALSE; }
Also getting a warning on
$process[$metatag_field][]
I added a little more type safety on src/Annotation/WordpressMigrationExtensionPlugin.php
in wordpress_migrate_ui/src/Wizard L89
$generator->createMigrations();
should have a @throw exception (IDE notice Unhandled \Drupal\Core\Entity\EntityStorageException )Besides adding some documentation i think those are the main concerns. Great stuff. Setting up a MR based on current 8.x-3.x
- Merge request !16Issue #3173973 by phjou, hongpong, Irina Matusevich: Support contrib wordpress... → (Open) created by hongpong
- 🇺🇸United States hongpong Philadelphia
I think we can put it through when these issues are resolved...
- 🇩🇰Denmark ressa Copenhagen
Great work @phjou and @hongpong!
Support for importing custom fields, would be an awesome feature. And I guess, adding support for some of the most popular plugins, starting with Yoast, and adding more down the line? Thanks!
- 🇺🇸United States msielski
@ressa I am hoping to use the plugin architecture from this issue to start working on an ACF plugin. I'll open a issue for it sometime soon.
- 🇩🇰Denmark ressa Copenhagen
Thanks @msielski, that sounds incredible! I am subscribed to all WordPress Migrate issues, so I will be ready to test a MR, when I see it in the issue queue.
I have limited experience with WordPress, but this video got me started with SCF Unlock WordPress Potential: A Beginner's Guide to Advanced Custom Fields. I managed to create a Custom Post Type with a few fields, but not a List (Drupal: View) ... it looks like the method changed a bit ...
- 🇩🇰Denmark ressa Copenhagen
A pro pos Custom Post Types, Craft CMS yesterday announced their new wp-import tool in Migrating from WordPress.
Maybe it can serve as inspiration for importing custom fields into Drupal, and possibly also the process or GUI?
- 🇺🇸United States hongpong Philadelphia
Say, can you folks look at the latest D11 patch from tolstoydotcom - includes a migration generator interface patch and see how it jibes with your patch? thank you
https://www.drupal.org/files/issues/2024-11-14/d11update-3435726-13.patch →
It seems like something may overlap here?via 📌 Automated Drupal 11 compatibility fixes for wordpress_migrate Needs review Thanks all.
- 🇩🇰Denmark ressa Copenhagen
Increasing Priority, since it is placed under "Major issues" header in 🌱 Plan for wordpress_migrate 8.x-3.x beta release Active , so I am also adding that as parent issue.
- 🇺🇸United States hongpong Philadelphia
I made an XML file with a little bit of yoast SEO data on a few posts in it here: https://gitlab.com/HongPong/wordpress-test-imports/-/blob/master/wp-impo...
Please feel free to use that. Is from today's versions of WordPress and Yoast SEO.
- 🇺🇸United States hongpong Philadelphia
We are getting there, hoping to have this rolled in after the next tagged release. Thank you everyone for continuing to test and refine this.
- 🇺🇸United States msielski
Testing out the MR on this thread with Drupal 10. I ran into the following issues and hopefully fixed:
- isActive in YoastPluginExtension needs to be typed bool to match the base class.
- file_create_url is deprecated. I switched to using a FileUrlGenerator instead.
- (note file_get_contents is also deprecated but still functions, leaving it for now)
- _yoast_wpseo_title was not found in the sample export provided - may be too restrictive to assume this will be present. I changed to searching the export for _yoast_wpseo.
- I made findFieldByFieldType private since it needed a scope.
If it helps anyone, here were my test steps:
- Clone the repo for this module, fetch the MR branch, and check it out per instructions on the repo page.
- Enable wordpress_migrate and its dependencies.
- Install and enable metatag module.
- Add a metatag field to Article content type - doesn't matter the name.
- Download XML from comment #18 and use as the import.
- Choose to create Posts as Articles.
- Enable Yoast as part of the migration.
- Save, then execute the Posts migration.
- Edit the "Nulla atque tenetur repellendus ut" content and verify the description in Drupal is "the post meta description here"
I did not see Yoast meta titles in the XML so that part of the migration doesn't get tested. Also, looks like there's some extra migration for the Drupal yoast_seo module which I didn't install or test.
Finally, I'm not sure I can push my commit to the MR branch.. it says "Members who can merge are allowed to add commits." I'll throw a patch here at the least.
- 🇺🇸United States msielski
Something related to my own work trying to make some additional plugins. I believe we will sooner than later need a way to provide an optional UI for these plugins - for example to let users configure a destination field or to set a variety of migration options. If a plugin offers a UI then it would get inserted into the wizard process before the final step. I built this out as a proof of concept locally and am almost done.
I am basing some of the architecture on the core image module which has ImageEffect plugins which can load their own config UI. That module's approach is have a base plugin class like ImageEffectBase and then a configurable one inherited from it like ConfigurableImageEffectBase. I did the same, but ConfigurableWordpressMigrationExtensionPluginBase and ConfigurableWordpressMigrationExtensionPluginInterface are quite wordy. While I'm prepared to submit this work as a separate issue, I was wondering in this issue if we couldn't shorten the name of WordpressMigrationExtensionPlugin. I'd suggest dropping Plugin since conventionally most Drupal "plugins" don't have plugin in the class. Not a huge deal, but if I was going to ask now seems like the right time. Thank you!
- 🇺🇸United States hongpong Philadelphia
Is there a "Get push access" button on this page for you? that's what they tell me on slack rn.
Anyone can click the "Get push access" button on the issue to get push access to all branches in the issue fork. (its kind-of covered as a bullet on https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... → )
( doesn't say → ). I will pull this in though.
We are not going to roll this into the Alpha7 release but hopefully we will be close for Alpha8. Thank you for going deep into it!!
- 🇩🇰Denmark ressa Copenhagen
Yes, if you have access @msielski and can push to our own issues, there should be a green button, as seen here:
From the documentation page https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... →
- 🇺🇸United States msielski
Ah I was looking in drupalcode for the button. I see it here and pushed it. Thanks.
- 🇺🇸United States hongpong Philadelphia
i may have messed up the commits in the MR just now - however the MR should be up to date with 8.x-3.x HEAD and your patch as well .. i think we have all our pre alpha7 commits in now? thanks for all this.
- 🇺🇸United States hongpong Philadelphia
I had a couple ideas to add in the mix. Hopefully this is not excessive. I think if we can future proof this, by adding a couple parameters to the plugin loader, and WordpressMigrationExtensionPluginInterface.php and / or src/Plugin/WordpressMigrationExtensionPluginBase.php. Additional possible arguments for extension plugins:
$api_version (int), $extra_args (array), $suggested_weight (int). The purpose of $extra_args would be a catch all that could provide special directives outside of other configuration objects, for now just an empty array. (this is kind of common in WordPress already.) $suggested_weight could be used to direct the sorting of these plugins in their execution order, which right now seems to be arbitrary. (e.g. higher weight executes last, or whatever. If the weights match then sort by name?)In the properties of WordpressMigrationExtensionPluginManager.php it could be added, static int for min_api_version, max_api_version and a static function, isApiVersionSupported(int) that returns true if the int is within the supported range. (This would let other modules poll to make sure that they are compatible.)
Thus if the attempted plugin loading is coded with a higher API number then this WordpressMigrationExtensionPluginManager will not load it, or it can throw an exception if it tries to execute. This could also help people who are making custom migrations privately, they could set up their own API number ranges for internal use. (This seems like a cleaner solution than using the Semver of our module to enforce this kind of thing).
Second idea is to add a preprocessor plugin manager that works similarly to this plugin manager. It could be triggered at the end of SourceSelectForm.php: submitForm, and similarly could load the entire uploaded file to a "WordpressMigrationPreprocessorExtensionPlugin" (perhaps selected from another manager wizard page) before the process moves on to AuthorForm.php . I figure this would be the best way to add an XML linter or other security measures ( #3248042: Add XML Lint to check the WordPress exports → ). It could look for encoded XML entities or other oddball or security issues. The plugin manager could also have static int properties for min_preprocessor_api_version, and max_preprocessor_api_version, and a static bool function, isPreprocessorApiVersionSupported(int).
The point with the API versions is that the execution of the plugins processes could change in the future and this would give the reliability of linking an API number to known implementations without locking us into it, and with a minimum API number we can safely jettison old APIs years down the road. I feel like the people making custom extensions for private projects would also benefit from some ideas like this. Stop me if this seems unnecessary or cumbersome but I think it could work well for everyone.
- 🇨🇦Canada phjou Vancouver 🇨🇦 🇪🇺
@hongpong
I really like your suggestions.The weight feature is definitely something we need to add. That could be problematic for some projects in case plugins are affecting the same data and the order matters.
I also like the idea of adding API versions in there, that would give the module more flexibility for the future and be able to support different plugins versions at the same time. Not sure if this will block this issue much longer though. When I needed to do my migration 4 years ago, that plugin system was definitely a big missing piece to be able to actually use that module.
- 🇳🇮Nicaragua dinarcon
For the new system, we can take a look at the https://www.drupal.org/project/migrate_upgrade → module. In addition to generating configuration entities like this module does, it also:
- Alters a subset of migrations. See
\Drupal\migrate_upgrade\MigrateUpgradeDrushRunner::applyFilePath
- Applies a prefix to all migrations: See
\Drupal\migrate_upgrade\MigrateUpgradeDrushRunner::modifyId
- Checks for existing migration IDs to avoid conflicts: See
\Drupal\migrate_upgrade\MigrateUpgradeDrushRunner::substituteIds
- Takes into account migration derivers: See
\Drupal\migrate_upgrade\MigrateUpgradeDrushRunner::expandPluginIds
For the sorting by weight logic, I suggest we leverage Core patterns: See
\Drupal\Component\Utility\SortArray
Something like this in
\Drupal\views\Plugin\views\filter\FilterPluginBase::buildGroupSubmit
uasort($group_items, ['Drupal\Component\Utility\SortArray', 'sortByWeightElement']);
There are more examples in
\Drupal\Tests\Component\Utility\SortArrayTest
Regarding min_api_version and max_api_version properties, does the WXR file contains such information? If we queried the database, we can probably find the version of all plugins installed. But I don't think that information is available in the WXR export. At least I could not see plugin version data in this reference file with Yoast data provided by @hongpong. I guess we would have to provide a way for the end user to indicate which version of a plugin they are using. Even auto-detecting which plugins were used in the WordPress site out of the content of the WXR file might be challenging.
Another observation is that the current wizard would play nice to interactively select and configure plugins to use. What would be the approach to take for generating the migrations with Drush? A single interactive command? Individual commands for each migration?
Some of the comments might be out of scope for the specific issue. But the plugin system would be a foundational part of the module and the way it is designed/implemented will have an impact at how other features are implemented.
- Alters a subset of migrations. See
- 🇺🇸United States hongpong Philadelphia
Thanks for these insights! My idea for a plugin API version is not based on embedded data in the WXR files. It is just a way to indicate if we have internally changed the processing within this module of posts, authors, comments significantly. And in turn it would also let the custom developers write another plugin manager which could accept different API version numbers for their own custom imports - this would let them run separate 'lanes' of imports within this open ended manager framework that we can implement.