🇺🇸United States @hongpong

regarding entity reference for images, we need this one to be committed - ✨ Import as media Needs review - i did a bit of refactor and hit a small snag with the iterator being a little messy earlier this month.

on the plus side string_long isn't that bad to include. ref: https://www.drupal.org/docs/drupal-apis/entity-api/fieldtypes-fieldwidge... → - https://gist.github.com/cesarmiquel/48404d99c8f7d9f274705b7a601c5554

WIP latest: I found the logic errors on the isset (it needed to consider strings of length 0 - avoiding empty() ) however things are hanging now.

I think missing attachments can also make it hang but that is a separate issue. ✨ attachments / featured images not fault tolerant if missing Active . Something is still mixed up but made incremental progress anyway.

hongpong → created an issue.

don't bother testing till the fields are saving imo.

the idea is that there is the combo field choice first (text_with_summary) that saves in the sub fields, or alternative, the options to split full body vs summary into two text fields (of both types, with and without summary). i see what you mean about showing a conditional field but this struck me as easier to execute.

re conditionals i am also wondering re the Text formats picker. as with some other steps in the wizard it doesn't check if that text format you picked is actually permitted on the text field you select. (likewise it doesn't check if the image field you pick is assigned to the node type). I don't want to treat this as a blocker because, with a little additional help text we can walk people thru it for now.

Okay here is the WIP on MR 42. https://git.drupalcode.org/project/wordpress_migrate/-/merge_requests/42

The data is definitely not saving anymore (under any configuration!) but I think the wizard UI and form submit validator is pretty good. Back to the drawing board again later - some subtle improvement should get it working. (this was mainly generated by Claude AI).

Hmm I am getting similar errors in the attachments step as well. I had the Claude AI do a little investigation and am attaching one date type safety process plugin it gave me, but the problem is actually a missing ID value.

Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: '' for column `db`.`migrate_map_myjapan2wordpress_attachments`.`sourceid1` at row 1: INSERT INTO "migrate_map_myjapan2wordpress_attachments" ("source_ids_hash", "sourceid1", "source_row_status", "rollback_action", "hash", "last_imported") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array
(
[:db_insert_placeholder_0] => 02db060eff943c9d3e99f79dd573e1e2ab0d52d4dd041f59a0bc63f453f0af82
[:db_insert_placeholder_1] => 
[:db_insert_placeholder_2] => 3
[:db_insert_placeholder_3] => 0
[:db_insert_placeholder_4] => 
[:db_insert_placeholder_5] => 1744081739
)
in Drupal\migrate\Plugin\migrate\id_map\Sql->saveIdMapping() (line 711 of core/modules/migrate/src/Plugin/migrate/id_map/Sql.php). Drupal\Core\Database\StatementWrapperIterator->execute() (Line: 34)
Drupal\mysql\Driver\Database\mysql\Insert->execute() (Line: 372)
Drupal\Core\Database\Query\Merge->execute() (Line: 711)
Drupal\migrate\Plugin\migrate\id_map\Sql->saveIdMapping() (Line: 54)
Drupal\migrate_tools\IdMapFilter->saveIdMapping() (Line: 226)
Drupal\migrate\MigrateExecutable->import() (Line: 236)
Drupal\migrate_tools\MigrateBatchExecutable::batchProcessImport() (Line: 297)
_batch_process() (Line: 139)
_batch_do() (Line: 95)
_batch_page() (Line: 52)
Drupal\system\Controller\BatchController->batchPage()
call_user_func_array() (Line: 123)

I generated a safe date checker plugin using Claude as a basis to start resolving this, however the issue continues. The ai reported the following theory:

Looking at this error, the issue isn't with the date processing but with the source ID mapping in your migration. This is a different problem than what our SafeDateProcess plugin was designed to solve.
The key part of the error is:
SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: '' for column `db`.`migrate_map_myjapan2wordpress_attachments`.`sourceid1` at row 1
This indicates that:

The database is expecting an integer value for the sourceid1 column in the migration map table
It's receiving an empty string ('') instead

This happens because:

In your migration configuration, you've defined post_id as the ID field:
ids:
post_id:
type: integer

But for some attachment items in your XML, this field is either missing or empty, so the migration tries to use an empty string as an ID value

Here are potential solutions:

Filter out items without IDs - Add a process step to skip items with no post_id:
process:
post_id:
plugin: skip_on_empty
method: row
source: post_id

Add a default ID - If the items without IDs are still valid attachments:
process:
post_id:
plugin: default_value
source: post_id
default_value: 0 # Or generate a unique negative number

Fix the source data - Inspect your WordPress XML export and ensure all attachments have valid post IDs
Use a different ID field - If there's another unique identifier that's more reliable:

ids:
attachment_url:
type: string

The error occurs at the ID mapping stage, which happens earlier in the process than date transformation, so our SafeDateProcess plugin isn't even getting a chance to run in these cases because the migration is failing at the ID mapping step.

I think it is reasonable to
- make $migrate the MigrateEntity optional by putting it within the $opts array param. this follows a lot of patterns where there is a free form array as a parameter.
- add to migration extension plugin manager base class, a method return boolean to validate if the $migrateEntity is present in the $opts. if true proceed with the alter workflow, and if false, it is for preprocessing or another context where the migrateEntity doesn't exist.
- i think "wordpress_type" as the other plugin alter action string param name could be changed to "wp_alter_stage" better communicates this applies to some stage of the migration. (eg post_comments, tags, xml_preprocess, etc) ..

in this fashion all the plugins can be grouped together as one system and provide full flexibility to any point in the wizard import process. (as opposed to a separate plugin system just for preprocessors). so limits complexity by making the plugin param nested within the $opts array).

in addition 'weight' of the plugins needs to be implemented still, none of that is in.

reverting to active until another patch. xpete reported 📌 Automated Drupal 11 compatibility fixes for wordpress_migrate Needs review :
I tested the 8.x-3.x branch again with version Drupal 10.4.6 and Drupal CMS 1.1.0. It's fixed on version 10.4.6 but with Drupal CMS the Blog and Page fields are not visible on the body field picker so the body still can't be imported.
Drupal CMS don't use text_with_summary. It uses regular text fields. You can read a discussion about it here https://www.drupal.org/project/drupal/issues/3477043 📌 Change automatic body field creation to use formatted text field instead of text_with_summary Active
////
Therefore i would propose that the UI be revised with additional selectors for (text,formatted,long). I think the first dropdown should say it is for "content and excerpt/summary, essentially two fields in one" in the UI help. Then a second and third dropdown, which would select both (text,formatted,long) and (text, formatted, summary) - if selecting the latter no content would go in summary.

then the third dropdown allows selecting both (text,formatted,long) and (text, formatted, summary) fields as well.

The submit or validator should enforce each field selected is unique.

in processing the body fields, now the @_body/summary needs to be associated to that field and @_body/content to the selected one. it should be OK to leave one both or all unselected. (currently it is OK and processes OK i believe).

also i think it could be OK, but would take more trickery, to allow the first dropdown to put into (text, formatted, summary), AND allow the latter two fields to split off either / both body and summary into other text,formatted,long) and / or (text, formatted, summary) fields. in other words there would have to be more implementation to save in both, but i think it is OK to dupe stuff at migrate time that way.

Otherwise (not preferred), the field design could disable 2+3 if 1 is selected anything and vice versa at the front end, and enforcing similar at submit time.

OK xpete I will re open the other issue ✨ Add body field picker w default to wizard & processing Active to address the body field issue. I would still like to add some exception handling here on this issue before closing it

re what else goes in next alph tagging the release: assuming nobody is going to nudge ahead on ✨ Support contrib wordpress data / custom data | Yoast SEO Needs review which is close but I am trying to extend its functionality to preprocess -- and that is not a 11.x blocker at all so it should not hold up tagging for 11.x. i think this body field thing ought to be done before tagging the release tho.

Thank you for testing xpete! can you confirm if it is a "text_with_summary" type of body field? (screenshot if needed).

Also i created a Drupal CMS "recipe" module here last week but not tested it yet - i don't have a good drupal CMS test process https://www.drupal.org/project/wordpress_migrate_recipe →

Work in progress. Hopefully this is not a silly direction to take this.
- I have changed the alterMigration to a static function alterMigrationViaPlugin in WordPressMigrationGenerator. Added params to remove $this configuration array reference. Now allows a dummy array instead of a Migrate element.
- trying to 'sneak in' a way to manage XML preprocess plugins within the overall set of plugins, (don't want to make a second plugin manager)
- preprocessor can just throw exceptions ahead of showstopper xml corruption, or edit the xml file right at this wizard step, like find-replace strings etc etc.
- Added an 'opts' array parameter for passing any hodgepodge options.
- For preprocessor Will put the uploaded file path in there keyed like 'xml_filepath'
- Added weight integer to the plugin interface with default value 0.
- Added stub getDefinitionsPreProcess() to WordpressMigrationExtensionPluginManager
- Added XmlPreProcessValidXMLExtension.php to the Extension plugins
- Added throw exceptions in case of various mistakes.
- Updated some stuff like weight and known file handler deprecation issue note to YoastPluginExtension

Need to sort processing by weight, add an API version, and test further. Just noticed we have inconsistent caps re WordPress in the code filenames. There is also an issue re MigrateInterface vs EntityInterface needs to be sorted out - various locations in code are causing it to use the latter when we want the former, or cast for type safety if needed.

Ok tested on 11.1 and 10.4 I think this is ready to go. Includes UI improvements on form elements, BodyFieldSelectForm, static function getBundleFieldnameOptionsFromFieldmap on importwizard to make field selectors work better. changes to wordpress_content.yml, the migration generator body field handling. it was a lot of improvements. really solid.

It also became apparent that the /other/ steps do not force the field to be assigned to the content node - it lets you pick any field regardless of bundle. so to clarify that more UI messages added for now (could be enforced better later by tweaking the getBundleFieldnameOptionsFromFieldmap to cases there there is no "drupal_content_type" selected, that is fields which are being assigned for the entire migration group, not just one content type. or, they could be reorganized and split per content type (post/page) fully. body field does work this way now, in any case that pattern could be followed, or a second migration group created).

I think this covers our main D11 release blocker but would be great to get plugins migration extensions manager working as well.

hongpong → created an issue.

hongpong → created an issue.

got welcome assistance from mikelutz and benjifisher on slack:: notes:

Even looking at MR 40, I am not sure what the issue is.
In the MR, I see a form where the user gets to select a field. From the help text, the idea is to migrate the main content from the WP post into that field.
What happens after that? Is the strategy to create a migrate_plus.migration.* config entity based on the choices made in that form?
Another option would be to use hook_migration_plugins_alter() to change the field based on the choices made in that form.

At a quick glance, I think you are on the right track. Here’s the fixes you need:
In the wordpress_content yml file, change
'body/value':
- plugin: get
source: content
'body/summary':
- plugin: get
source: excerpt
to
_content/value: content
_content/summary: excerpt

And in the generator change

$process['body_field'] = $this->configuration[$wordpress_type]['body_field'];
// @todo more body field
// body/summary = 'excerpt'
// body/value = 'content'

$process['body/format'] = [
'plugin' => 'default_value',
'default_value' => $this->configuration[$wordpress_type]['text_format'],
to
$process['_content/format'] = [
'plugin' => 'default_value',
'default_value' => $this->configuration[$wordpress_type]['text_format'],
$process[$this->configuration[$wordpress_type]['body_field']] = '@_content';

You may want some constraints or error checking around $this->configuration[$wordpress_type]['body_field'] to make sure you aren’t overwriting any other processes, I’m not sure, I don’t have a big picture of the module’s workings, but the above changes should do what you are trying to do from what I can see at a quick glance.

But that should work whether $this->configuration[$wordpress_type]['body_field'] ends up being ‘body’ or some other long format text field.

If you want to default to body if the configureation is null, you can add a default by changing $process[$this->configuration[$wordpress_type]['body_field']] = '@_content'; to $process[$this->configuration[$wordpress_type]['body_field'] ?? 'body'] = '@_content'; or something like that, but it looks like your from will return body if that’s where the destination should be. Looks like it has an option to not import it anywhere though, so wrap that generator section in an if (!empty($this->configuration[$wordpress_type]['body_field'])) check probably.

Updated this to 8.x-3.x-HEAD. Still reasonable to use this issue for improving exceptions and logging. There are various comments that would be removed. Also the logger should be improved still.

Thanks @baltowen for sorting this thing out!!

Rebased / merged the MR to current head 8.x-3.x . I think this issue should be applied before ✨ Add body field picker w default to wizard & processing Active which would add an additional step to the wizard.

Hi all I have rerolled the MR16 to match the current 8.x-3.x head. I think it is very promising for sure.

The WIP is currently pretty close to what is needed (only tested on 11.1). inside function createContentMigration something similar to the comment set , work on $process['body_field'] . If anyone else wants to fiddle with the body field assignment in migrate entity set stage, I think that is the last major thing to resolve.

I think it is correctly auto selecting 'body' in Drupal 11.1 if that is the name of the body field.

Marking it as a major, I think we need this fixed to deal with Drupal CMS.

The form has a kind of warning message on it if the field is unselected. It should tolerate being unselected without killing the migration (which is the current case with 8.x-3.x HEAD) but we want to avoid this.

K I tried to extract the media field loop however I think I missed the mark. I also extracted the MediaTypes array to the properties of the class.

Warning: Undefined array key "plugin" in Drupal\migrate\Plugin\Migration->findMigrationDependencies() (line 685 of core/modules/migrate/src/Plugin/Migration.php). (and line 688). Appears after specifying all the fields at urls like - : /admin/structure/migrate/manage/my_wp_firstmedia1/migrations - the imports are "no data found".

Drupal\migrate\Plugin\Migration->findMigrationDependencies() (Line: 655)
Drupal\migrate\Plugin\Migration->getMigrationDependencies() (Line: 124)
Drupal\migrate\Plugin\MigrationPluginManager->createInstances() (Line: 93)
Drupal\migrate\Plugin\MigrationPluginManager->createInstance() (Line: 132)
Drupal\migrate_tools\Controller\MigrationListBuilder->buildRow() (Line: 272)
Drupal\Core\Entity\EntityListBuilder->render() (Line: 23)
Drupal\Core\Entity\Controller\EntityListController->listing()
call_user_func_array() (Line: 123)

On Drupal 11.1.5. Tried to extract as.

foreach ($media_types as $media_type_id => $media_type) {
        $config_key = $media_type_id . '_media_type';
        if ($this->configuration[$config_key]) {
          $this->CreateSingleMediaTypeMigration($media_type['migration_id'], $media_type_id, $process, $config_key);
        }
      }

and new function

  public function CreateSingleMediaTypeMigration($migration_id, string $media_type_id, $process, string $config_key): void {
    $this->migrateInfo[$migration_id] = $this->configuration['prefix'] . 'wordpress_media_' . $media_type_id;
    $migration = $this->createEntityFromPlugin('wordpress_media_' . $media_type_id, $this->migrateInfo[$migration_id]);
    $migration->set('migration_group', $this->configuration['group_id']);
   ...
    $migration->save();
    unset($process, $migration);

Added the unset, trying to make a more clean function.

I muddled things and needed to put in a rebase on latest 8.x-3.x HEAD after Drupal 11 support added and refactored WordPressMigrationGenerator. going to refactor a bit to make these functions smaller. Pulled the previous MR into MR39: https://git.drupalcode.org/project/wordpress_migrate/-/merge_requests/39

Re upping what abarrio suggested if anyone wants to get in:

• Test each media type to check that is completely working automatically without changing yml
• Fix coding standard of all new code
• Maybe be able to check if xml file is completely correct to avoid problems when importing in recent days ( I started a new module to cover this - https://gitlab.com/HongPong/xml_vuln_detector ) - not needed to do this now imo
• Create some unit tests
• Fix all problems that appears on merges

I think we can bring the patch in. I am testing in Drupal 10.4 11.0 11.1 to finally put it to bed to this point.

However re xpath note I believe we need a better body field picker in Drupal 11.1 (similar to the other field pickers in the wizard) . New issue added ✨ Add body field picker w default to wizard & processing Active . It works in Drupal 10.4 and 11.1 with the default 'body' field but it doesn't warn the user if this situation isn't connected.

hongpong → created an issue.

hongpong → created an issue.

Great call thank you abarrio! Appreciate your patience and help on this.

hongpong → made their first commit to this issue’s fork.

Ah i saw your comment on the other thread - tags and categories were missing. So at least, I did not get confused and now we have a good logger. 💬 Import Taxonomy URL Alias Needs review .

- Readme should get updated later
- Duplicate terms should be prevented (could happen with matching term IDs or human name conflicting I think)
- Maybe we can get a garbled entries with multiple imports colliding and not preventing duplicates.

I think i will call this closed / fixed for now since we addressed the sample here.

Say ryumaou ticsmart, I compared your file (thanks again for this). Per migrations/wordpress_categories.yml and wordpress_tags.yml there are source item_selector:

source:
  plugin: url
  data_fetcher_plugin: file
  data_parser_plugin: xml
  urls: []
  item_selector: /rss/channel/wp:category

and

source:
  plugin: url
  data_fetcher_plugin: file
  data_parser_plugin: xml
  urls: []
  item_selector: /rss/channel/wp:tag

There should be statements like

<wp:tag>
  <wp:term_id>651</wp:term_id>
  <wp:tag_slug>articles</wp:tag_slug>
  <wp:tag_name><![CDATA[articles]]></wp:tag_name>
</wp:tag>
<wp:tag>
<wp:category>
  <wp:term_id>1</wp:term_id>
  <wp:category_nicename>uncategorized</wp:category_nicename>
  <wp:category_parent/>
  <wp:cat_name><![CDATA[Uncategorized]]></wp:cat_name>
</wp:category>

And there are not any of those in the sample file provided. There are references to tags like these in your file. but note how it would not have a term_id and the element is not wp;tag or wp:category. So it is not selected.

										<category domain="post_tag" nicename="creativity"><![CDATA[creativity]]></category>
		<category domain="post_tag" nicename="entertainment-culture"><![CDATA[entertainment culture]]></category>
		<category domain="post_tag" nicename="human-interest"><![CDATA[Human Interest]]></category>
		<category domain="category" nicename="on-creativity"><![CDATA[On Creativity]]></category>
		<category domain="post_tag" nicename="paul-zii"><![CDATA[Paul Zii]]></category>
		<category domain="post_tag" nicename="photography"><![CDATA[photography]]></category>
		<category domain="post_tag" nicename="pinhole-camera"><![CDATA[pinhole camera]]></category>
		<category domain="post_tag" nicename="psychology"><![CDATA[psychology]]></category>
		<category domain="post_tag" nicename="stay-creative"><![CDATA[Stay Creative]]></category>
						<wp:postmeta>

@wotts can you say where you changed this? was it in wordpress_attachments.yml at :

source:
  plugin: url
  data_fetcher_plugin: file
  data_parser_plugin: xml
  urls: []
  item_selector: '/rss/channel/item[wp:post_type="attachment"]'

Thank you.

Hello all, I tried installing colorbox on the current 11.1.x version and having it display a media field (not an image field). It seems the menu options for the display widget are not clearly described anywhere I could find nor info on best practices for attaching to a media field, so those two aspects would be the most helpful.

well this worked great anyway. call it fixed imo. well done.

hongpong → created an issue.

fixing intro link slightly

slight brush up on wordpress migrate section.

cool thank you ryumaou. i'm going to block in late Jan or early Feb another sprint on this module for a few days so anything else you can nail down helps a ton.

Regardless of the 'nice to have' like xml check, unit tests and certain code standard flags, I think we need to be assured that the migration is working correctly for most use cases.
Forgive my silly question but is "document" like PDF files and things of that nature? In other words if someone has added a plugin that allows the PDF uploads in the media library, this would cover importing those. I'm a little more familiar with the built in audio content type.

regarding checking the xml file see #3248042: Add XML Lint to check the WordPress exports → xml lint could be added to composer but i figured that would intensify requirements. as well as ✨ WP import xml entities & security considerations Active for a note on other considerations.

regarding testing media types I suppose this would require a new test site altogether with the types of assets in question.

if you think we can subdivide out any portion of the patch as well (in other words if you think some standalone portion would actually be RTBC as it is), let me know. Thanks for all this!

excellent stuff abarrio. do you think it is RTBC yet? I don't have a test xml with audio attachment to test.

Thanks so much abarrio for rebasing. Any ideas what we can do to fix the "Access to an undefined property" stuff flagged above? That seems like the biggest flagged snag currently.

Maybe this needs to be added - https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Plugi...

It could be that the title of the tags are longer than the taxonomy entity title field permits? I think this would work for trimming the tag name.

Note we do not have substr as a process here: https://git.drupalcode.org/project/wordpress_migrate/-/blob/8.x-3.x/migr...

the max taxonomy tag title length might be 255 chars https://www.drupal.org/forum/support/post-installation/2009-10-22/overri... →

could help determine if this is the case: https://www.drupal.org/project/title_length →

programmatic checker idea here: https://drupal.stackexchange.com/questions/107758/how-do-i-find-the-max-...

@ryumaou yes if you can test it it might help with 🐛 Tags and categories not migrating Active as well. Sorry I haven't had more time to hack on this one lately (or yours). I also wonder if the issue is similar to #2955482: Pingbacks are imported as comments - nodes with random names are created → which puzzled me from way back.

Regarding catching exceptions which I think is generally a good idea and best practice. Just adding some links here to refer to later, re best practices and how other modules are doing it, what is around in the migrate subsystem and so on.

I had some old WIP patches on a couple issues in the project to add these - linked below. The API for exceptions and watchdog logging changed after 10.1.x so it is good we are switching to 10.2.x minimum.

• On branches 9.x to 11.x: https://api.drupal.org/api/drupal/namespace/Drupal%21migrate%21Plugin%21...
• badplugindefinitionexception: https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Plugi...
• https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Migra...
• https://api.drupal.org/api/drupal/namespace/Drupal%21migrate%21Exception... includes EntityValidationException and RequirementsException
• watchdog_exception was deprecated https://www.drupal.org/node/2932520 → . A number of classes that were using watchdog_exception() now explicitly inject a logger to be able to pass to \Drupal\Core\Utility\Error::logException()
• example implementation: https://git.drupalcode.org/project/webform/-/merge_requests/406/diffs
• https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Utility%2...

Code for adding log entries should now be like:

<?php
use \Drupal\Core\Utility\Error;

$logger = \Drupal::logger('modulename');
Error::logException($logger, $exception);
?>

• Exceptions also discussed in: 💬 XML (XHR) invalid characters issue Postponed: needs info within wordpress_migrate. another test I made WIP: 🐛 Duplicative generated migration group causes uncaught error at review stage EntityStorageException Needs work . using exceptions with the pluggable interface - some ideas: ✨ Support contrib wordpress data / custom data | Yoast SEO Needs review . duplicate user exception: #3123393: Users are duplicated (Integrity constraint violation) Duplicate entry → . some notes here similar: 🐛 Better exception handling for if username does not exist Active
• Nice patch for showing lines of errors: https://www.drupal.org/files/issues/2020-07-30/2969551-25.patch → - via #2969551: Migrate messages from caught exceptions need file and line details →
• Way to do badplugindefinitionexception: #2908282: Throw exception for source plugins without a source_module property → see patch https://www.drupal.org/files/issues/2908282-83.patch →
• Main doc page about exceptions in coding standards: https://www.drupal.org/node/608166 →
• Old forum thread, adding this because it uses "throwable" as a catchall - https://www.drupal.org/forum/support/module-development-and-code-questio... →
• Example of how it was patched for 10.1+ this year in webform: 📌 [Drupal:10.1.x] watchdog_exception() is deprecated in drupal:10.1.0 and is removed from drupal:11.0.0 Fixed

Thanks all!

Thanks for this!!

thank you!

hongpong → made their first commit to this issue’s fork.

hongpong → made their first commit to this issue’s fork.

thank you for checking on this!

Thanks for filing this. Yeah let's get the xml posted to test against .. if it doesn't allow the upload I can be emailed as well ` hongpong @ hongpong.com ` - if you like I can add it to that outside Repo for test XML files I made as well. Thanks!

Might want to check this out as well for debugging - https://www.drupal.org/project/migrate_devel →

Thanks for these insights! My idea for a plugin API version is not based on embedded data in the WXR files. It is just a way to indicate if we have internally changed the processing within this module of posts, authors, comments significantly. And in turn it would also let the custom developers write another plugin manager which could accept different API version numbers for their own custom imports - this would let them run separate 'lanes' of imports within this open ended manager framework that we can implement.

thanks baltowen very good stuff. Regarding the equals signs, there are some silly rules in phpcs that already exist. From today's HEAD (not your patch). I

Drupal.Formatting.MultipleStatementAlignment.NotSame
Drupal.Formatting.MultipleStatementAlignment.Incorrect

 197 | ERROR | [x] Equals sign not aligned correctly; expected 1 space but
     |       |     found 16 spaces
     |       |     (Drupal.Formatting.MultipleStatementAlignment.Incorrect)
 261 | ERROR | [x] Equals sign not aligned with surrounding assignments;
     |       |     expected 21 spaces but found 1 space
     |       |     (Drupal.Formatting.MultipleStatementAlignment.NotSame)
 267 | ERROR | [x] Equals sign not aligned with surrounding assignments;
     |       |     expected 27 spaces but found 1 space
     |       |     (Drupal.Formatting.MultipleStatementAlignment.NotSame)

I think the alignments are very silly for them to impose on us. phpcs.xml.dist (or maybe without .dist) could perhaps override this in Drupal flavored GitlabCI within this project?
Using statements like https://git.drupalcode.org/project/drupal/blob/8.8.x/core/phpcs.xml.dist

  <rule ref="Drupal.Arrays.Array">
    <!-- Sniff for these errors: CommaLastItem -->
    <exclude name="Drupal.Arrays.Array.ArrayClosingIndentation"/>
    <exclude name="Drupal.Arrays.Array.ArrayIndentation"/>
    <exclude name="Drupal.Arrays.Array.LongLineDeclaration"/>
  </rule>

see #2994956: Excluding sniffs from phpcs →

I think looking ahead we want to make sure these are caught not just in the UI wizard but also in the migration generator which could be later triggered from drush (after drush 12 is implemented!). In other words malformed elements could be coming in that UI measures cannot prevent. This was flagged by mikeryan on one of the oldest issues. #2742283: Validate configuration in WordPressMigrationGenerator::createMigrations → ..

to be fair this issue could be postponed until we have drush working again. ✨ Implement Drush 12+ on wordpress_migrate Active . I think adding exceptions, like in the WIP here, is still a good idea when entities are being stored and similar.

wordpress_migrate Alpha7 is released, thank you everyone! Release note: https://www.drupal.org/project/wordpress_migrate/releases/8.x-3.0-alpha7 →

I had a couple ideas to add in the mix. Hopefully this is not excessive. I think if we can future proof this, by adding a couple parameters to the plugin loader, and WordpressMigrationExtensionPluginInterface.php and / or src/Plugin/WordpressMigrationExtensionPluginBase.php. Additional possible arguments for extension plugins:
$api_version (int), $extra_args (array), $suggested_weight (int). The purpose of $extra_args would be a catch all that could provide special directives outside of other configuration objects, for now just an empty array. (this is kind of common in WordPress already.) $suggested_weight could be used to direct the sorting of these plugins in their execution order, which right now seems to be arbitrary. (e.g. higher weight executes last, or whatever. If the weights match then sort by name?)

In the properties of WordpressMigrationExtensionPluginManager.php it could be added, static int for min_api_version, max_api_version and a static function, isApiVersionSupported(int) that returns true if the int is within the supported range. (This would let other modules poll to make sure that they are compatible.)

Thus if the attempted plugin loading is coded with a higher API number then this WordpressMigrationExtensionPluginManager will not load it, or it can throw an exception if it tries to execute. This could also help people who are making custom migrations privately, they could set up their own API number ranges for internal use. (This seems like a cleaner solution than using the Semver of our module to enforce this kind of thing).

Second idea is to add a preprocessor plugin manager that works similarly to this plugin manager. It could be triggered at the end of SourceSelectForm.php: submitForm, and similarly could load the entire uploaded file to a "WordpressMigrationPreprocessorExtensionPlugin" (perhaps selected from another manager wizard page) before the process moves on to AuthorForm.php . I figure this would be the best way to add an XML linter or other security measures ( #3248042: Add XML Lint to check the WordPress exports → ). It could look for encoded XML entities or other oddball or security issues. The plugin manager could also have static int properties for min_preprocessor_api_version, and max_preprocessor_api_version, and a static bool function, isPreprocessorApiVersionSupported(int).

The point with the API versions is that the execution of the plugins processes could change in the future and this would give the reliability of linking an API number to known implementations without locking us into it, and with a minimum API number we can safely jettison old APIs years down the road. I feel like the people making custom extensions for private projects would also benefit from some ideas like this. Stop me if this seems unnecessary or cumbersome but I think it could work well for everyone.

hongpong → created an issue.

hongpong → created an issue.

hongpong → created an issue.

added note re getting pluggable nature refactor on this ✨ Support contrib wordpress data / custom data | Yoast SEO Needs review

thanks everyone for collaborating on this thing and particularly ressa for revamping the documentation entirely. it was a huge volunteer effort. if nobody has any errors to flag i should tag the alpha7 release today or tomorrow . I have a decent release note already drafted. Excellent work everyone and thank you!!!

change to describe how sandboxes are superseded and lead them to the new docs

add links to "using gitlab to contribute to drupal" which has additional info

fixing these links that were broken and clarifying some things

i may have messed up the commits in the MR just now - however the MR should be up to date with 8.x-3.x HEAD and your patch as well .. i think we have all our pre alpha7 commits in now? thanks for all this.

user duplicates and migration group duplicates have been fixed in other issues so this issue is shrinking away...

postponing as we need reproducible XML data to test against.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it. There is now a WordPress SQL module we have it linked on the front of this module.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.

Closing this as we move on from 7.x. Feel free to open again if anyone works on it.