DatabaseExceptionWrapper: Invalid datetime format

Created on 31 March 2025, 24 days ago

Problem/Motivation

When importing the Japan test xml file ( https://gitlab.com/HongPong/wordpress-test-imports/-/blob/v1.0.0/wp-import-9-japan-2013.xml?ref_type=tags ) there is a fatal error.

It is part of the \id_map\Sql->saveIdMapping and migrate_tools\IdMapFilter->saveIdMapping() step. Probably this needs a fallback when there is an invalid value.

The website encountered an unexpected error. Try again later.Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: '' for column `db`.`migrate_map_japan_oldv_2bwordpress_content_page`.`sourceid1` at row 1: INSERT INTO "migrate_map_japan_oldv_2bwordpress_content_page" ("source_ids_hash", "sourceid1", "source_row_status", "rollback_action", "hash", "last_imported") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array

(
[:db_insert_placeholder_0] => 02db060eff943c9d3e99f79dd573e1e2ab0d52d4dd041f59a0bc63f453f0af82
[:db_insert_placeholder_1] => 
[:db_insert_placeholder_2] => 3
[:db_insert_placeholder_3] => 0
[:db_insert_placeholder_4] => 
[:db_insert_placeholder_5] => 1743449089
)
in Drupal\migrate\Plugin\migrate\id_map\Sql->saveIdMapping() (line 716 of core/modules/migrate/src/Plugin/migrate/id_map/Sql.php). Drupal\Core\Database\StatementWrapperIterator->execute() (Line: 44)
Drupal\mysql\Driver\Database\mysql\Insert->execute() (Line: 385)
Drupal\Core\Database\Query\Merge->execute() (Line: 716)
Drupal\migrate\Plugin\migrate\id_map\Sql->saveIdMapping() (Line: 54)
Drupal\migrate_tools\IdMapFilter->saveIdMapping() (Line: 272)
Drupal\migrate\MigrateExecutable->import() (Line: 236)
Drupal\migrate_tools\MigrateBatchExecutable::batchProcessImport() (Line: 297)

it is possible this is an encoding error. It may just be this file is glitchy but I have tried running it before, not sure the issue.

I ran into this when testing πŸ“Œ Automated Drupal 11 compatibility fixes for wordpress_migrate Needs review but found that the same exact error is happening on head.

Steps to reproduce

After importing tags and categories this occurs. with migrate_tools and migrate_plugs 6.0.5 on Drupal 10.4.5

The "page" will import 18 of 19 and list one unprocessed. The "post" will list 39 of 42 imported with 1 unprocessed.

Proposed resolution

Rewrite the function to tolerate whatever the issue / constraint is. Could use migrate_devel to inspect more closely.

It would also be good to catch and log this error instead of a 500 halt.

Also would be good to pass in a migration "message" here, and/or a Logger log.

Remaining tasks

  • Patch to tolerate the value
  • User interface changes

    Maybe something in the readme or notice. etc

    API changes

    None.

    Data model changes

πŸ› Bug report
Status

Active

Version

3.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States hongpong Philadelphia

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @hongpong
  • πŸ‡ΊπŸ‡ΈUnited States hongpong Philadelphia

    Hmm I am getting similar errors in the attachments step as well. I had the Claude AI do a little investigation and am attaching one date type safety process plugin it gave me, but the problem is actually a missing ID value.

    Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: '' for column `db`.`migrate_map_myjapan2wordpress_attachments`.`sourceid1` at row 1: INSERT INTO "migrate_map_myjapan2wordpress_attachments" ("source_ids_hash", "sourceid1", "source_row_status", "rollback_action", "hash", "last_imported") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array
    (
    [:db_insert_placeholder_0] => 02db060eff943c9d3e99f79dd573e1e2ab0d52d4dd041f59a0bc63f453f0af82
    [:db_insert_placeholder_1] => 
    [:db_insert_placeholder_2] => 3
    [:db_insert_placeholder_3] => 0
    [:db_insert_placeholder_4] => 
    [:db_insert_placeholder_5] => 1744081739
    )
    in Drupal\migrate\Plugin\migrate\id_map\Sql->saveIdMapping() (line 711 of core/modules/migrate/src/Plugin/migrate/id_map/Sql.php). Drupal\Core\Database\StatementWrapperIterator->execute() (Line: 34)
    Drupal\mysql\Driver\Database\mysql\Insert->execute() (Line: 372)
    Drupal\Core\Database\Query\Merge->execute() (Line: 711)
    Drupal\migrate\Plugin\migrate\id_map\Sql->saveIdMapping() (Line: 54)
    Drupal\migrate_tools\IdMapFilter->saveIdMapping() (Line: 226)
    Drupal\migrate\MigrateExecutable->import() (Line: 236)
    Drupal\migrate_tools\MigrateBatchExecutable::batchProcessImport() (Line: 297)
    _batch_process() (Line: 139)
    _batch_do() (Line: 95)
    _batch_page() (Line: 52)
    Drupal\system\Controller\BatchController->batchPage()
    call_user_func_array() (Line: 123)
    

    I generated a safe date checker plugin using Claude as a basis to start resolving this, however the issue continues. The ai reported the following theory:

    Looking at this error, the issue isn't with the date processing but with the source ID mapping in your migration. This is a different problem than what our SafeDateProcess plugin was designed to solve.
    The key part of the error is:
    SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: '' for column `db`.`migrate_map_myjapan2wordpress_attachments`.`sourceid1` at row 1
    This indicates that:

    The database is expecting an integer value for the sourceid1 column in the migration map table
    It's receiving an empty string ('') instead

    This happens because:

    In your migration configuration, you've defined post_id as the ID field:
    ids:
    post_id:
    type: integer

    But for some attachment items in your XML, this field is either missing or empty, so the migration tries to use an empty string as an ID value

    Here are potential solutions:

    Filter out items without IDs - Add a process step to skip items with no post_id:
    process:
    post_id:
    plugin: skip_on_empty
    method: row
    source: post_id

    Add a default ID - If the items without IDs are still valid attachments:
    process:
    post_id:
    plugin: default_value
    source: post_id
    default_value: 0 # Or generate a unique negative number

    Fix the source data - Inspect your WordPress XML export and ensure all attachments have valid post IDs
    Use a different ID field - If there's another unique identifier that's more reliable:

    ids:
    attachment_url:
    type: string

    The error occurs at the ID mapping stage, which happens earlier in the process than date transformation, so our SafeDateProcess plugin isn't even getting a chance to run in these cases because the migration is failing at the ID mapping step.

Production build 0.71.5 2024