Do not touch CiviCRM database when migrating Webform submissions

Created on 10 November 2022, about 2 years ago
Updated 18 July 2023, over 1 year ago

1.0 Problem/Motivation

During Drupal migrations (e.g. from Drupal 7 to Drupal 9), when migrating submissions of Webforms with the CiviCRM handler being active, all handlers perform their tasks as if during runtime, i.e. there is no distinguishing between creating a Webform submission during migration or not. Thus, the CiviCRM database is being altered by migrating Webform submissions when it shouldn't, since those submissions have already been accounted for when they were initially created on the old environment.

2.0 Detailed steps to reproduce

  • Migrate CiviCRM Webforms with the webform_civicrm module enabled on the target environment.

3.0 Proposed resolution

There seems to be no context passed around that the current operation of creating a Webform submission is during a migration (please prove me wrong!), so this might become tricky (e.g. evaluating backtraces).

A possible workaround would be migrating the CiviCRM database afterwards, overwriting any changes to the CiviCRM database that were made by this module during migration. However, there seems to be odds going on during migration causing errors due to webform_civicrm chiming in for each migrated Webform submission. We've seen Data too long errors for civicrm_log table entries getting things appended to over and over, which might, of course, not be related to the migration at all, but it led us to noticing that this module is actually altering the CiviCRM database during migration.

🐛 Bug report
Status

Active

Version

6.0

Component

CiviCRM Data Handling

Created by

🇩🇪Germany jensschuppe

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇨🇦Canada nubeli

    I can confirm that jensschuppe reported. Not only does it write to the CiviCRM tables, such as recreate activities, etc, it also will send out confirmation emails if a webform was used to subscribe to mailing groups. The latter can be handled by disabling outgoing emails during the migration, but it cleaner if there was a way to avoid all processing on a migration since the processing already happened the first time.

    Perhaps there's a way to temporarily disable the CiviCRM handler in a migration script. I've been using https://github.com/fuzionnz/webform_civicrm_migrate so perhaps it could be done there. This looks like a promising hook:

    hook_webform_handler_invoke_alter(\Drupal\webform\Plugin\WebformHandlerInterface $handler, $method_name, array &$args)
    

    . If the CiviCRM handler is disabled then there would need to be a submission handler specific for the webform_civicrm_submissions table, similar to D7WebformSubmission in the webform_migrate module.

  • 🇨🇦Canada nubeli

    In regards to this question by Karen:

    And why are you migrating webform submissions that have civicrm components? The key usage for webform module is to get data into CiviCRM so it's just a front end/method to do so.

    Our client is using it as semi-permanent storage. They have over 40 webforms connected to CiviCRM and some of them are tracking data long-term for users/contacts. And they've also got a webform with a component that references a content type that is also "owned" by the user. Users make yearly submissions that can then tracked in Views to show changes over time in user responses.

    So that's why we're making sure submissions get migrated over properly.

  • 🇨🇦Canada karing 🇨🇦

    I see you have connected with Luke from Fuzion on Mattermost re: https://github.com/fuzionnz/webform_civicrm_migrate - I agree that's the place for migration related issues.

  • 🇨🇦Canada nubeli

    Turns out even disabling the webform_civicrm handler doesn't quite prevent webform_civicrm from being called. https://www.drupal.org/project/webform_civicrm/issues/3375337 🐛 Disabling the webform_civicrm handler doesn't prevent it from being called Needs work

Production build 0.71.5 2024