Migration from Drupal 7 to Drupal 9

Hi @JackG102 I am also trying to migrate TFA data from Drupal 7 to Drupal 9.

Were you able to migrate the data of TFA? Thanks.

I've pushed an initial prototype of this to the issue-fork.

I have not yet run a full end-to-end test on this yet as my d7 and d8 labs are not currently setup with the same cryptography.

This code at the moment is mostly provided to a base to jump-start discussion. Ideally sites should have been off D7 before Aug 1 deadline when D.O. and the DST cut support going forward, however without having a migration in place we certainly have contributed to sites missing that transfer date.

We do need some form of migration as none of this is stored in the D7 equvilient of user_data and without migrating the token data this isn't just "users need to log in again" this is "users need to setup their tokens again" which would be a security failure.

Key items as this sample exists now:

• No config is imported, TFA must be manually configured in D8+. Import of config may be doable I just have not looked into it since the real necessity as noted in #3 is that we get the token data.
• It is required that the same encryption is used on the source D7 install and the destination D8+ install as this makes no attempt to rotate encryption settings. We might need to check with the Encrypt module on if they plan to create a process plugin for encryption rotation during migrations.
• No attempt to import the trusted browsers has yet been made. This also may be doable however since the impact of not doing so is just a UX requirement that a user is prompted to provide a token it may be a much lower priority.

Concerns:
The tfa_basic module doesn't appear to list the TOTP plugin or the Trusted Browser plugin as a configured plugin in its users data field so when we are currently not importing that they are enabled. This may require a consult with the tfa_basic maintainer to determine how to detect and support.

I too have this requirement and I have worked on the logic for the data migration from D7 to D10. But I am having some issue with encryption. I would like to have some input on the error I am getting.

I have made a contributed module for TFA Migration → I would like to someone other than me to test it and provide input. Currently the module has functionality to migrate TFA Seed and TFA Users Settings.

My Workflow is as follows :

Get the TFA Seed from D7 and decrypt it using the private key from D7 and Encrypt it with the Key from D10.

I am able to create the functionality to process this. I am getting the correct seed size in D10 but seems decryption and encryption logic need correction as I am getting below error on D10 once form is submitted after OTP is enter in the site.

The website encountered an unexpected error. Please try again later.

RangeException: Base32::doDecode() only expects characters in the correct base32 alphabet in ParagonIE\ConstantTime\Base32::doDecode() (line 421 of /var/www/html/vendor/paragonie/constant_time_encoding/src/Base32.php).
ParagonIE\ConstantTime\Base32::decodeUpper('') (Line: 81)
ParagonIE\ConstantTime\Encoding::base32DecodeUpper('') (Line: 266)
Drupal\tfa\Plugin\TfaValidation\TfaTotpValidation->validate('405072') (Line: 222)
Drupal\tfa\Plugin\TfaValidation\TfaTotpValidation->validateForm(Array, Object) (Line: 273)
Drupal\tfa\Form\EntryForm->validateForm(Array, Object)
call_user_func_array(Array, Array) (Line: 82)
Drupal\Core\Form\FormValidator->executeValidateHandlers(Array, Object) (Line: 275)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object, 'tfa_entry_form') (Line: 118)
Drupal\Core\Form\FormValidator->validateForm('tfa_entry_form', Array, Object) (Line: 593)
Drupal\Core\Form\FormBuilder->processForm('tfa_entry_form', Array, Object) (Line: 325)
Drupal\Core\Form\FormBuilder->buildForm(Object, Object) (Line: 73)
Drupal\Core\Controller\FormController->getContentResult(Object, Object)
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 592)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 121)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 182)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 44)
Drupal\redirect_after_login\RedirectMiddleware->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 704)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

Any advice will be helpful. My logic for decryption and encryption is available here

@cmlara I have looked into your code in the Fork, I believe it wont work as the seed size is different in D7 and D10.

I think my approach is correct.

D7 Seed 64 Chars long 

gIhLBSTBzzDOi8D0LceftNtyFHyZVTttmB2WKzlIvrIXS2KnJcAGaTYe3gxzX33r

D10 268 Chars long  

ZGVmNTAyMDA2YmU2MTljZTFjNjA1YmI5MmQwZjEyM2ZkNmU0M2RhODE0ZGM1NWE0NTQyYTdiNWJjNWE0ZDNhMjAwMTFhNzYxMmU2NGE5MzMxYWU0MWIzNzhlMDg0YTcwMWJiNmQ4MzY0ZWI5MjBiZDE3ZjA5NzZiMmRhNDNlNzNkMzAxYzUzODU3NTFkNDY3ZTE0MDkxOWVjMjk3ZjY0MDE4YTY4NDMyMDIwNmZmYmQyMTAyY2RhOWJjZDQwNTNiN2RmMTZiNzM=

This issue partially stalled out on waiting for tfa_basic to fix [# 3386346], though with recent security fixes we may need to evaluate if that is still a blocker.

Coming to accepted codes I think we can not utlize migrated data as in Drupal 7 Accepted codes are stored using this logic

That does indeed pose an issue, as we now use sha256.
We could still import the previous used codes and it would require a custom plugin that can review older tokens as well as current.

I will note for the TFA included plugins we are working to move away from accepted codes in 📌 Remove usage of alreadyAcceptedCode()/storeAccepedCode() in the TOTP,HOTP, Recovery Code plugins. Active . For the tfa_basic/totp plugin migrations could do similar as we plan to do by assuming the last accepted token is (roughly) time()/30 + Max Window Count

In the code you are just copying seed data from D7 to D10 without processing.

It was written (but never tested) under the assumption that one will use the same encryption on the source and destination. This means that the data should be returned by decryption the same in D7 and D10.

Seed length isn't necessarily a good indicator of functioning correctly, its dependent upon both the secret length, and encryption method (including decryption method key length)

Regarding your error in tfa_migrate:

RangeException: Base32::doDecode() only expects characters in the correct base32 alphabet in ParagonIE\ConstantTime\Base32::doDecode() (line 421 of /var/www/html/vendor/paragonie/constant_time_encoding/src/Base32.php).
ParagonIE\ConstantTime\Bas e32::decodeUpper('') (Line: 81)
ParagonIE\ConstantTime\Encoding::base32DecodeUpper('') (Line: 266)
Drupal\tfa\Plugin\TfaValidation\TfaTotpValidation->validate('931540') (Line: 222)

This appears an empty value is being provided to the base32 decoder, getSeed() will return false if the seed is empty (the Real AES Encrypt plugin has been known to return an empty string rather than throw an exception in its early releases, I can't recall if/when this was fixed.) This also could be a fault in conversion with the data not stored correctly.

Coming to accepted codes I think we can not utlize migrated data as in Drupal 7 Accepted codes are stored using this logic

That does indeed pose an issue, as we now use sha256.
We could still import the previous used codes and it would require a custom plugin that can review older tokens as well as current.

For that we need to store D7 Site Hash in the D10 Site. Got any suggestions on where we can store it ? Normally we would store it in migration related configurations as we will be normally uninstalling migration related modules. But we can't do it here as we need that value later. Should we add a new configuration in TFA module ?

During Migration we can prefix old TFA Accepted codes with legacy word.

Got any suggestions on where we can store it ?

Ideally the 7.x->8.x conversion would of been directly compatible in TfaPluginBase, however we are long past being able to correct that. At this point I'm not sure we can make this TFA responsibility.

it would likely be the plugin provider responsibility to store the old site hashes and to lookup if any are needed. TFA wouldn't need to know that older codes are being queried as that would happen inside the plugins.

During Migration we can prefix old TFA Accepted codes with legacy word.

🐛 storeAcceptedCode()/alreadyAcceptedCode should check token id. Active would be relevant, at the moment the code storage has no support for naming the source of a used validation code. That is assuming we even keep storing validated codes in 2.x. Once we no longer use it internally there is room to question if we should leave it on the base API or not.

I think we can use the below logic to validate the Legacy Accepted Code.

  /**
   * Whether the code has been used in Legacy System.
   *
   * @param string $code
   *   Entered OTP Code.
   * @param string $hash_salt
   *   Legacy Site Hash.
   * @param int $uid
   *   UID of the User.
   *
   * @return bool
   *   Returns wheter the code already used or not.
   */
  protected function alreadyAcceptedCodeInLegacySystem(string $code, string $hash_salt, int $uid): bool {

    $fields = [
      'uid',
      'module',
    ];

    $hash = hash('sha1', $hash_salt . $code);

    $legacy_code = 'legacy_tfa_accepted_code_' . $hash;

    $query = $this->connection->select('users_data', 'ud')
      ->fields('ud', $fields)
      ->condition('name', $legacy_code)
      ->condition('uid', $uid)
      ->execute();

    $result = $query->fetchAssoc();

    if (!empty($result)) {
      return TRUE;
    }

    return FALSE;

  }

We can Just plugin in the above code in the validate function and add OR condition to check legacy and current system accepted codes.

That would (minus the breaches of API by directly querying the user_data table) look roughly correct for what a plugin would need to do to lookup historical codes.

I think I have fixed the issue 🐛 Fix Decoding error after migration Needs review can any one test it and confirm ?