Do not always update when used with track_changes: true

I changed the implementation so only keys that are listed in the fields option of the source plugin are used in the calculation of the hash. That excludes keys like item_id and changed, so it should fix your issue.

Automatically closed - issue fixed for 2 weeks with no activity.

Hi, sorry for getting back to this so late. I've tried your new version a few months ago and retried now. I'm still getting issues however.

It looks like the migrations are not properly processed:
- The content is not created ;
- The leases are created but not removed.

I'm not sure why. Here's what has been working for me since I posted the issue:
- migrate_source_queue:1.0.0
- core patch attached... I found no other way.

Hi Dieter,

Scratch that, the non-released items issue seems to occur when I initially import content with the core patch, then import them again without the patch, or the contrary.

Still, the real issue is that despite your patch, items added in the migration queue are still updated each time, even if they are unchanged.

In Drupal\migrate\Row::rehash(), where I added my Drupal core patch, I dumped the source without and with your commit:

Without:

array(11) {
  ["data"]=>
  array(2) {
    ["url"]=>
    string(76) "REDACTED"
    ["name"]=>
    string(53) "REDACTED"
  }
  ["created"]=>
  string(10) "1720703551"
  ["item_id"]=>
  string(8) "21746920"
  ["url"]=>
  string(76) "REDACTED"
  ["plugin"]=>
  string(5) "queue"
  ["queue_name"]=>
  string(13) "dr_crm.medias"
  ["track_changes"]=>
  bool(true)
  ["fields"]=>
  array(2) {
    [0]=>
    string(3) "url"
    [1]=>
    string(4) "name"
  }
  ["keys"]=>
  array(1) {
    ["url"]=>
    array(1) {
      ["type"]=>
      string(6) "string"
    }
  }
  ["name"]=>
  string(53) "REDACTED"
  ["target_bundles"]=>
  array(0) {
  }
}

With:

array(11) {
["data"]=>
NULL
["created"]=>
NULL
["item_id"]=>
NULL
["url"]=>
string(76) "REDACTED"
["plugin"]=>
NULL
["queue_name"]=>
NULL
["track_changes"]=>
NULL
["fields"]=>
NULL
["keys"]=>
NULL
["name"]=>
string(53) "REDACTED"
["target_bundles"]=>
array(0) {
}
}

In the 2 situations, with my patch it does not matter if there is item_id and created, since they are removed before hashing, so the hash will always stay the same. And the content is not updated each time.

Without my patch and with your commit, the hash should also remain the same, but the content is still updated each time. I guess there's probably too much that you are removing from the row source, that might impact the process in another key area responsible for determining if the update is needed or not, and making the update even if the hash is unchanged. Maybe track_changes?