Highwater condition with unjoined maps skips unprocessed and NEEDS_UPDATE rows

Created on 9 March 2017, over 7 years ago
Updated 30 July 2024, about 1 month ago

Follow-up to #2824267: Highwater condition isn't added (on remote databases) β†’

Problem/Motivation

per ohthehugemanatee in https://www.drupal.org/node/2824267#comment-11946115: β†’

What I don't understand, is how does this work when the map is NOT joinable? If mapJoinable() returns FALSE, the query conditions look like: "(original conditions) AND (above high water)." How do we know which rows have NEEDS_UPDATE set?

For a detailed analysis, see #20 through #23.

Proposed resolution

Do not add the highwater query condition if the map table is not joinable (if the map table in the destination site cannot be joined against the source table in the source site).

Remaining tasks

  1. Review

User interface changes

None.

API changes

None.

Data model changes

None.

πŸ› Bug report
Status

Needs work

Version

11.0 πŸ”₯

Component
MigrationΒ  β†’

Last updated 1 day ago

Created by

πŸ‡ΊπŸ‡ΈUnited States mikeryan Murphysboro, IL, USA

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    The patch has this code:

             if ($this->mapJoinable() && $high_water !== NULL) {
               $conditions->condition($high_water_field, $high_water, '>');
               $condition_added = TRUE;
             }
    

    But then, inside mapJoinable we have this:

        if ($this->batchSize > 0) {
          return FALSE;
        }
    

    So, if I understand correctly, with the solution suggested on the patch, there is no way to use both high water marks and batch size. If you set a batch size, then the water mark conditions will not be executed.

  • πŸ‡«πŸ‡·France Benito25 Lyon

    Patch #58 didn't work for D10.3.

    Added new patch for D10.3.

Production build 0.71.5 2024