Migrate SQL source query isn't ordered

Created on 23 January 2017, almost 8 years ago
Updated 13 April 2023, over 1 year ago

The query that Migrate's SqlBase source class produces doesn't appear to have any order applied.

For example, in my source D6 database, my nodes of type foo start at nid 14614, and run to nid 95771. But when I migrate a single node, it's nid 75396 that comes through.

This is rather confusing!

Feature request
Status

Active

Version

10.1

Component
Migration 

Last updated about 8 hours ago

Created by

🇬🇧United Kingdom joachim

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇬🇧United Kingdom joachim

    > The consensus was that this will not be fixed because it would break highwater system. There is more discussion in the minutes

    Couldn't this be done in a way that doesn't break highwater?

  • Status changed to Active over 1 year ago
  • 🇬🇧United Kingdom joachim

    I see two simple ways to make this compatible with highwater.

    SqlBase does this:

            // Always sort by the high water field, to ensure that the first run
            // (before we have a high water value) also has the results in a
            // consistent order.
            $this->query->orderBy($high_water_field);
    

    So either:

    A. Define that a custom ordering may only be applied in the absence of a highwater. Change the code here to throw an exception if there is already a sort order on the query.

    B. If there is a sort order already on the query, alter the sort order to add the highwater field first. For example, if the sort is on field 'foo', change it so the sort is now 'highwater, foo'.

Production build 0.71.5 2024