Feed dependencies are not imported in the correct order during batch import

Created on 26 March 2021, over 3 years ago
Updated 30 March 2023, over 1 year ago

Problem/Motivation

Feed dependencies are not importing in the correct order when importing in batches.

In FeedDependencyImportHandler->startBatchImport there is a comment that says

// With a batch the latest task added are run first.
// So the dependencies are added in last.

I am using Drupal 9.1 and maybe this was true but it appears to not be true anymore. The dependencies are always imported after the main feed when importing this way. I imagine this is the same for cron and push imports too.

Steps to reproduce

Add a dependency for a feed and click import via the UI.

Proposed resolution

Remaining tasks

N/A

User interface changes

N/A

API changes

N/A

Data model changes

N/A

🐛 Bug report
Status

Needs review

Version

1.0

Component

Code

Created by

achap 🇦🇺

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • achap 🇦🇺

    I see what you mean looking at the code now. I guess the same would be true for the pushImport feed handler as it uses queue api and would have the same lease times. Does that mean I should remove the changes above for cronImport and pushImport or should it be left as is?

    I'm only using the import (via drush) and startBatchImport methods in my project so haven't tested the other methods.

  • 🇳🇱Netherlands megachriz

    @achap
    cronImport() and pushImport() at least look broken to me. So because of that I think you could leave them out for your patch.

    I have in fact played with the idea of putting feed imports in a specific order on cron run. And I drafted a module for that (which I haven't published yet because its features would overlap with this project). Before the start of the first import it calculates which feeds need to be imported and in what order. That data is written to a state variable. An event subscriber listening to the FeedsEvents::IMPORT_FINISHED event then checks if it should start an other feed import by checking the state variable.

    One issue I ran into though is that all feeds part of the sequence need to get locked. Else the order cannot be guaranteed. Because an other process could start an import for the second feed while the first one is still running. But if a feed is already locked, the event subscriber cannot start a cron import for that.

    An other limitation of the module as I wrote it is that only one sequence of imports can be started at the same time. And if one of the feeds in the sequence has already an import running, starting the import sequence fails.

    But maybe all this should be addressed in a separate issue so this issue can concentrate on the batch import in the UI specifically. ;)

  • First commit to issue fork.
Production build 0.71.5 2024