Delete the feeds_item field from the entity

Created on 14 February 2024, 4 months ago
Updated 25 March 2024, 3 months ago

Problem/Motivation

Once I've created a feed to import some entities the feeds_item field was added to that entity type.

But I don't want that field. This is another left join every time I want to load the entity, so potentially it may reduce my site speed especially when there are tons of requests by multiple users.

I understand that this field is used for internal needs, for example to find previously imported entities. In my case I can live without that feature as my general use case is to import (time after time) some entities from csv file, so it's totally ok to match entities by title.

Also I have a custom entity where all fields are stored in a single table and there are no left joins at all atm.

Proposed resolution

It would be great to have some checkbox inside the feed or feed type to avoid the feeds module adding the feeds_item field to the entity.

Or to make it possible for site admin to remove that field manually from respective entity type configuration.

Thanks in advance.

✨ Feature request
Status

Active

Component

Code

Created by

πŸ‡·πŸ‡ΊRussia i-grou

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @i-grou
  • πŸ‡³πŸ‡±Netherlands MegaChriz

    The feeds_item field is very much coupled to how the Feeds processor works currently so I think that it would take a quite some effort to make it optional.

    Start with creating a custom FeedsProcessor plugin

    You are welcome however to find a solution to use Feeds without using a feeds_item field. For that I would recommend to start with writing a custom FeedsProcessor plugin. Because most of the feeds_item field's usage is tied to how \Drupal\feeds\Feeds\Processor\EntityProcessorBase works (and all FeedsProcessor plugins that extend this class). You could create a duplicate of EntityProcessorBase and then strip out everything that is feeds_item related.

    Some FeedsTarget plugins also expect that the feeds_item field exist. These are the entity reference targets and the book target. We could probably add an extra check for if the field exists. And then skip the action in case it doesn't exist. For example, the entity reference target clears the hash on the feeds_item field in case the referenced entity is not found. It doesn't hurt the basic functionality of Feeds if we should skip that action. The hash gets cleared only to retry importing the referenced entity on a next import. And with no feeds_item field in place, there would be no "next" import of the same content: all data would then be considered new on each import as checks for if something was already imported before would be skipped.

    These FeedsTarget adjustments is what we could focus on in this issue first. I expect these adjustments to be quite small, so that would be some easy wins to resolve in Feeds. I recommend to also add test coverage for it so that these targets can keep working without the feeds_item field in future Feeds releases.

    After having written a custom processor and having updated the FeedsTarget plugins, it's time to test if this works for you. It is possible that you face some other issues along the way, maybe there exists more code that expect the presence of the feeds_item field. When these are resolved, we could consider to add this feature to Feeds. You could also decide to keep it as a custom solution, if you're fine with that. Though with future Feeds releases there is always a chance that it would break. But that essentially applies to all customizations that changes Feeds behavior on a bit deeper level.

    I hope that this give you enough pointers to start working on a solution yourself. :)

  • πŸ‡·πŸ‡ΊRussia i-grou

    Thanks @MegaChriz for your reply and suggestions.

    I think I'll go ahead with Trucie (True Content Import) β†’ module for now as it fully meets my needs.

  • πŸ‡ΊπŸ‡ΈUnited States irinaz

    @i-grou, you might also take a look at https://www.drupal.org/project/feeds_migrate β†’ which is UI on top of Migrate. It is dev module and processors have UI but are not connected, it might cover more use cases when ready.

Production build 0.69.0 2024