Contrib.social

Feeds import randomly unpublishes hundreds of nodes (external URL)

Open on Drupal.org →
Created on 25 February 2022, about 3 years ago
Updated 25 January 2025, 3 months ago

Problem/Motivation

So my site has a feed that runs 15 minutes. Unfortunately, we are downloading this feed from a URL every time...and the feed is pretty large (about 8MB). We set this up so Feeds unpublishes previously imported items that are no longer in the feed.

The issue is, about once every few weeks, Drupal seems to unpublish hundreds of these at a time. Once in a while it's only a few...but lately we have had issues with a large amount getting unpublished. I figured this was an issue with our vendor (who supplies the URL we fetch from). I assumed that perhaps a huge chunk of content went missing from the feed at some point, and then when the import ran, Drupal noticed they were missing from the feed so it unpublished all those nodes. However, the vendor is denying that this ever happened.

Let's say that the feed never went down, and all content in the feed was present for each import. Is it possible that Feeds is unpublishing these items accidently? Perhaps Drupal had an issue downloading the 8MB file? I've been trying to debug this but I haven't been able to figure out why this has been happening. We have the timeout set to 30 seconds so I don't think there is an issue downloading the file.

This has been happening fairly often for us. When it does, we need to republish all the content that was unpublished. Could the issue here be with that 8MB file? I just feel that if there was an issue downloading that file, the import would fail and it would attempt to run again during the next cron run. We have a few other feeds that do not rely on an external URL, and those run fine. I'm just not sure if this is a bug within feeds, or if the vendor supplied URL is the problem.

It seems to me, the only way that feeds would ever unpublish anything is if we set the 'Previously imported items' setting to 'unpublish' if they are no longer in the feed. Is this correct? Or is there any other possible way that feeds could be unpublishing content?

🐛 Bug report
Status

Active

Version

3.0

Component

Code

Created by

tlo405

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024