Large feeds do not import entirely

I thought I had fixed this issue, see 📌 Missing temporary files in load balanced environments Fixed . /tmp/feeds_http_fetcherXxx is also the old location were the temporary file got stored. The file to import should be stored now at private://feeds/in_progress/[feed id]. If it is still /tmp/feeds_http_fetcherXxx then the most logical cause of that is that you updated from an older version of Feeds while there was still an import task on the queue. If so, unlock the feed in question and retry to start an import. If you cannot unlock the feed because it is not locked, make sure that all feeds tasks in the queue table are removed first. Then you should be able to start fresh with imports and temporary files will be stored at a location that hopefully doesn't cause issues on Pantheon.

Can it be that you updated from 8.x-3.0-beta3 recently? Because the issues was fixed in 8.x-3.0-beta4, though only for new imports, not for unfinished ones that started before the update to 8.x-3.0-beta4 or later.

This issue was fixed and did not appear until now, as I have been using this module for quite some time. The feed in question was set to inactive for a few months, then set to active on 9/20/24 . The upgrade on feeds from 3.0.0-beta4 => 3.0.0-rc2 was performed on 10/16/24.

I am looking at the file location in the active feed log at /feed/XX/log which does show the source path private://feeds/log/XXXXX/source/feeds_http_fetcherXxx, but the DBLog error shows /tmp/feeds_http_fetcherXxx.

I just added a New Feed to admin/content/feeds for this same feed. The source path for the active feed was private://feeds/log/XXXXX/source/feeds_http_fetcherXxx, and resulted in only 400 entries out of 1042 entries. Same error was logged as before. This particular feed does use feeds_fetcher_post module. Let me know if there is something else I should be checking.

This particular feed does use feeds_fetcher_post module

Ah, then that must be it. I see that feeds_fetcher_post does still uses the previous directory style, which thus can cause issues on Pantheon.

Here that code can be seen:
https://git.drupalcode.org/project/feeds_fetcher_post/-/blob/2.x/src/Fee...

If feeds_fetcher_post could override less of the HttpFetcher than it does now, then this issue can also be fixed when that fetcher is used.

Ah, thank you! I have created an issue for the feeds_fetcher_post module: https://www.drupal.org/project/feeds_fetcher_post/issues/3481810 🐛 Large feeds do not import entirely due to missing temp file Active

Okay, then I'm closing this one!