How to speed up Feeds import

Created on 19 September 2019, almost 6 years ago
Updated 4 June 2025, 27 days ago

We have a server 4 x CPU core, RAM 30Gb

php.ini
max_execution_time = 10000
max_input_vars = 10000
memory_limit = 19200M
post_max_size = 500M

import configuration done here 💬 feeds_process_limit Fixed line_limit: 10000
but imports are very slow and use 10% of RAM

How to speed up Feeds import?

💬 Support request
Status

Fixed

Version

3.0

Component

Code

Created by

🇷🇺Russia mailfox

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇷🇴Romania oana.hulpoi

    Hi,

    I created a patch for that addresses performance issues with large feeds by optimizing hash comparison and reducing unnecessary entity loads.

    The key changes are:

    1. Direct hash retrieval: Added getStoredHash() method to avoid loading full entities just for hash comparison - significant improvement when processing feeds with many existing entities.
    2. Cached mapping serialization: Eliminated repeated serialization of mapping configuration during processing.
    3. Optimized batch processing: Increased clear operation batch size from 10 to 100 entities.
    4. Database index: Added performance index on (entity_id, feeds_item_target_id, feeds_item_hash) for faster hash lookups.

    Looking forward to your feedback!
    Oana

Production build 0.71.5 2024