How to import a million of nodes? Very slow import.

Created on 29 January 2020, over 5 years ago
Updated 4 June 2025, 27 days ago

I've been dealing with this problem for several days. I need to import around 1,600,000 nodes (one time). Everything is set correctly, but the speed is horrible.

Both on the shared server and on the localhost (WAMP and attempts to set the optimal my.ini file). The import is simply horribly slow and after importing 400,000 nodes is even slower.

I understand that processing in Drupal 8 happens per item, but how to set up / edit the import to make it faster?

Thanks for advice!

🐛 Bug report
Status

Fixed

Component

Feeds Import (feature)

Created by

🇨🇿Czech Republic Search4

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇷🇴Romania oana.hulpoi

    Hi,

    I've created a patch that addresses the concerns discussed here. The patch is available on the related issue #3082278 💬 How to speed up Feeds import Fixed .
    The optimizations include avoiding unnecessary entity loads during hash comparison and adding database indexes for better performance with large feeds.

    Please test and provide feedback, thank you!
    Oana

Production build 0.71.5 2024