Wanted to raise this idea for discussion, but right now, because Search API maintains its index status in a separate table that is outside the QueueWorker workflow, I can't run multiple simultaneous processes to force through a larger site reindex because each drush sapi-i process is consuming the same list in the same order, vs claiming an item from a queue.
I'm wondering if this could/should be explored. Particularly for some of our larger sites with 20k+ nodes, and several hierarchical object relationships that require child objects to be indexed when parents are changed, etc, we often run into issues where it takes longer to process through day-to-day indexing tasks, and being able to parallelize some of that work, it would be very beneficial.
Opening this up for discussion-- obviously search_api is built in more of a batch-focused approach, but wanted to see if anyone else was encountering a similar use case and seeing how/if people are solving this problem in another way.
Active
1.0
Plugins
Not all content is available!
It's likely this issue predates Contrib.social: some issue and comment data are missing.
The Search API Solr module now has support for parallel indexing, see this commit. Nothing Solr-specific in there, either, so if we make this a bit more flexible we could also add this to the Search API module directly.
I'm also interested in this. It looks like someone has created a parallel tracker exactly like that described in #2.