Solr server instance reindex in parallel whille other solr responding searches

Created on 18 March 2023, over 1 year ago
Updated 31 March 2023, over 1 year ago

Problem/Motivation

To reduce the impact of losing the indexed content during a full reindexation, after a deployment when a new solr core need to be deployed and specially for sites where the reindexation process takes long.

Steps to reproduce

I would like to have 2 separate search api solr servers, using different solr core instances.

- Solr1 would run in read only mode as default and would respond to the searches of the site.
- Sol2 would run a reindexation of the content with the latest version of the core.

The solr2 shouldn´t receive any search request until the reindexation process it´s finished.
Once the reindexation is finish, we would swap or drop the solr1 and leave only solr2 to start receiving search requests.

Proposed resolution

I don´t know if this it´s possible using the
drush sapi-sis command to mark the solr1 as default server for searches and that would avoid sending search request to the solr2.
Also was thinking about the possibility to use an override on the settings of the search api, from a backend node, and trigger the reindexation from there, but I'm not sure if that could create some issues with drupal cache, if the frontends are pointing to the other solr server.

Is this technically possible with the search_api module configuration for solr, or any similar approach for that ?

Any hint or idea is welcome.

Thanks in advance, for the support.

Remaining tasks

💬 Support request
Status

Closed: duplicate

Version

4.2

Component

Miscellaneous

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @acarus
  • 🇦🇹Austria drunken monkey Vienna, Austria

    I can think of a lot of approaches that might work, but unfortunately none which would work without some custom code. So, I hope that’s OK for you.
    Probably the simplest of those would be to implement an event listener for \Solarium\Core\Event\PreExecuteRequest events and change the request URL according to the type of request: the old core for searches, the new one for all other operations. Then just queue all items for re-indexing and re-index them. Once that is finished, you can remove/disable your event listener and then (for instance) use the Solr SWAP command to swap the place of the two cores – or just edit the Search API server settings to point the server to the new core. Then, delete the old core.

  • Hi, and thanks for your reply, that already show me the limittions on the issue, that I had no clear idea.

    Unfortunately, Implementing custom code is not an option, at least for the moment.
    So in that case, avoid zero index lost, is not an option for the moment in that case.

    But about reducing the lmpact as much as possible, how would impact having the second core being indexed to the site?
    Would the searches retrieve content from both cores? and having the risk of showing duplicated content ?

    In the other hand, what it´s the safest approach, or the recomendation on drupal 9 to handle the reindexation of a site after the update of the solr core to the new version generated.
    Just clear all the index and reindex ?
    Is there a way to minize the impact or update only content affected by the new configuration version of solr core, so there is no need to clear the whole indexed content ?

    Thanks.
    regards.

  • 🇦🇹Austria drunken monkey Vienna, Austria

    Would the searches retrieve content from both cores? and having the risk of showing duplicated content ?

    No, that will never happen. A search will always only go to a single core, unless you do some very complicated setup to achieve the opposite.
    So, if you manage to index to a new core while still searching from the old one, there’s no risk of duplicates from that.

    For the rest, I think I’m gonna move this issue to the Search API Solr issue queue, as the experts are sitting there.

  • Status changed to Closed: duplicate over 1 year ago
Production build 0.71.5 2024