To many unnecessary requests to Solr 7

Created on 10 December 2021, about 3 years ago
Updated 11 July 2023, over 1 year ago

Problem/Motivation

This issue is limited to Apachesolr search ecosystem (search API is out of scope).

After switching to Solr 7 according to the https://docs.acquia.com/acquia-search/modules/, it was discovered that autosuggest requests take about 10 times more time to proceed. Milliseconds in Solr 3 and 4, and from 2 to 9 seconds for Solr 7.

Further investigation showed that the root cause is the architecture of the new Acquia Connector module.
Inside we have a submodule AcuaiSearch and AcquiSearchSolrApi class and getIndexes() method.

This method is being called multiple times ( while checking the response by isValidResponse method, while building authCookie by createAuthCookie method, etc), and the request takes up to 1 second to complete (based on the location of the initial request).

This is applicable also for common searches. So basically, for any request to Solr we have one extra request before, and one extra request after.

Stack traces
This is just a building auth cookie

common.inc:800, drupal_http_request()
AcquiaSearchSolrApi.php:147, AcquiaSearchSolrApi->getIndexes()
AcquiaSearchSolrApi.php:179, AcquiaSearchSolrApi->getPreferredIndexService()
AcquiaSearchSolrService.php:195, AcquiaSearchSolrService->buildAuthString()
AcquiaSearchSolrService.php:173, AcquiaSearchSolrService->createAuthCookie()
AcquiaSearchSolrService.php:76, AcquiaSearchSolrService->prepareRequest()
AcquiaSearchSolrService.php:31, AcquiaSearchSolrService->_sendRawGet()
Drupal_Apache_Solr_Service.php:945, AcquiaSearchSolrService->search()
Solr_Base_Query.php:701, SolrBaseQuery->search()
apachesolr_autocomplete.module:346, apachesolr_autocomplete_suggest()
apachesolr_autocomplete.module:259, apachesolr_autocomplete_suggest_word_completion()
apachesolr_autocomplete.module:114, apachesolr_autocomplete_callback()
menu.inc:527, menu_execute_active_handler()
index.php:21, {main}()

this is the desired query of suggestions

common.inc:800, drupal_http_request()
Drupal_Apache_Solr_Service.php:557, AcquiaSearchSolrService->_makeHttpRequest()
AcquiaSearchSolrService.php:32, AcquiaSearchSolrService->_sendRawGet()
Drupal_Apache_Solr_Service.php:945, AcquiaSearchSolrService->search()
Solr_Base_Query.php:701, SolrBaseQuery->search()
apachesolr_autocomplete.module:346, apachesolr_autocomplete_suggest()
apachesolr_autocomplete.module:259, apachesolr_autocomplete_suggest_word_completion()
apachesolr_autocomplete.module:114, apachesolr_autocomplete_callback()
menu.inc:527, menu_execute_active_handler()
index.php:21, {main}()

This is validation of response

common.inc:800, drupal_http_request()
AcquiaSearchSolrApi.php:147, AcquiaSearchSolrApi->getIndexes()
AcquiaSearchSolrService.php:296, AcquiaSearchSolrService->isValidResponse()
AcquiaSearchSolrService.php:141, AcquiaSearchSolrService->authenticateResponse()
AcquiaSearchSolrService.php:35, AcquiaSearchSolrService->_sendRawGet()
Drupal_Apache_Solr_Service.php:945, AcquiaSearchSolrService->search()
Solr_Base_Query.php:701, SolrBaseQuery->search()
apachesolr_autocomplete.module:346, apachesolr_autocomplete_suggest()
apachesolr_autocomplete.module:259, apachesolr_autocomplete_suggest_word_completion()
apachesolr_autocomplete.module:114, apachesolr_autocomplete_callback()
menu.inc:527, menu_execute_active_handler()
index.php:21, {main}()

Steps to reproduce

Perform autosuggest search (type something in search field) with spellchecking feature enabled.

Proposed resolution

Up to now, there are two options visible:

  1. Implement static cache inside getIndexes function. The easiest solution, reduce the number of request from 9 to 4 ( for autosuggest query with spellchecker option enabled
  2. Implement long-term cache of Solr 7 indexes ( might be complex variable) during setup of new Solr 7 instance. Might be the right way to go, but might require significant changes.

Remaining tasks

Gathering feedback from the community, continue investigation.

πŸ› Bug report
Status

Fixed

Version

4.0

Component

Code

Created by

πŸ‡΅πŸ‡±Poland driverok

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024