Contrib.social

Inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS error after upgrade

Open on Drupal.org →
Created on 30 September 2023, about 1 year ago

I found a solution to the problem, so I am only posting in case anyone else encounters this situation.

After updating Search API Solr from 4.2.12 to 4.3 I was getting an odd error when I tried to index:

$ drush search-api:reset-tracker && drush search-api:index
>  [error]  Drupal\search_api_solr\SearchApiSolrException while indexing item entity:node/101:en: Solr endpoint http://solr:8983/ bad request (code: 400, body: Exception writing document id 24a5te-solr_index-entity:node/101:en to the index; possible analysis error: cannot change field "tm_X3b_en_title" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, message: Solr HTTP error: OK (400)
> {
>   "responseHeader":{
>     "status":400,
>     "QTime":2},
>   "error":{
>     "metadata":[
>       "error-class","org.apache.solr.common.SolrException",
>       "root-error-class","java.lang.IllegalArgumentException"],
>     "msg":"Exception writing document id 24a5te-solr_index-entity:node/101:en to the index; possible analysis error: cannot change field \"tm_X3b_en_title\" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS",
>     "code":400}}
> ). in Drupal\search_api_solr\SolrConnector\SolrConnectorPluginBase->handleHttpException() (line 1089 of /var/www/html/web/modules/contrib/search_api_solr/src/SolrConnector/SolrConnectorPluginBase.php). 
>  [error]  Drupal\search_api_solr\SearchApiSolrException while indexing item entity:node/147:en: Solr endpoint http://solr:8983/ bad request (code: 400, body: Exception writing document id 24a5te-solr_index-entity:node/147:en to the index; possible analysis error: cannot change field "tm_X3b_en_title" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, message: Solr HTTP error: OK (400)

I tried a few things, and am not sure which of them worked, but here they are:

  • Cleared all data in Solr with this command, this is probably what solved it. Replace mysite with the name of your DDEV instance and solrcore with your Solr core machine name:

    curl -X POST -H 'Content-Type: application/json' --data-binary '{"delete":{"query":"*:*" }}' http://mysite.ddev.site:8983/solr/solrcore/update

    From https://solr.apache.org/guide/solr/latest/indexing-guide/reindexing.html

  • I noticed these values for the "Configured" and "Detected" Server (/admin/config/search/search-api/server/solr_server):

    Configured Solr Version 8.0.0
    Detected Solr Version 8.11.2

    So I updated "Server" > "Connector Workarounds" > "Solr version override" from 8.x to Determine automatically, and they now match.

    Configured Solr Version 8.11.2
    Detected Solr Version 8.11.2

    I re-uploaded the Solr config and restarted DDEV to make it take effect.

  • Reset tracker and re-indexed successfully:

    $ drush search-api:reset-tracker
$ drush search-api:index
📌 Task
Status

Closed: works as designed

Version

4.0

Component

Documentation

Created by

🇩🇰Denmark ressa Copenhagen

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @ressa
  • Comment about 1 year ago →
    🇩🇰Denmark ressa Copenhagen
  • Comment about 1 year ago →
    🇩🇰Denmark ressa Copenhagen
  • Comment about 1 year ago →
    🇩🇰Denmark ressa Copenhagen
  • Comment about 1 year ago →
    🇩🇰Denmark ressa Copenhagen
  • Comment about 1 year ago →
    🇩🇪Germany mkalkbrenner 🇩🇪

    From the release notes:

    In order to support advanced highlighting and other features, the Solr schema provided by Search API Solr 4.3 got some fundamental changes, for example the usage of StandardTokenizer and storeOffsetsWithPositions.
    When you upgrade the module to 4.3.x from 4.2.x and earlier you still can read an existing Solr index and perform searches.
    But if you write (index) data you'll run into errors like cannot change field "xyz" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS.
    There're different ways to deal with the required update. But all require Solr knowledge.
    But the safest way is to delete the existing core (or collection in case of Solr Cloud) and to create a new core (or collection) using a new Solr config-set generated by Search API Solr 4.3.x and to re-index your data afterwards.
    If you want to avoid downtime of your service, you can also create a new core (or collection), clone and adjust the Search API server and index configs and index all your data twice, while users can still search via the old core (or collection). Afterwards you can use Solr's rename or alias capabilities to switch both cores (or collections) and delete the cloned Search API server and index configs.

  • Comment about 1 year ago →
    🇩🇰Denmark ressa Copenhagen

    Thanks for clarifying what the problem and solution is, and updating the Release Note with a section about it today.

  • Comment about 1 year ago →
    🇺🇸United States somebodysysop

    I have a question about creating a new core.

    This is my normal process:

    from /var/solr/data:

    mkdir newCore
    mkdir newCorei/data
    cp -fr oldCore/conf/ newCore

    Then create the core in the Solr dashboard and "Add Core"

    Sounds like the only thing different about this process will be to upload the configuration files from configs.zip to newCore/conf instead of copying them from oldCore.

  • Comment 13 days ago →
    🇺🇸United States bburg Washington D.C.

    Steps to fix the index via drush, with a Pantheon focus.

    1. Get the list of the servers to get IDs

    drush search-api:server-list

    2. Run these commands to clear the Search schema schema from the API.

    drush sapi-sd SERVER_ID
drush sapi-se SERVER_ID
drush solr-reload SERVER_ID

    Note: the search_api_pantheon_admin module must be enabled for this command

    drush search-api:clear
drush search-api-pantheon:force-cleanup SERVER_ID

    Note: Use server ID in step 1
    3. Repost schema and re-enable the server

    drush search-api-pantheon:postSchema SERVER_ID
drush sapi-ena SERVER_ID

    4. Queue all content for re-indexing.

    drush search-api:reset-tracker

    5. Run the reindex.
    drush search-api:index

    Credit goes completely to mattdanger →

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024